21k12

“Not everything that counts can be counted, and not everything that can be counted counts.” Albert Einstein

I greatly admire Alfie Kohn.  I have read him since the eighties; I have only heard him speak once, over a decade ago, but it was unforgettable.   I am grateful for his piece in a recent edweek, “Turning Children into Data: A Skeptic’s Guide to Assessment Programs.”

And yet, I wish for more.  The quote Kohn provides from Einstein atop his piece (provided atop my piece) doesn’t call for ignoring all data; it declares that there is indeed data that does and should count, but that we need to be choosy and skeptical about what data we use and how we use it.   Unlike Einstein’s quote, in Kohn’s piece there is not a single thing which he suggests should count; it only about what we shouldn’t count.  As a result, it strikes me he has misappropriated Einstein for his purpose.

As regular readers here know, I think we need to be serious about setting our goals as an educational institution, and then identify what we  things we should count to measure our progress, hold ourselves accountable for success, and most importantly, guide our continuous improvement.   I think too that what we choose to measure sends signals, to teachers, to parents, and to students, about what is most important, and we should be intentional about the signals we send.  

If we measure nothing, which is what Kohn leaves us inferring we should do, we have the potential result, in a world where measurement is so prevalent,of inadvertently communicating that nothing is important, or allowing others who aren’t educators to decide what should be measured.   The measurement is the message.

Kohn’s tone strikes me as strident, and, I have to sadly say, pretty demeaning and nasty to those who does indeed wish to measure significant, meaningful educational outcomes, such as “critical thinking “and “authentic outcomes,” and which seek to promote teaching which is” differentiated,” ” collaborative,” or focused on” facilitating learning,” not just “facilitating teaching.”

Now I do realize that Kohn’s (and similarly Diane Ravitch’s) anger and vehemence is well founded: we are living in an era where measurements, accountability, and data driven decisionmaking in education has crested to a far higher level than perhaps ever before, and it is distorting and devaluing quality education in many ways.  But my suggestion to both is that we best respond to the overly aggressive data driven decision-making movement by not dismissing and impugning all data uses, but rather explaining how we can and should use data meaningfully.   That response I think is the more effective one, rather than the snide sneering that this piece, to me, conveys, towards all data collection for educational improvement.

Kohn offers six questions for interrogating potential measurements; he does not offer to any of these single questions a single positive example of an assessment that might meet his standards.   I am going to try to answer each question with my thoughts about how three measurements I am advocating for do meet his criteria: the CWRA (College and Work Readiness Assessment), HSSSE (High School Survey of Student Engagement), and MAP (Measurement of Academic Progress).

1. What is its basic conception of assessment?

Kohn says that “we ought to focus upon the actual learning students do over time; we need to concentrate on deep learning that consists of more than practicing skills and memorizing facts.”   I agree entirely with that assertion, but I think that CWRA is an authentic assessment where students do demonstrate their accomplishment of deep learning by their ability to write effectively about complex topics, analyze evidence, and solve complex problems.   HSSSE doesn’t measure learning per se, but its basic conception is that student perspective matters and we should seek to collect and make meaningful that perspective.  MAP is more conventional, and I am sure there are aspects of it Kohn would criticize, but it is very much about measuring students’ learning over time; it tests three times a year, and gives immediate feedback on student progress.

2. What is its goal?

“Ask not only what the program does but why it exists.” My trio, though, have outstanding goals, and they are not just to “raise student achievement.”   CWRA’s goals are both to measure a school’s value-add to critical thinking and writing effectiveness, so a school can know how effective it is in this important project, and also to stimulate inside a school a greater attention to assessment by performance task and applied, real-world situated, problem-solving, drawing upon Wiggins’ advice that the we need to begin with the end in mind.

HSSSE’s goal is take students own view of education seriously in school improvement, and that improving student engagement is an absolutely critical step toward improving learning.  MAP’s goal is exactly what Kohn does call for: it is not about generating school-wide averages but giving teachers immediate feedback about individual student progress.

3. Does it reduce everything to numbers?

In the case of CWRA and MAP, pretty much yes.  HSSSE does collect student narrative comments and provides them  to schools, but the other two reduce results primarily to numbers.   Kohn tells us that “most assessment systems are based on an outdated behavioral model that assumes nearly everything can–and should, be quantified.”   I don’t think that is fair.  These three tools have identified some discrete things they believe both matter and can be quantified, and offer vehicle to do so; they don’t suggest that everything can or should be.

4. Is it about “doing to” or “working with?”

Kohn is right, entirely right, to worry that some or many contemporary assessments are being done unto students and are perceived by them as punishments or misery.  But when he says students’ autonomy should be respected, I’d point him right to HSSSE, which meets exactly this standard: let’s be serious about the student voice in crafting our learning environments so that we are indeed better “working with” our students.   As for CWRA, please see my students discuss, without prompting or guidance, their own view of the test: they do think it is meaningful and constructive.  MAP is a test designed to be responsive to individual students; whereas most tests are either way too boringly easy for some students, and devastatingly too difficult to others, MAP adjusts after each question to continually offer students appropriately challenging questions.

5. Is its priority to support kids’ interest?

Again, this is a meaningful question; Kohn is right to say we should make this a priority.   HSSSE is exactly designed to generate data for schools to better support kids’ interests.   My students told me that they found CWRA to be very interesting and stimulated them to want to improve their thinking skills to tackle future problems they are interested in.    MAP– well I don’t know if it does, but I know its responsiveness is intended to allow it to be more crafted to students’ proficiency level which will be more interesting and meaningful to them.

6. Does it avoid excessive assessment?

Like Kohn, I think students should spend far, far more time learning and doing and learning by doing than testing.  CWRA and HSSSE take only an hour or two a year.   MAP is more time; in order to gauge effectively individual student growth over time, it does take place three times a year, and in doing so, fails this Kohn test.  In our use of MAP at St. Gregory, we will have to very carefully evaluate whether we are allocating to it too much time, and we are prepared to turn away from it if we find we are.

We can and should use the right kind of data, in the right kinds of ways, to improve our schools; we must also use many other tools to improve schools which have nothing to do with data.

At St. Gregory we are dramatically enlarging teacher collaboration and collaborative learning time; we are providing students more sleep time; we are giving them more leadership opportunities and more project based learning experiences, in many of which they get to choose what topics they will pursue; and we are strengthening student-to-student and student teacher relationships, none of which have anything to do with use of data.

But we are also, newly using CWRA, HSSSE, and MAP to improve our students’ learning, and I would urge Alfie Kohn, whom I have long admired, to respect that schools certainly can use data tools in meaningful, respectful ways which promote school improvement and do not turn children into data.