Another Sputnik, it was called last week: the latest OECD results were released, and Shanghai schools topped the list, with the US far down the ladder.    This is disappointing, to say the least, and we in the US should indeed be deeply concerned.

But let’s be clear about what we should be concerned.   Readers of the New York Times article, for instance, (and probably that source is the most common source), might not have the opportunity to recognize and appreciate what is really being tested in the PISA until the very last sentence.

I fear NY Times readers might read carefully only through to the quote from Secretary Duncan: ““The United States came in 23rd or 24th in most subjects. We can quibble, or we can face the brutal truth that we’re being out-educated.”  In doing so, they might think that PISA is a conventional bubble test of “basic skills,” and that what Duncan is suggesting we take from it is that we need more NCLB type teaching and basic skill development, because unfortunately that is what Duncan department of Education has become known for: NLCB on steroids.

(I don’t think this is an entirely fair characterization of Duncan’s leadership and vision, but it is what has become the connotative representation of his administration thus far.)

This would be wrong. In fact, the PISA results suggest quite the opposite: we need to break away from the curricular narrowing effects of conventional standardized testing, basic skills emphasis, and rote memorization, and unleash in our schools a revolution of applied problem-solving to real-world situations.

In the very last sentence of the article,  the Times begins, and only barely, to capture what is most important and unique about PISA testing:

“While that’s important, for me the real significance of these results is that they refute the commonly held hypothesis that China just produces rote learning.  Large fractions of these students demonstrate their ability to extrapolate from what they know and apply their knowledge very creatively in novel situations,” Schleicher said.

What the PISA assesses, and I think it is a “test worth teaching to” and a test worthy of significance, is applied thinking and problem-solving.

Here is Tony Wagner on PISA:

the OECD leaders were concerned about the extent to which these subject-content skills translate into the kinds of skills adults need in life. … The goal was to measure what they call cross-curricular competencies– that is to directly assess life competencies that apply across different areas of the school curriculum.

The assessment measured students’ problem-solving abilities in three areas:

  • Making decisions under constraints.
  • Evaluating and Designing Systems for a particular situation.
  • Trouble-shooting a malfunctioning device or system based on a set of symptoms.

It is 15 year olds which are taking the PISA, and these results require we rethink how our middle school and early high school students are learning in our schools.   Far, far too many of them are reading textbooks, listening to lectures, and then being asked to complete worksheets or repeat back the content they have learned, and apply simple formulas to rote scenarios.    These experiences might, in some cases, lift their scores on conventional bubble tests, but we need to do better by them: we need to ask them to tackle complex, difficult problems, and think through how to solve them in multiple ways, or with applied logic.

Fascinatingly, the educational vanguard in China is recognizing this also, and is swiftly upending its hide-bound traditionalism in educational practices and in its more forward looking city, Shanghai, is seemingly revolutionizing education to promote effective thinking over mindless memorization and regurgitation.   Yong Zhao has been warning for half a decade now that even as American education is prioritizing basic skills mastery, China, or some sectors there, is transforming itself into true education for innovation.

I don’t begin to know how exactly to explain Shanghai’s sudden and startling success beyond what I have read from Zhao.  We may learn more still about how its students were selected for testing, and the NY Times makes the point that its students were seemingly told before test-taking that the nation’s pride rested on their shoulders, and they responded in a way akin to Olympic athletes do.   It is unclear whether American students heard this message, and perhaps it is unclear whether American students would respond the same way.

A strong shout-out goes here to Peter Pappas, who has a terrific blog entitled Copy/Paste: Dedicated to Relinquishing Responsibility to Learning to Students, and whose post, What PISA Test Really Tells us about American Students, has greatly influenced this post.  Pappas reminds us to keep our focus on the poor American performance and the reasons why.

We spend a lot of time in school getting students to learn sequential information – timelines, progressions, life cycle of a moth, steps for how to. Typically the teacher teaches the student the sequence and the student correctly identifies the sequence for teacher on the test. Thus we treat a sequence as a ordered collection of facts to be learned, not as a thinking process for students to use.  This memorization reduces the student’s “mastery” of the chronology to lower order thinking.

Pappas has on his blog post a nicely laid out example of a sample PISA test question, which nicely demonstrates the kind of thinking we need more of in our students and schools.

PISA sample test questions are available online, and I have the ambition, as yet unrealized,  to do things with them in my school.  Could we ask students to do them in a administration-driven problem of the week style?  Could we share them with our teachers and ask them to use them as models for their own test-making?  What if I just administered a sample test to my own 15 year olds every year, and worked to improve our school’s performance.    Or do a set with my 12 year olds, my 15 year olds, and my 18 year olds, and trace cohort performance, and assess what interventions help?

Alternatively, we can ask our educators to sit down with PISA sample questions, study them closely, and work with backward design techniques to think more thoroughly: what kind of teaching in our classrooms will really support the kind of learning that produces success on these tests.   I say this knowing that teaching to the test is much and justly maligned, but it doesn’t have to be maligned when the test itself assesses effectively the kind of thinking skills we really want for our students.

As I understand it, US schools cannot sign up to take the PISA test in any official way, though international curriculum schools in the US can enroll in the ISA, a PISA formatted test for international schools around the world.   I am envious of this opportunity for international schools, and feel a bit of resentment it isn’t more widely available.

OECD PISA is an important organization, and I think its Head, Andreas Schleicher, deserves important recognition as a leading thinker and actor in the world of education advancement that this blog seeks to be a part of.  Schleicher can be followed on Twitter here.

An important connecting tissue now exists between the fine work of OECD/PISA/Schleicher and work I am involved in here in the US, advancing the College and Work Readiness Assessment; OECD has recently adopted the CWRA-College Learning Assessment test (CLA), as its tool for evaluating the comparative effectiveness of European universities.   As regular readers here know, I think one of the best ways we can embrace seriously the problem of preparing students for success in the realm that PISA measures is to administer the CWRA and take the data we receive seriously for the work of improving our schools!