–
Fairfax County, which is headed by one of the nation’s finest superintendents, Jack Dale, has recently posted up a comprehensive website about PISA/OECD testing in their district, and it is fascinating and impressive in many ways.
The video above, produced by America Achieves, which is active in the OECD testing initiative in the US, offers a nice overview of the Fairfax program, and demonstrates the seriousness of the way in which they in the district intend to use and apply the results to improve learning.
PISA testing, newly available for individual schools and districts under the name of OECD test, was recently praised by Tom Friedman in his column, and has long been admired by thoughtful and informed educators such as Tony Wagner. Simply put, it is a better kind of test, much more designed to evaluate students abilities to use what they have learned to tackle new and complex problems, evaluating their application and analytic skills. Below is a video about the PISA test.
–
But just because a test is better designed for assessing student skills and proficiencies doesn’t mean necessarily that it is especially effective in providing results and information which inform practice. PISA/OECD, however, and in contrast to many others, has made this a priority, and Fairfax County is an exemplar.
I want to offer a number of commendations to and draw a few lessons from Fairfax County, America Achieves and PISA OECD based on what has been posted on the County’s website here.
1. Huge Kudos for the transparency of the County to share so boldly and generously everything about the OECD testing program. When you go to the site and click on the tab, School reports, you will find the full, comprehensive, detailed school testing result reports– un-redacted. Now there is no student reporting here, privacy is honored, but the willingness to show and share is a model for every other school district and school.
2. The breadth and depth of the OECD testing report, entitled “How Your School Compares Internationally,” as can be seen in these examples, is staggering. This is a 161 page document, with scores of boxes, figures, and tables. Schools and districts are informed deeply about the effectiveness of their program and offered thoughtful, thorough analysis regarding opportunities for improvement. For private school educators reading this post, there are even included sections comparing your school’s results to private school sampling across the US. There are dozens of examples of useful sections– in one part of the report, where the impact of soci0-economics are assessed in your school and in national results, the report shares lessons from PISA research on how the most successful school systems in this regard support disadvantaged students.
3. The people at PISA are clearly students of and practitioners in the developing field of graphically communicating information– and they have generated many interesting graphs in these reports. See the slides below, downloaded from the Fairfax County Site, which showcases some fine examples.
–
4. Having a high quality test and having a high quality report are the first two critical steps for employing external testing results– but the third is as well. How do you then use the report internally? Fairfax, as demonstrated on their website, is doing extraordinary work here as well, work that I think can be highly informative to the practice of others. They offer this graphic for one way to communicate what they are doing.
—
On the Fairfax website, under the tab “School reports,” an eight week action plan is displayed, explaining how the leadership team will study the report in sections. Even included are materials on how each weeks conversations will be conducted, with downloadable powerpoints. The plan is embedded here below:
–
Week 8, which has yet to occur on this schedule, is ultimately the most interesting– and I’d imagine it might only set the stage for a next phase, perhaps over the summer, of work taking the suggested actions from the conversation and mold them into an educational action plan. Recognizing the outstanding transparency of Dale’s leadership, perhaps we can look forward to such a plan being published here in the months to come.
5. Going into the actual analysis, what is so interesting and valuable is that OECD is helping schools and districts unpack, deeply, their results and find meaningful insights. The report doesn’t just include student test results, but student survey data on topics such as their sense of self-efficacy, their reports of school disciplinary environments, the relationships they have with teachers, and their instrumental motivations, and more– broken out by student performance levels. One example of many of what can be gleaned, quoting the report:
While 9 out of 10 top-performing students report that the teachers rarely have to wait a long time for the students to quiet down, only 7 out of 10 low-performing students have a similarly positive experience in their English classes.
More about OECD testing in the US is available at America Achieves, here.
Again, my praise, admiration, and appreciation go to Fairfax County for making so available for sharing these materials.
21k12 
April 29, 2013 at 3:23 pm
LOVE this, Jonathan – thanks for your post!
Two thoughts resonate strongly with me:
1) “… a better kind of test, much more designed to evaluate students’ abilities to use what they have learned to tackle new and complex problems, evaluating their application and analytic skills.” The ongoing shift in k-12 education toward more rigorous, quantitative analysis can be powerful – when we are measuring that which we value. PISA represents a meaningful step in the direction of aligning quantitative methods with the sorts of skills that students will need to develop in order to be engaged, involved citizens in our democracy… and productive citizens who contribute to our economy, too.
2) “effective in providing results and information which inform practice” – I appreciate that PISA works to bridge the divide between producing data and using data effectively as part of an ongoing professional growth process… too often, it seems, there’s a big gap between those who produce and analyze data and consumers (teachers) for whom data could be powerful – it it were presented in ways that appreciate and connect effectively to the work that teachers do in classrooms each day…
… well done, Jonathan! I appreciate you for continuing to publish thoughtful work that INSPIRES me to think more deeply about teaching and learning.
April 30, 2013 at 5:01 pm
Jonathan, this is a wonderful summary and a great example of how Fairfax County School District has stayed on top. Having only your blog as background about the PISA-OECD, I’m wondering if you have an idea about how it compares to the MAP (Measures of Academic Progress). It sounds similar in some ways, especially with regard to the use of data to inform instruction.
April 30, 2013 at 5:07 pm
Hi Donna:
I’m very familiar with MAP, and have administered it for several years. It is funny because I think of MAP and OECD testing as very different, though they have a few commonalities. One common piece is that both are very committed and serious to helping schools/districts use the results meaningfully, more so than most other tests in my experience. But, that said, in my perception, MAP is more about evaluating very discrete “basic skills,” breaking them down into a laddered set of many individual pieces. MAP is also administered multiple times in a school year and many in a school life, and is for individually targeting student performance to differentiate instruction and track growth. PISA is holistic application of understanding, anything but individual discrete tasks, and is given once in a student’s lifetime, or not even, because big schools would only do a sampling, and so is not at all for individual student evaluation but for institutional review.