21k12

(Disclosure: I wrote this following piece for new client ProExam, on whose blog it originally appeared.  People often inquire what I’m up; this is one of my newer and larger projects, consulting to ProExam on the development and promotion of their forthcoming tool.  

This– better school and individual level assessment and measurement, primarily for formative purposes, of noncognitive skill and character development — is something I’ve been enthusiastic about for years, of course.   Readers here may remember my enthusiasm dating back to 2013 for the Mission Skills Assessment, built by ETS for INDEX; I wrote the MSA user’s guide and toolkit in 2014.   Rich Roberts, Ph.D. and Jeremy Burrus, Ph.D., the two ETS scientist-researchers who originally designed the MSA, are now at ProExam, and have recently developed the next-generation tool described below.  Some of the evidence basis and research underpinning this new product can be found in a paper I co-authored with Dr. Roberts for the Asia Society, “A Rosetta Stone for NonCognitive Skills.”)

I’m assisting with recruiting schools interested, at no expense, in piloting the tool this winter/spring; contact me if you are interested at jonathanemartin@gmail.com)

Social Emotional Learning and Noncognitive Character Strengths Matter…and How We Measure Them is the Key to Their Improvement

Perhaps the greatest consensus in K-12 learning today centers upon the critical importance of student social and emotional learning and the development of their noncognitive character strengths—their skills for success in school and life.

This is not news to teachers.  Ask a preschool assistant teacher or ask an AP Physics teacher and you’ll find resounding, even impassioned agreement: dependability, persistence, ambition, curiosity, and getting along with others matter as much, or very often much more, than cognitive ability.  Education leaders have similarly embraced this understanding, with ASCD making the “whole child” its signature slogan and state and district leaders shifting the emphasis of schooling to skills and life success.

In the past decade or so, the common sense point of view of teachers in the field and educational leaders has been emphatically endorsed by researchers, social scientists, and think tanks, including Nobel Prize-winning economist James Heckman, New York Times journalist Paul Tough, MacArthur “genius” prize winner Angela Duckworth, the Hewlett Foundation, the RAND Corporation, the National Research Council, the Brookings Institute, and the New America Foundation, just to name a few.

As the educational field works to strengthen its effectiveness in developing and implementing social and emotional curricula, in planning and guiding ongoing improvement in this arena and holding themselves accountable therein, and in providing meaningful feedback to students in their growth and proficiency, an enormous gap is being increasingly perceived by nearly all involved.  We lack effective assessment and measurement of social and emotional learning and noncognitive character strengths: the skills of success.

Nearly every individual and organization listed above can be cited to this effect: We lack the assessments we need.   (more…)

–

Perhaps this is a bit too much inside baseball, (and I know it makes me out to be a test-geek), but I want to write briefly today to cheer the news that the OECD Test for Schools (based on PISA), abbreviated to TFS, has been taken up by NorthWest Evaluation Association, coming over from its previous administrator, CTB-McGraw Hill.

(Note: I have no current or past affiliation with NWEA nor with OECD).

NWEA is best know for its MAP testing, the Measures of Academic Progress computer adaptive test.  More about MAP below (after jump):

Regular readers know of my enthusiasm for PISA testing in schools; in 2013-14 I researched and wrote a 64 page user’s guide and toolkit on the PISA TFS, (for EdLeader21 funded by the Hewlett Foundation) sharing how it is being and how it can be valuably used at the school level.
It was fascinating to observe and learn how unique and exciting this tool is: PISA questions are richer, more challenging, more open-ended, far more demanding of higher order thinking than those of most other tests. (Sample questions can be seen in slides above or in this excellent compendium)

The TFS 160 page school results report is a thing of beauty: colorful, studded with scores of charts and tables, enabling comparison of your students to norms globally and US.  There’s information about your school’s performance compared to like-socioeconomic schools as well.  There’s also a bunch of data about how students perceptions of their schooling– reading practices, teacher relations, instrumental motivation– can be mapped onto their performance, giving schools far more insight into opportunities for improving student performance than most other tests or tools.

The report is also studded with great little nuggets about evidence-based educational practices from around the world.  Disappointed by your school’s reading results?  Read about how reading is taught in Finland.  And so on.  Administering the TFS is an awesome window into comparative educational practice research and opens up perspective and understanding for school-leaders brilliantly.

I often tell people the story of Fairfax County (VA), which enrolled ten of its high schools in the PISA TFS pilot, at no cost, and was so delighted by the value of the reports they received that they then expanded their involvement, enrolling 27 high schools in the test at $12,000 each!

But, as successful as the TFS has been to date, it has been limited a bit by its previous high price point (about $12K) and its less-than-terrific management by the for-profit corporate behemoth that is CTB-McGraw Hill.

So how great it is to learn that its being taken up by the not-for-profit (501(c)3) NWEA, which is well regarded nationally (and in my own personal experience as a school-leader) for its skillful management of the MAP (more about below).   MAP has long been administered online, and now TFS will also be administered.  Because of the greater economic efficiency of online testing, NWEA will be slashing the price point for the TFS, down to about $6500 (and only $5000 this pilot year though June), and it is my recommendation that every high school strongly consider adding the PISA TFS on an annual basis– or perhaps every other year or third year.  Do it every three years and your amortized price is about $2k a year– worth it.  (more…)

Think Through Math, an online math support, personalized learning, and tutoring platform, commissioned this ebook, Lead by Example,  for their audience of district and school leaders and administrators.    You can find it here for free download.

The book’s core argument is that we are all 21st century learners, (or we all need to be), and that we can most effectively lead our schools through the necessary transformation our fast-changing times demand by better practicing what we preach to our students.

When we ask our students to show their work, to learn from their mistakes, to state their thesis, to play well with others, to share, to ask more questions, to be creative, and to revise their work,  do we stop to ask ourselves how well we are doing the same?  Do we consider the example we are setting?  How might we better lead by example?

In the eb0ok, each short lesson is illustrated with concrete examples of educational leaders “leading by example” in their everyday practice.  I’ve also tried to connect each practice to Marzano’s meta-analysis of evidence-based leadership strategies, explained in his book, School Leadership that Works.

Some of the leaders referenced in this e-book are Pam Moran, Eric Sheninger, Chris Lehmann, Eric Juli, Brett Jacobsen, Mark Crotty, and Sue Szachowicz.   I thank them for the many ways they inspire me and exemplify leadership by example.

Pleased to share this recent project, commissioned by Blackbaud and Whipple Hill.

Assessment has long been a focal point for me and my work, but in the last two years, I’ve also been increasingly engaged with questions of how schools and educators can strengthen grading as a key component of a broader assessment philosophy and strategy.  It’s been a pleasure to present and consult to several faculties and schools on this topic recently, and I’m delighted to have had this opportunity to write up some of my thoughts in this new short publication.

Although this free 36 page illustrated ebook, which can be downloaded over at Blackbaud K-12, is written particularly for educators in online learning environments, many of the concepts and strategies are nevertheless pertinent and applicable to conventional classrooms as well.

Many thanks to the many online learning experts who provided very valuable input to this project:

Enjoy.

SSATB just posted this new “special report,” which I researched and wrote most of this past spring.   It profiles new (and relatively new) innovative educational models which are becoming more significant alternatives in the landscape of educational choice, and includes interviews with leaders of these alternative models.

Enjoy

As I do every June on the blog here, it is my pleasure to offer a suggested summer reading list for educators.   This annual list also doubles as my top ten list of best book titles from the previous (school) year, in this case, 2014-15.

(For the past years’ recommendations, you can click the following for

• Summer 2014
• Summer 2013,
• Summer 2012,
• Summer 2011.)

It’s never easy making this list, reducing a year’s worth of reading to only ten titles.   I’ve added a few honorable mentions to round things out.   Nor is it easy, as I do ever year, to name a book of the year, the singular standout, and this year saw  a very tight competition between the number one and number two titles.

Unlike in some previous lists, this year’s top ten exclusively come from books published in the last sixteen months or so, all of them 2014 or 2015 titles.  As I did last year, I’ve also added toward the bottom some recommended free reports which can be found online in pdf, and some recommended fiction reading.

I should make note that  I can’t claim this list to be entirely free from bias.   As I’ve become a bit of a writer myself in the past seven years, I’ve come to know many others writers in education—and it’s become impossible to stand entirely apart from those relationships as I prepare my recommendations. Counting the honorable mention titles, five books here are authored by people I know well and/or have worked with in various capacities, and one other, the top title, is by someone I’ve met a few times.   Full disclosure.

Note: After the first two titles, the remaining are listed in no particular order.

#1 (Book of the Year): Transforming Schools: Using Project-Based Learning, Performance Assessments, and Common Core Standards, by Bob Lenz, with Justin Wells and Sally Kingston.

Bob Lenz has long been a leader in both thought and action for Project-Based learning and for transformative educational program design in the Bay Area, and it’s terrific to get this book from him now, which should expand his voice nationally and internationally.  His network of Envision schools in San Francisco, Oakland, and Hayward, at one of which I visited and shadowed a student at in 2008, are places which live up to the two-fold promise of the book’s title: they transform the lives of students, and in their example, show the way on how to transform schools.

The book speaks effectively to both classroom teachers and school/district leaders, more so than many other books of this type. The chapter on PBL, the book’s longest, stands out as one of the best succinct explanations of best practice. Particularly for teachers in more traditional schools, the section which promises that yes indeed, “PBL can start in your classroom” offers plenty of actionable, bite-size practical steps. The chapter also effectively empowers educators to respond to PBL skeptics, and addresses the coverage, rigor, and demographic fallacies—all of which I encounter frequently in my own work.

One of the most frequent debates I’ve had in the past half-decade about PBL is whether it can be effectively implemented incrementally. Some representatives of deeper learning argue, in my experience, vigorously that it can’t—it is an all or nothing proposition. Lenz, however, will have none of that—he sees it as the work of our entire century, not something that should or can be done overnight. And he offers useful advice on how to take first steps for principals and district leaders: ensuring “at least one deep learning experience per year; developing a graduate profile for your school; spearheading one structural change.”

Helpfully, because let’s not ever underestimate how hard it is to execute on the promise of deeper learning, Lenz ends his inspirational call for school transformation with a note of humility and frank acknowledgement of the challenge. “Don’t mistake our passion for overconfidence. Our failures have been many. Our vision is still evolving. Our schools are works in progress.   The successes we’ve had can be frustratingly impermanent.   The quality of our PBL ebbs and flows.   School culture can feel healthy, and then you turn to the next thing and it falters.   The work is hard, and it’s never done.”

#2: (Runner Up, Book of the Year).   Learning To Improve: How America’s Schools Can Get Better at Getting Better by Tony Bryk with Louis Gomez, Alicia Grunow, and Pal LeMahieu.

It was tough deciding whether it should replace the Lenz as number one, but that the Lenz speaks a bit more widely to educators of all stripes helped it to its first position.   But for school leaders (at all levels), strategic planners, and consultants who work on helping schools get better, this may become a bible of sorts for a long time to come.

Bryk, who is President of the Carnegie Foundation for the Advancement of Teaching, launches the inquiry by pointing out that despite tremendous initiative, our schools aren’t actually doing a good job of improvement.   But instead of jumping to the offering of solutions—so tempting!— by suggesting that what we really need is more (or less testing), or more, or less, pedagogical innovation, he turns his attention instead to the growing (but to me, previously largely unfamiliar) field of improvement science.   What we need as educational transformers to become better at is not what is needed for improvement, but how improvement actually can be brought to life.

(more…)

This post originally appeared on the Education Week blog Global Learning, May 21, 2015.

i’ve been working on various projects around noncognitive (social-emotional leaning and 21st century competency) assessments for the past three years, as some readers know, including for Secondary School Admission Test Board (SSAT-B) and for Index and its Mission Skill Assessment.  Recently the Asia Society published a new paper, A Rosetta Stone for NonCognitive Skills, which I co-authored with Richard Roberts, Ph.D., the former ETS research scientist who designed the Mission Skills Assessment.    The following was posted to EdWeek as a teaser of sorts for the full paper. 

—-

“It’s become a Tower of Babel,” Harvard Graduate School of Education’s Chris Gabrieli said recently when presenting at the Brookings panel, “Ready to be Counted? Incorporating Noncognitive Skills into Education Policy.” Gabrieli listed the nearly dozen labels used to describe  21^st century skills or noncognitive skills, including character, soft skills, emotional intelligence, social intelligence, and many more.

It’s not just here in the U.S.: this is a global phenomenon. In one recent post, UNESCO’s Asia Pacific Regional Bureau for Education examined the shift toward transversal competencies in the Asia-Pacific region. The OECD calls them skills for social progress.

These terms each come with a different list of associated skills, strengths, competencies, and attributes.   The table we’ve prepared below names almost 50 of these skills, and it isn’t even exhaustive.

As the pendulum swings back from the narrow focus of No Child Left Behind, researchers, policymakers, and educators alike are returning their attention to the enormous importance of student affect, attitude, and effort, and their ability to self-control, collaborate, and commit themselves to learning.  Nearly every school system now recognizes that in order to have a world-class education system, students must be advancing in more than the cognitive and academic achievement domain.

To make the most of the opportunity to address the social and emotional skills gap, we have to be able to make sense of the Tower of Babel. How do we know how all these various skills and attributes relate to one another—and whether one group’s teamwork is another’s collaboration; one group’s work ethic is another’s responsibility?

The Big Five Factor Personality Model
There is a powerful way to reconcile, translate, and unify the myriad of terms and constructs that have emerged over the past decade. It isn’t new, and in fairness, in many psychology circles it is no secret, but it has been problematically underappreciated in education. It is the “Big Five” factor model of personality.

Under the assumption that all important matters in life have been named and are thus represented in our language, researchers in the 1930s searched Webster’s New International Dictionary from 1925 for English words that described human characteristics. In total, 18,000 English words were selected, with 4,500 being classified as descriptions of stable personal traits. They then analyzed the underlying patterns among them to reduce the massive list of traits, and studied personality data from different sources (e.g., interpersonal ratings, objective measures of daily behavior, and questionnaire results), and measured these traits in diverse populations to arrive at first 16, and then five, major personality factors.

These analyses consistently yielded five factors that were labeled Extraversion, Agreeableness, Conscientiousness, Neuroticism/Emotional Stability, and Openness.

Even though they were first discovered in the English language, replication studies in other major languages resulted in the same five factors. Indeed, this research has proven the Big Five’s universality in the vast majority of countries, cultures, and languages across the world. In short, the Big Five play an important role in human nature, independent of the environment writ large. See below for the countries where the Big Five have thus far been replicated.

(more…)

–

Last winter, the Headmaster of The Hill School, Zachary Lehman, had a vision for his annual board retreat: he would engage trustees more meaningfully and educate them about the 21st century learning concepts which he’s leading his school toward by reconstructing the board retreat itself as a 21st century classroom.     Headmaster Lehman asked my assistance, which I was delighted to provide, and we fashioned a 2 day program which was by all accounts highly successful.    The Hill School’s videographers fashioned the video above to tell the story, at least in part. Enjoy.

Zack Lehman and I presented about this new kind of board meeting, which can also be a model for other types of meetings in school, including admin retreats and faculty days, at the annual meetings of both PAIS and TABS (Washington DC, December.  For NAIS (Boston, February), I’ll be accompanied by Clair Ward, Head of School at Valley School of Ligonier; if you are attending either event, please consider this video a trailer and teaser for the full session, and we hope you will join us.

Having enjoyed the success of this experience, the Hill School board intends never to revert to previous form; this is the new format of their meetings.  At their most recent event, they went on to employ Design Thinking to develop programs as part of the capital campaign and master plan planning.     We both, Headmaster Lehman and myself, are converts and believers, and are eager to advance this model of meeting and learning, and are both happy to answer questions or be of service to schools wishing to employ this model in their meetings. Contact me at jonathanemartin@gmail.com for more information.

INDEX, a school benchmarking association, developed the Mission Skills Assessment with researchers at ETS to measure students’ proficiency and growth in six character or noncognitive skill areas: teamwork, creativity, ethics, resilience, curiosity, and time management.

In a report prepared for the ASIA Society of deeper learning schools by the RAND Corp entitled Measuring 21st century Competencies, researchers reviewed the breadth of tools currently available for assessing these competencies, and declared the MSA, which is now used at over 90 schools nationally and globally, as the singularly most impressive, finding it especially cost-effective, easy to use,  with strong reliability, validity, and resistance to faking.

I was fortunate enough to have been invited by the INDEX board and its executive director, Lisa Pullman, to research and write this user’s guide and toolkit for schools, empowering them to better use the MSA to improve student growth in the six Mission Skills.   INDEX has graciously permitted me to share it here.

Below the toolkit I’ve also posted a charming video about the MSA from one of its original founding schools, New Canaan Country School.

Pleased to share here the just completed SSAT Think Tank on the Future of Assessment Special Report 2014.   For the 2013 report, click here.   It was a pleasure to work with my SSAT colleagues in preparing this, and a delight to speak with so many fascinating people whom we profile and interview in the report.

Enjoy.

I made reference to this excellent National Research Council report just last week, explaining how I drew upon it for my “explorations of the 21st century learning landscape” presentation.  But that was just a glancing reference to it, really; now, in a short series of posts over the next few days, I want to pull out and share some key pieces of this report.

As valuable as this 204 page report it, it is not entirely easy reading, and surely some who are reading this are not likely to get to it anytime soon.   Its value is multi-faceted; first, because of its comprehensively broad survey of the educational priorities and methods required for successfully preparing students for the coming decades, and second, because its imprimatur from the National Research Council, part of the National Academy of Sciences, makes it impossible to dismiss as the wild-eyed ideas of a partisan progressive radical.

Among its priorities are establishing the critical competencies– defining them by citing a 2005 OECD PISA report, which defines competencies as being

…more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilizing psychosocial resources (including skills and attitudes) in a particular context.

This report reviews the many which came before it, citing one which reviewed 59 in total, finding that the most commonly cited 21st century skills were “collaboration, communication, information and communications technology (ICT) literacy, and social or cultural competencies.”

Taking a slight tangent, the authors here very usefully surface a never-ending issue around the identification, listing and taxonomy of such 21st century skills: how do we separate these out from each other, when so many overlap so greatly.   They call this the Jangle fallacy— a term previously unfamiliar to me but which I’m so glad to have in my vocabulary.  It is when “investigators sometimes used different measures—with different names–to study a single psychological construct or competency.”  Examples abound, such as collaboration and teamwork, or flexibility and adaptability: are these different competencies, or are we jangling here?  The problem is more important when you get to this issue: perhaps “separate measures of self-esteem, neuroticism, locus of control, and generalized self-efficacy were in fact focusing on a single core construct?”

Hence, the NRC report creates separate tiers for their taxonomy of the essential competencies, beginning with a big three set of bubble which are certainly distinguishable, dropping down to a second tier of critical components of each of the big three bubbles, and then sprinkling the bubbles with a dozen or more individual terms which are increasingly hard to distinguish altogether.

If you’re curious about which of the many non-cog competencies is singularly most important, by their research, here is your answer:

Among intrapersonal and interpersonal competencies, conscientiousness (a tendency to be organized, responsible, and hardworking) is most highly correlated with desirable outcomes in education and the workplace. Anti-social behavior, which has both intrapersonal and interpersonal dimensions, is negatively correlated with these outcomes.

Hardly surprising–and not too terribly illuminating, because in and of itself, it doesn’t seem to easy to teach or develop directly.  (But see the Chicago report, “Teaching Adolescents to Become Learners” for more guidance on this.

So what is the point of educating for these competencies, anyway?  Why are they so important, if we are educating for life and work?

The committee views the broad call for  “21st century skills” as reflecting a long-standing issue in education and training – the desire that individuals develop transferable knowledge and skills.

Associated with this is the challenge of creating learning environments that support development of the cognitive, interpersonal, and intrapersonal competencies that enable learners to transfer what they have learned to new situations and new problems. These competencies include both knowledge in a domain and knowledge of how, why, and when to apply this knowledge to answer questions and solve problems—integrated forms of knowledge that we refer to as “21st century competencies” and discuss further below.

Now, if our students will have little need to “transfer what they have learned to new situations and new problems,” then there is no need to develop these capacities.   But, as we know, routine work is no longer routine.

So for nonroutine work, these competencies are more essential.  (more…)