www.indexgroups.org msa docs MSA-Toolkit-Interactive.pdf

INDEX, a school benchmarking association, developed the Mission Skills Assessment with researchers at ETS to measure students’ proficiency and growth in six character or noncognitive skill areas: teamwork, creativity, ethics, resilience, curiosity, and time management.

In a report prepared for the ASIA Society of deeper learning schools by the RAND Corp entitled Measuring 21st century Competencies, researchers reviewed the breadth of tools currently available for assessing these competencies, and declared the MSA, which is now used at over 90 schools nationally and globally, as the singularly most impressive, finding it especially cost-effective, easy to use,  with strong reliability, validity, and resistance to faking.

I was fortunate enough to have been invited by the INDEX board and its executive director, Lisa Pullman, to research and write this user’s guide and toolkit for schools, empowering them to better use the MSA to improve student growth in the six Mission Skills.   INDEX has graciously permitted me to share it here.

Below the toolkit I’ve also posted a charming video about the MSA from one of its original founding schools, New Canaan Country School.

 

 

Pleased to share here the just completed SSAT Think Tank on the Future of Assessment Special Report 2014.   For the 2013 report, click here.   It was a pleasure to work with my SSAT colleagues in preparing this, and a delight to speak with so many fascinating people whom we profile and interview in the report.

Enjoy.

photo (25)
If you’re like me, one of the joys of summer is the time it makes available to catch up and jump ahead on the reading list.  I know I spend each spring nearly as much time deciding what to read as I spend reading itself.

Drawing upon my own reading in the past twelve months,  I’m pleased to share here some summer reading recommendations for Summer 2014.

(For the past years’ recommendations, you can click the following for

My annual lists are usually populated primarily by books published in the past year or so, but this year’s list is a bit broader, with about a third of its titles dating back over the past three or four years.   This is because as I worked this past year on preparing the OECD Test for Schools toolkit (see previous post), I did a deeper dive on two related topics, using data in schools and international benchmarking, and doing so brought me to some terrific, previously overlooked,  titles I am delighted to share on this list.

(Note: I write a monthly book recommendation column, “Sparks” for the L+D newsletter, and in some of the bits below, I’ve “self-plagiarized” a bit,  drawing upon and re-purposing from some of those pieces.)

The Top Ten 2014 Summer Reading Recommendation for 21st century educators (this year with indication on titles suitable for Beach Reading!)

(For those on a budget, Scroll to the bottom for five additional recommended freebies!).

berger   1. 2014 is only half over, but the front-runner for 2014’s  educational book of the year has to be Ron Berger’s Leaders of their Own Learning. This book elevates assessment to the its rightful place in the center, not the after-thought back-end, of learning, and to its rightful home in the heart and mind of each individual student.   For Berger, assessment is collaboration: “As students are given the tools to understand and assess their own strengths and challenges, their ability to take ownership increases.  In very concrete ways, students become leaders of their own learning- understanding learning targets, tracking their progress, using feedback to revise their work, and presenting their learning publicly-and partners with their teachers.”

The book is chock-full of action items and organizing lists for implementing this program, but especially wonderful are the charming, lovely, and sometimes even tear-inducing short essays with Berger which open each chapter: nobody writes as beautifully about children learning.

berger ethicPAIR THIS BOOK> Berger’s 2003 title, An Ethic of Excellence: Building a Culture of Craftsmanship belongs in the Hall of Fame.  In this extended, almost-lyrical essay, Berger writes as both a master cabinet maker and as an elementary school teacher to testify that it is the work that matters, and that when this is our guide and foundation, we can see student work soar and sparkle spectacularly.

 

Make-Space-Cover-e1325835564910-730x7302.  Wonderfully eclectic and beautifully graphic, both how-to manual and philosophical essay, Make Space: How to Set the Stage for Creative Collaboration  published in 2012 was my singularly favorite read of 2013 and is my most frequently recommended book of the past year.   Co-authors Scott Doorley and Scott Witthoft, who are also co-directors of the “Environmental Collaborative” of the Stanford d. School (Hasso Plattner Institute of Design), bring the intellectual animation and ingenuity of the d. School to life on these 250 pages.

third teacherHow_Buildings_Learn_(Stewart_Brand_book)_coverPAIR THIS BOOK> Make Space stands tall among a set of wonderful titles in this extremely fascinating genre, such as the comprehensive and highly visual survey of learning space enhancements, The Third Teacher (2010) and Stewart Brand’s brilliant 1995 treatise on the importance of adaptability and continuous evolution of space, How Buildings Learn.


Data-Wise-in-Action-DWIA__35995.1361422570.325.4003. Data Wise  (rev. 2103) and Data Wise in Action (2007) are comdata wisepanion titles
from the Harvard GSE and the its Data Wise project.  I think they are dynamite resources for any school administrator or teacher who wants to become more intentional, systematic, and action-oriented in using student assessment results meaningfully.    Every school collects data of various kinds, but very few school-folks, I fear, and I know I speak for my past-self as a principal, effectively use the ever-growing pile of numbers and statistics to make meaning and make change. In these titles, the path is clearly marked (in Data Wise) and effectively illustrated (in in Action).  The spirit of these books is very appealing, empowering and honoring the professionalism of teachers in an approach that is student-centered, inclusive, collaborative and reflective.  Highly recommended for all administrators and for those teachers interested in this subject and wanting see teaching to become not data-DRIVEN but data-WISE.  

 

sticks and stones4. Emily Bazelon, a legal journalist for Slate, writes from outside our ranks with highly attuned empathy, compelling storytelling, and great common sense to the enormously complex phenomenon in schools which we problematically reduce to the one-word characterization “bullying” in her 2013 title, Sticks and Stone: Defeating the Culture of Bullying and Rediscovering the Power of Character and Empathy.   Carefully recounting in great detail three notorious examples of bullying in schools, she calls it like she sees it, applying a careful definition—repeated and sustained physical or verbal abuse in a situation of unequal power—to each event and unpacks the complexity that lies behind the actions and decisions.  School-leaders are spared little mercy for their mistakes, but are offered sympathy as their situations escalate and misunderstandings accrue.   The book isn’t perfect: the promised-by-the-introduction section sharing solutions doesn’t bring the reader as far as one would have hoped, though the resource guide at the back is a useful tool.  PAIR THIS>  I’d suggest pairing this with the excellent anti-bullying report prepared for Harvard’s Berkman Center and the Born This Way foundation by danah boyd and Andover Headmaster John Palfrey: they are really great resources on this subject.  (Bazelon: BEACH READING!)

 

SmartestKidsintheWorldRipley9.125. This book isn’t by any means, the smartest book of the year: it disappoints on several fronts, but it’s still a worthy (and easy) read.   Ripley beautifully honors they student-eye perspective on high school,  illuminating international educational practices through the eyes of students she literally follows  around the world to Poland, Finland, and South Korea.  By reflection upon their authentic experiences she evaluates schooling: would that we all did the same more regularly. She cross references the anecdotal with authoritative PISA data to offer fascinating insights about comparative educational practice, and she doesn’t hesitate to proffer the strong opinions she has drawn from her research, controversial though they might be (against technology and athletics, for high stakes exit testing and national standards).   Very much worth reading as a conversation (or argument) starter and as an accessible introduction to international benchmarking and comparative educational studies. (BEACH READING!)

 

49398_Hargreaves_Global_Fourth_Way_72ppiRGB_150pixw6. Reading Ripley might entice some to go deeper into global educational studies, and there are some terrific, richer and wiser, books available.   Having read three of them recently, my pick among this trio is from Boston College professors Hargreaves and Shirley: The Global Fourth Way: The Quest for Educational Excellence.    The authors explain what they mean by the fourth way of educational change, which includes broadening the range of learning beyond the basics, using data to inform teacher inquiry not “drive it,” testing by sampling, and teachers as the co-creators of curriculum, not the deliverers of it.  The discussion of the fourth way by itself is worthy of extended examination and discussion as a model for educational transformation—it is worth the while.

The authors also tackle the fascinating and complex problem of whether innovation and improvemeallure_of_order-194x300nt are at odds or can be reconciled in the pursuit of excellence, and they then survey six surpassing shanghaidifferent case studies to exemplify the fourth way.  Some of the six they profile are a bit too familiar (Finland, Singapore) or fall a bit short in convincingly illuminating the Fourth Way in practice (California especially), but the sections on England (a particular school there), Ontario as a province, and especially the province of Alberta are inspiring and informative.  Bravo Alberta!

PAIR THIS> with Tucker’s Surpassing Shanghai and or Mehta’s Allure of Order, both of which similarly scan the international landscape and draw fascinating and applicable insights for US education.

 

leadingOnline.225x225-757.  If you are reading this blog post, you are probably already someone practicing online learning and leadership, and so you’ll easily agree with me that it was great to see this past year three Jersey educators issue two excellent guides for the practice of digital and online leadership.  From Montclair Kimberly Academy, Richards and Valentine argue in their short, colorfully illustrated Leading Online:  “These days, leaders look out for their organizations by looking out of their organizations… they know you don’t have to leave school to leave school.”   And they know that no matter how powerful the technology, human decision-making is still demanded.   As they explain in a chapter entitled “Design for Spaces, Care for Spaces,” tools cannot reorganize, synthesize, or make meaning:  “these tasks fall to the leader.” PAIR THIS> with the following.

 

sheninger8. New Jersey high school principal Eric Sheninger offers Digital Leadership: Changing Paradigms for Changing Times, which comprehensively covers the terrain of his topic and his passion.  This is a how-to introduction, with chapters on communication, public relations, branding, professional growth and PD, and increasing student engagement. Sheninger recognizes the challenge of digital leadership, and recounts mistakes he has made.   He opposes the idea of cultivating buy-in, which he says “requires a salesman-like approach that might contain if-then rewards.  We have no mandates to use technology at [my school.]”   Instead, he urges “Embracement attained through empowerment and autonomy.”

 

smarter-than-you-think9. Wired magazine columnist Clive Thompson published last fall Smarter Than You Think, which will brighten your outlook and bring many ideas to mind about the future of learning in a digitally enhanced, connected world.  The “huge impact on our cognition” digital tools are making is grouped into three major categories: “they allow for prodigious external memory; … make it easier for us to find connections– between ideas, pictures, bits of news– that were previously invisible; … [and] encourage a superfluity of communication and publishing which has many surprising effects.” (BEACH READ!)

 

download10. Promoting Grit, Tenacity and Perseverance: Critical Factors for Success in the 21st century, comes from what until recently Karen Cator’s excellent EdTEch shop at the US Department of Education—and it is free for download.  Available at the USDOE since Febraury 2013, it is confusingly still labeled a “draft,” but that doesn’t make it less worthwhile reading.  As the importance of non-cognitive factors for student success has surged back into consciousness these past several years, and most particularly that of what Duckworth calls grit,  many have been responding: OK, stipulated, but HOW?   Even Duckworth’s own TED talk offers next to no guidance on this question.

This DOE 100 page free resource, after reviewing terminology and fostering gritassessments, walks the readers through more than fifty intervention programs and methods for doing so, and then provides valuable recommendations for policy-makers, educational leaders, and parents for this work.   PAIR WITH>  Hoerr’s ASCD “Aria” sort book, Fostering Grit—though if you have time for only one, choose the DOE’s freebie.

 —

Bonus: Five more Freebies
As noted, number 10 on the list above is yours for free, and do remember that your summer reading doesn’t have to cost you the equivalent of a special restaurant  with your best friend.  Many excellent special reports and even full book titles are available for free as downloads.

 

nrc report cover1. One is the National Research Council’s comprehensive, evidence-based, rich and rewarding 200 page 2012 book, Education for Life and Work. It’s not the lightest reading, but it provides an excellent foundation in what is known already, what we must inquire into more thoroughly, and develop more completely, to advance true 21st century learning.

The NRC frames this “education for life and work” as “Deeper Learning:”
“Calls for such “21st century skills” as innovation, creativity, and creative problem-solving can also be seen as calls for deeper learning—helping students develop transferable knowledge that can be applied to solve new problems or respond effectively to new situations.”
photo2. The National Research Council has also helpfully summarized its research for educators, in the succinct and accessible Practitioner’s Guide.

-

measuring 21st century competencies3. The Asia Society engaged researchers from the RAND corporation to prepare a really interesting and useful report which I think would be found valuable reading for all who are engaged in advancing 21st century learning.    Called “Measuring 21st century Competencies: Guidance for Educators,” it is about sixty pages in length, and after helpfully defining this set of competencies, and reviewing the key issues in testing and assessment generally, it reviews a wide array of techniques and tools for doing so.   Interestingly, it chooses one as being most worthy of a “deeper look,” the Mission Skills Assessment from the Index group, and the discussion it provides about this fast emerging tool, (full disclosure: one with which I have an association) is very informative about the future of non-cognitive assessment.

-
PISA-2012-(vol4)--Cover-(eng)-miniature4. The OECD, based in Paris, owns and manages PISA, the Programme for International Student Assessment—which makes news every three years for its “league tables” ranking national educational performance.  Less well known is that its team of educational researchers spends years reviewing the testing data and studying international educational practices to prepare fascinating reports on what works in education—and they are all free.  There are half a dozen or more I especially like, including the two Lessons for the US from PISA (2009) and (20012), but my top pick is the recent 2013 publication: PISA Results 2012:  What makes school successful.  Learn what these researchers have found about areas such as governance, school climate, student grouping, grade repetition, student-teacher ratio, teacher professional development, and much more.

 

SSATB  Who We Are » Think Tank » Think Tank Special Report 2013 (1)5. Forgive me tooting my own horn, but I’d add last to this list the “Special Report on the Future of Assessment” published last summer by SSAT, a 24 page review of the big ideas, leading thinkers, and innovative practices in non-cognitive assessment.    (Note, a second SSAT special report is due out in a few weeks.  Check back here or at the SSAT website to grab it when it is available).

—-

If you’re looking for even more “freebies” for 21st century learning reading, be sure also to check out the DML website of the MacArthur foundation.  They’ve teamed up with MIT press to publish a series of fascinating volumes such as Measuring What Matters Most and The Future of the Curriculum, and they’re all available for free download.

Enjoy your summer reading, and please: take a moment to use the comment box to share your suggestions for summer reading for 21st century educators.

This sixty page guide is really several things in one.

  • It is in part a guide to this particular tool, the OECD Test for Schools (Based on PISA), a test which individual schools, public and private, can participate in.
  • It also provides some high level treatment of the test’s alignment of PISA testing with 21st century skills and  “Deeper Learning.”  See the Appendix.
  • Finally, in the first full section, “leading your OECD program” and in the Case Studies section, the assessment example is OECD testing, but the framework and the treatments can serve as a guidance generally for how schools can best manage a new assessment tool project, using that new test or tool to advance student learning outcomes.

Enjoy.

 

nrc report coverI made reference to this excellent National Research Council report just last week, explaining how I drew upon it for my “explorations of the 21st century learning landscape” presentation.  But that was just a glancing reference to it, really; now, in a short series of posts over the next few days, I want to pull out and share some key pieces of this report.

As valuable as this 204 page report it, it is not entirely easy reading, and surely some who are reading this are not likely to get to it anytime soon.   Its value is multi-faceted; first, because of its comprehensively broad survey of the educational priorities and methods required for successfully preparing students for the coming decades, and second, because its imprimatur from the National Research Council, part of the National Academy of Sciences, makes it impossible to dismiss as the wild-eyed ideas of a partisan progressive radical.

Among its priorities are establishing the critical competencies– defining them by citing a 2005 OECD PISA report, which defines competencies as being

…more than just knowledge and skills. It involves the ability to meet complex demands, by drawing on and mobilizing psychosocial resources (including skills and attitudes) in a particular context.

This report reviews the many which came before it, citing one which reviewed 59 in total, finding that the most commonly cited 21st century skills were “collaboration, communication, information and communications technology (ICT) literacy, and social or cultural competencies.”

Taking a slight tangent, the authors here very usefully surface a never-ending issue around the identification, listing and taxonomy of such 21st century skills: how do we separate these out from each other, when so many overlap so greatly.   They call this the Jangle fallacy– a term previously unfamiliar to me but which I’m so glad to have in my vocabulary.  It is when “investigators sometimes used different measures—with different names–to study a single psychological construct or competency.”  Examples abound, such as collaboration and teamwork, or flexibility and adaptability: are these different competencies, or are we jangling here?  The problem is more important when you get to this issue: perhaps “separate measures of self-esteem, neuroticism, locus of control, and generalized self-efficacy were in fact focusing on a single core construct?”

Hence, the NRC report creates separate tiers for their taxonomy of the essential competencies, beginning with a big three set of bubble which are certainly distinguishable, dropping down to a second tier of critical components of each of the big three bubbles, and then sprinkling the bubbles with a dozen or more individual terms which are increasingly hard to distinguish altogether.

NRC report bubbles

If you’re curious about which of the many non-cog competencies is singularly most important, by their research, here is your answer:

Among intrapersonal and interpersonal competencies, conscientiousness (a tendency to be organized, responsible, and hardworking) is most highly correlated with desirable outcomes in education and the workplace. Anti-social behavior, which has both intrapersonal and interpersonal dimensions, is negatively correlated with these outcomes.

Hardly surprising–and not too terribly illuminating, because in and of itself, it doesn’t seem to easy to teach or develop directly.  (But see the Chicago report, “Teaching Adolescents to Become Learners” for more guidance on this.

So what is the point of educating for these competencies, anyway?  Why are they so important, if we are educating for life and work?

The committee views the broad call for  “21st century skills” as reflecting a long-standing issue in education and training – the desire that individuals develop transferable knowledge and skills.

Associated with this is the challenge of creating learning environments that support development of the cognitive, interpersonal, and intrapersonal competencies that enable learners to transfer what they have learned to new situations and new problems. These competencies include both knowledge in a domain and knowledge of how, why, and when to apply this knowledge to answer questions and solve problems—integrated forms of knowledge that we refer to as “21st century competencies” and discuss further below.

Now, if our students will have little need to “transfer what they have learned to new situations and new problems,” then there is no need to develop these capacities.   But, as we know, routine work is no longer routine.

work routines

So for nonroutine work, these competencies are more essential.  (more…)

-

There are two axes on which we can map and evaluate the excellence of a standardized test, or in other words,  external measurement of learning outcomes.   The first axis is does it measure what matters: does the work it requires students to do, the learning it requires students demonstrate, matter to successful citizenship and professional accomplishment?   Does it assess not content measurement but application of knowledge to original, complex problems; does it assess students’s ability to do that most important of things, TRANSFER?

The second axis is what is done with the results of the test?  Are the results used to punish schools, teachers, kids?  Are the results published in ways which rank kids, or rank teachers– even in the newspaper?  In a neutral position on this axis is the null effect: are the results provided to educators, without repercussions or rewards, but also without great guidance, insights, analysis, comparisons, examples?    Or, at the high end of this axis,  are the results used as and for compelling and creative information, illumination, and inspiration to improve learning for ALL students?

Map your favorite– or least favorite– tests on these axes– and then watch Andreas Schleicher in the TED talk above discuss the PISA test and map PISA.

(Full disclosure, I’m no longer, as of about a month ago, a neutral or objective commentator on this topic, having taken on an engagement for a PISA related project.   But, I was thrilled to seek and accept the engagement because of my passion for the power of PISA.)

Tony Wagner  was one of my influences here, particularly as he argued for the value of PISA, particularly what it measures, in a postscript to his film, the Finland Phenomenon.

“The PISA tests measure different things from TIMMS, which measures factual recall of memorized content knowledge.   This is a test asking you to apply knowledge to a new question or problem you’ve never seen before.”

Wagner interviews Roger Bybee, Ph.D., who explains that

“TIMMS and NAEP are similar in that they both are curriculum based, seeing what the standard curriculum contains and designing an assessment to see how well students have attained what is in curriculum.  PISA is an assessment that takes a literacy approach– not just the content one learns from the curriculum but how well you can apply that content in life situations in everyday use of the knowledge and the abilities one has.

About measuring what matters, Schleicher has a funny line:

So with PISA, we try to change this by measuring the knowledge and skills of people directly. And we took a very special angle to this. We were less interested in whether students can simply reproduce what they have learned in school, but we wanted to test whether they can extrapolate from what they know and apply their knowledge in novel situations. 

Now, some people have criticized us for this. They say, you know, such a way of measuring outcomes is terribly unfair to people, because we test students with problems they haven’t seen before

But if you take that logic, you know, you should consider life unfair, because the test of truth in life is not whether we can remember what we learned in school, but whether we are prepared for change, whether we are prepared for jobs that haven’t been created, to use technologies that haven’t been invented, to solve problems we just can’t anticipate today.

The bulk of the talk is dedicated to what I described above as the second axis: what do we do with these results?  OECD and PISA have dedicated 15 years to making sense of international testing results and comparisons– and it is a treasure trove. (more…)

-

I greatly enjoyed a lovely morning here in Santa Barbara today presenting to the Cate school faculty on a topic of their choosing, touring the 21st century learning landscape with an emphasis on the skills, competencies, habits and attitudes which are most being emphasized in learning today.Cate_School_Logo

After an entertaining launch to our morning with the Marshmallow Challenge, we discussed the importance of collaboration and iteration in the 21st century learning, with a suggestion that PBL is the best way to support these elements.   Keying off of the TED talk about the Marshmallow Challenge, we also asked the question how we help our students act more like Kindergartners– prototyping and iterating– than MBA students, searching for the one perfect solution.

We then transitioned to a review of the the 21st century skills landscape, and drew heavily on the excellent National Research Council Report, Education for Life and Work.    (I’ve embedded this magnificent free, 200 page resource at the bottom of this post.   It is not an easy or quick read, but it is a very worthwhile study.)

That work features a tripartite construct, comprehensively communicating what we need to support the learning and development of the cognitive, intrapersonal, and interpersonal qualities in each student.  Below is the graphic with the key two or three elements the report emphasizes for each of the three major categories.

NRC report bubbles

Onto the tripartite structure I mapped the Cate’s 5 Educational Principles– Knowledge, Communication, Curiosity, Determination and Compassion– and my “top ten” list of key 21st century skills and habits of mind and character: the 4 Cs; play, iteration, and experimentation; 2 essential intrapersonal elements, grit and the growth mindset; and the four critical components of the internet era: connected learning, attention and self-regulation, information literacy, and positive digital citizenship.   You can see this all mapped out in the overly crowded slide (!) below, with Cate’s 5 elements in red, my set in black, set atop the NRC tripartite bubbles.

21st century learning landscape

Each of these areas we explored in varying depth, taking an extended pause in which participants were provided and actually asked to undertake/solve and discuss four pertinent self-assessment and test questions:  Duckworth’s grit survey, a Growth mindset survey; a sample CWRA question; and sample PISA questions.

Below are the videos we used in the program.  My thanks go to everyone at Cate, including my main contact Jay Dorion, Assistant Headmaster, and Ben Williams IV, Headmaster of Cate.

-

-

-

-

 

Follow

Get every new post delivered to your Inbox.

Join 5,262 other followers