It was terrific to have the chance this month both to see the keynote from Angela Duckworth at NPEA and to have 90 minutes sitting with her in a small group conversation with the SSATB Think Tank.

As many now know, she has become something of “the guru of grit” in the last year or two, particularly with the attention brought to her work by the writing of Paul Tough in his book and New York Times magazine cover story.  She is an Assistant Professor at the University of Pennsylvania.  I wrote about her work, her TEDx talk, and the Tough book previously here. 

images (4)Duckworth opened her keynote with the message that academic skill development is always interwoven with so-called “non-cog” skills.

The stuff kids need to learn in school is hard.   It’s really hard.  But it is not too hard.  Every child in my classroom– whether it took two hours or twenty hours– could learn this.    It isn’t quantum mechanics, it is Algebra.    In other countries most kids get it because they have the expectation that everyone can do this and they have attitude that it just takes a lot of work to get there.

IQ is not the limiting factor for most of our children.” We shouldn’t tolerate lower expectations for some kids.

Algebra is hard in another way- psychologically, for instance.  Is it hard to persist when it is challenging.

“if you can build non-cog skills, you will boost academic achievement. It is NOT either/or, but BOTH/AND.”

The message, of course, about the value of persistence, is not just for our kids: it is for all of us.   As she explained, and tied it to her own work and the work of everyone in the audience at NPEA, doing the hard work of providing quality education to disadvantaged youth, “It’s not a one year or two year project for any of us in life, tackling something hard and trying to make a real difference.”

angeladuckworthGrit is about “remaining loyal to your commitments.  Perseverance and Passion for long-term goals. Achievement = talent x effort. Anything multiplied by 0 = 0. Grit is about some talent but more about passion and perseverance.”

But we are all deceived, so much of the time, by the false impressions most others give off of gently gliding along the surface, like a duck with no worries.    “We need to show kids, and help them see, that below the waterline we are all paddling furiously.”

Duckworth emphasized the importance of not just teaching grit in some narrow method, but of deeply “Building a culture of grit, making it self-conscious and publicly visible for all.”

In an amusing and telling example, she shared the importance in Finland of a term roughly equivalent to grit, “sisu.”   There, she explained, Sisu is surfaced constantly:  “How’s your sisu today?”  “I’m feeling a bit down in my Sisu this week.”

Duckworth, speaking to an audience whose lives are devoted to helping students succeed in K-12 and collegiate education, stated the problem boldly and baldly: “We are not succeeding– we are getting kids well prepared academically, but they’re still not succeeding in college and careers– what do we need to do differently?”

We need to research, design interventions, experiment, and study results.  (more…)

Bill Fitzsimmons is the long-time Dean of Admissions at Harvard, and truly an important“dean” among university admissions officers.   I spent a few hours with him last week, listening to him present on a panel at NPEA, the National Partnership for Educational Access,  and then discussing admissions with him during a small group Think Tank conversation about the issues entailed in admissions assessment.

(Not e that quotes are roughly paraphrased from my notes on our conversation, and are not verbatim).

Fitzsimmons clearly loves his work.   He told us he is himself a first-generation college-goer, and reflects on that regularly in his work.    On April 10, a date you’d think would be a bit hectic for an Ivy league admissions dean, he spent two hours with our Think Tank, from 830pm to nearly 1030pm!

In the opening NPEA panel, to which he contributed greatly, much of his message was the importance he placed, and the progress Harvard (and other Ivies) are making, on widening access to under-represented populations, particularly now the lowest family income groups.

He told us of taking the Harvard undergrad population in the past six years or so from 11% to 17% Pell Grant eligible.  We’ve come a long way from asking ourselves whether “we were truly going to be  players in the educating of future leaders or boutiques for the wealthy and advantages.” “Private higher ed is back in the game.”   For 90% of Americans now, in contrast to ten years ago, it will cost less per year to send kids to Harvard (and other elite privates) than to their in-state flagship public institution.”

Still on the panel, he emphasized: “We need to look at all the human qualities of all our applicants, in all their complexity.”

In our smaller Think Tank conversation, he reminded us issues around expanding admissions criteria are neither new nor narrowly restricted: in the seventies Dean Willingham (?) of Williams College argued for the importance of selecting applicants for “persistent followthrough” which presages today’s focus on “grit,” and even today in China, land of the GaoKao, they are creating ways to accommodate rural applicants with lower test scores but greater perceived character traits than their urban peers.

Fitzsimmons told us that in round numbers at Harvard, which is as most know extraordinarily selective (2200 admitted out of 35,000—note that these 35,000 are those high school seniors self-selecting themselves to apply to Harvard), roughly 75% are admitted exclusively or especially for cognitive qualities, and 25% are admitted for the “bump” their applications get by demonstrated compelling non-cog attributes.    To compare these two groups finds no difference, he told us , in their success rate as undergrads: both have extremely high graduation rates, both at about 98%.

He said that among the things most important for their process is the teacher and counselor recommendations—particular when a student is rated as “one of the best in the past ten years.”

He also was emphatic about the importance of welcoming any and all student work which applicants wish to share: “we’ve been big on the portfolio piece for a long time.” (more…)

Think Tank“Working within a system,” Bill Sedlacek replied when Ray Diffley and I asked him about what one criterion among the many on his Non-Cog list he would choose to evaluate applicants, if he could choose only one.

Our conversation took place after Sedlacek’s keynote presentation at the University of Southern California’s Rossiter Center for Enrollment Research, Policy and Practice annual conference “Attributes That Matter,” which Ray and I attended on behalf of the SSATB Think Tank.

In his landmark book, Beyond the Big Test, Sedlacek elaborates: “the applicant’s ability to understand the role of the system in life and to develop a method of assessing the cultural or racial demands of the system and respond accordingly and assertively.” This is one of the eight research-based traits this distinguished scholar has, over a lifetime, determined “present a method of improving assessments for all students and are particularly useful for nontraditional students.”


We should be clear here: Sedlacek is not arguing we should dismiss traditional cognitive assessment in the form of the SAT or its analogues. Like Sternberg and other scholars in this field, he is calling for a more balanced approach, a both/and proposition that is entirely aligned with the mission of this Think Tank. To quote him in what might be something of a thesis for Beyond the Big Test:

We do not need to ignore our current tests; what we need is to add some new measures that expand the potential we can derive from the assessment. The goal of using non-cognitive variables is not to substitute this approach for the cognitive focus more commonly employed in assessments, but toadd to the range of attributes that we can consider in making the many judgments required of us all.” (Italics in original)

Read the full post at SSATB/The Admissions Organization…. 


When we are selecting for admission, do we prioritize selections of those who will succeed inside our halls, or those who will be most successful in using our education to be valuable after leaving our school?

In one of the most fascinating presentations of the USC conference, Sheldon Zadeck, a professor of industrial psychology at UC Berkeley shared the story of his ten plus year, as I understood it, project with the highly selective UC Berkeley Boalt Hall law school.

You can see his lengthy presentation above.

His work dates to the end of affirmative action, on the basis of race or ethnicity, in California by popular referendum.   As he explained, the result was a steep decline in the number of enrolled African Americans and Hispanics– because, he explained, their LSATs, and their combined rating of undergraduate GPA and LSAT scores, their INDEX,  were rarely high enough for admission.


Upon close review, it was also found that that combination could be “justified,” because it was a high predictor for first year law school GPA.

But- maybe first year GPA is not the only, or best, thing we should ask of prospective students applying to law school.   It not good law students we are seeking to add to the world’s ranks, it is high quality lawyers.

zedeckInstead, this professor spent years studying very intensely how one might establish the definition of a good attorney– and came up with 26 criteria.

Quoting the conference blog:

The 26 factors were organized into 8 categories including: intellectual and cognitive, research and information gathering, communications, planning and organizing, conflict resolution, client and business relationships- entrepreneurship, working with others (other attorneys), and character (passion and engagement, diligence, etc.).

Next up was collecting a sample set of over 1000 California attornies, Boalt Hall alums, who subjected themselves to rigorous evaluations– by themselves, peers, supervisors, etc– of their effectiveness on each of these 26.

The first question that follows: Did their LSAT scores predict their effectiveness as a lawyer?

For almost half– 12 of these factors, LSAT did correlate.

But for nearly a third, LSAT scores inversely correlated!

Regarding the INDEX of undergraduate GPA and LSATs, the INDEX predicted 9 of the 26 performance dimensions and 4 were negative.

As the blog quotes, Zedeck:

Thus, for some domains, the better you did on the LSAT, the less effective you would be in the workforce in that specific domain. (more…)

This is the third of four posts about the USC Rossier Attributes that Matter Conference.

Morning Sessions: Non-cognitive Variables in Action and Attributes of Good Students and Good Professionals

Where Bill Sedlacek (see previous post) laid out the intellectual concept of noncognitive assessment with a bit of history and a lot of theory, sharing his decades of research and his passionate advocacy, the following two sessions took us from theory to practice, as five university administrators and researcher told us about the fascinating work they’d done in this field.


Two bold and innovative directors of admissions at the university level, (Oregon State and DePaul), came to report that, despite their best efforts, their experiments with noncog assessment have had only very limited success in predicting student performance on campus.

As Eric Hoover reports in the Chronicle of Higher Ed about Noah Buckley’s leadership at OSU.

In 2004 the university added to its application the Insight Résumé, six short-answer questions.  One prompt asks applicants to describe how they overcame a challenge; another, to explain how they’ve developed knowledge in a given field.

The answers, scored on a 1-to-3 scale, inform admissions decisions in borderline cases, of applicants with less than a 3.0 GPA. “This gives us a way to say, ‘Hey, this is a diamond in the rough,'” Mr. Buckley says. For students with GPAs of 3.75 or higher, the scores help determine scholarship eligibility.

The Insight Résumé is a work in progress, Mr. Buckley says.

Reading 17,000 sets of essays requires a lot of time and training. Meanwhile, he believes the addition has helped Oregon State attract more-diverse applicants, but it’s hard to know for sure. A recent analysis found that although the scores positively correlated with retention and graduation rates, they did not offer “substantive improvements in predictions” of students’ success relative to other factors, especially high-school GPAs.

Details about the Insight Resume can be found in the slides above; it includes

Six short-answer questions asked as part of admissions application:
•Leadership / group contributions
•Knowledge in a field / creativity
•Dealing with adversity
•Community service
•Handling systemic changes / discrimination
•Goals / task commitment

Similarly, at DePaul as at OSU, very meaningful evidentiary results still stand further in the future.   (more…)

(2nd post in a series of there from the USC Attributes that Matter conference)

For several decades, going back to the 1960’s, University of Maryland Professor, now Emeritus, William “Bill” Sedlacek has been sounding the call, an inspirational one, to better appreciate the significance of non cognitive dimensions of learning and success, and has been calling upon all of us who are educators to integrate assessment of these “non-cogs” into university, and now K-12, admissions evaluation and student progress evaluation.

Bill sedlacekRegular readers here know of my passion for transparency and open-source sharing– that as much as is possible, we should make freely available our tools, resources, practices, and learnings, and welcome others to take and use them freely.   Sedlacek, who surely could have decided upon many moments in his career to seek to monetize his research and findings by creating a commercial, profit-making, noncognitive test, has instead taken the path of developing, posting, and sharing many different tools, welcoming schools and universities to take and adapt to his purposes.

As he explained, as a Professor he was successful, financially, enough– and “why would I need more? That others are using my tools, adapting them to their needs, to improve their process, their educational program, and opportunities for others is reward enough.” 

There is accordingly an extraordinary abundance of tools and resources freely available at Sedlacek’s site, here:

From that site, here is just a quick view of the particular instruments he is making available on his site, which could be of value to any and all educators seeking to expand their work in noncog assessment:

Professor Sedlacek’s presentation, which you can review in the slides above, and which is elaborated in writing in the document embedded below, was certainly to my eyes the highlight of the USC conference, Attributes that Matter: Beyond the Usual in College Admission and Success.


Why NonCog?  Obviously, this is the place to start, and Sedlacek laid out many purposes, all of which are very relevant to K-12 educators. (more…)

Attributes(First of 3 posts)

This conference surged with a spirited optimism.

We can do this: we can improve our ability

  • to predict who will succeed in our schools and colleges and who will benefit from our school’s programs and attributes;
  • to differentiate our institutions and market their unique values;  we can improve our ability to support our incoming students when they arrive by understanding them better in the admissions process;
  • and, perhaps most importantly, in the work of selective admissions, as complicated as they are, we can better choose those students who will best benefit from our programs to make a positive impact in their professions.

Last week I attended the USC Rossier Center for Enrollment Research Policy and Practice’s Attributes That Matter: Beyond the Usual in College Admission and Success  annual conference.  This is the first of three posts about the conference.

As regular readers know, exploring how we assess what our students are learning, and what things are most important for us to assess,  are  regular pursuits here.

This exciting conference looked at this topic through the prism of university admissions, but there is much we can learn and apply to the K-12 arena from this research.  This is the broader agenda of the this conference is explained by its chair:

What makes students succeed in school and how can we evaluate these things in students applying to our schools or preparing for further education, and what is important for students to learn, master and demonstrate for success later in life?

Eric Hoover at The Chronicle of Higher Ed explains the conference’s purpose this way:

Every year, presidents and professors expect freshmen who are curious, determined, and hungry for challenges. The traditional metrics of merit, however, can’t reveal such qualities. Standardized-test scores may or may not predict a given student’s long-term potential. Grade-point averages present only a partial view of an applicant’s talents and work habits. And so, some admissions officers say, it’s time for a new set of tools.

Critical discussion topics across the course of the conference included:

  1. Recognizing the noncog dimension is essential to all learning and thinking.
  2. Why noncog is increasingly essential to quality admissions assessment.
  3. What are key non-cog domains?
  4. Where do we find and how do we afford these tools?  How do we “operationalize” this?
  5. How’s is the progress of experiminents and initiatives underway in colleges and universities?

Of course, it is critical to recognize that the very Non-Cognitive, which emerged decades ago to describe these areas outside of IQ, are nevertheless entirely interwoven with our cognitive experiences and aptitudes.

Mary Helen Immordino Yang This was among the key messages made by the first night kick-off speaker, Mary Helen Immordino-Yang, from USC’s Brain and Creativity Institute, on the topic:   Embodied Brains, Social Minds: The importance of social reflectiveness and emotional awareness in young adult development

She argued that

Success in the 21st century requires not just positive results on the usual metrics but results measured in much, broader and richer ways.   We need to prepare people who can manage these changing times, creatively, responsibly, intellectually, socially.

What evidence are we gathering about the nature of emotions, to have a self, to have a subjective sense of purpose—how do we get that sense of self, how do we foster that and cultivate that in our young people?

Neuro-biologically, our minds are inherently social minds, the intellect that we have does not stand independent from the rest of our lives:  we use our intellect collaboratively—and we must take many inferences from the socially embedded minds. (more…)

Think TankNot everything that can be counted counts, and not everything that counts can be counted. — Albert Einstein

So much of what really matters in education just can’t be measured. — Independent school educators everywhere

Count me in. The quotes above are words I’ve uttered not dozens or scores but hundreds of times during my 15 years of independent school administration—and I very much believe I am in good company. Indeed, how can I argue with Albert Einstein?

But perhaps I am wrong. I’ve been enjoying reading this month a book which shakes my conviction that there is much of value that cannot be measured—and which gives very good guidance in how we can improve the way we capture in data just about anything we desire to know more about. The book is entitled How to Measure Anything by Douglas Hubbard—and although in my experience it is not a much discussed book in educational circles, I think it should be.

Grant Wiggins, author of the essential education book, Understanding by Design, is a fan of this book, directed me to Hubbard’s work in a blog post entitled “Oh You Can’t Measure That.”

Recently, I read a great book that might be of interest to anyone who wants to get beyond a knee-jerk reaction about what can and can’t be measured. The book makes the point from the git-go, in its title: How to Measure Anything: Finding the Value of Intangibles in Business, by Douglas Hubbard. Don’t let the ‘in Business” part throw you off. Almost everything in the book speaks to educational outcomes.

Hubbard writes with an axe to grind, and what becomes clear in the reading is that education is far from the only field or profession where managers express, frequently, their view that something, or most things, can’t be measured. This is Hubbard’s bête noir, one he is determined to confront with this book.

Often an important decision requires better knowledge of the alleged intangible, but when an executive believes something to be immeasurable, attempts to measure it will not even be considered.

As a result, decisions are less informed than they could be. The chance of error increases. Resources are misallocated, good ideas are rejected, and bad ideas are accepted.

Hubbard embeds as foundations to his argument three genuinely inspiring and impressive stories of measurement—times when individuals generated creative, ingenious methods for measuring something thought to be immeasurable—most famously and wondrously, Eratosthenes’ uncannily accurate measurement of the circumference of the earth two hundred years before the Common Era, using nothing more than shadows of the sun.

Click here to read the full post.

SSATBThink Tank

The New York Times Notable Books of 2012 list was published this week, and it was good to see its recognition of Paul Tough’s new book, How Children Succeed: Grit, Curiosity, and the Hidden Power of Character. Surely many blog readers who have not yet gotten to this new book will recall Tough’s widely circulated article featured on the cover of the New York Times Magazine in October 2011: “What is the Secret of Success is Failure?”

That article, which sits at the center of the new book, describes the work being conducted at two New York City schools that serve very different populations: Riverdale Country School (a SSATB Member) and KIPP mid
dle school. School leaders at both schools, however, have teamed to develop new tools and techniques to both cultivate and assess a set of character—particularly so-called “performance character”—skills and attributes, believing them to be equally important or, if we look to the Darwin quote above, superior to traditionally defined intelligence in making for future success.

As is fairly well known within our association, Choate Rosemary Hall has undertaken such an experiment over the past decade in an extraordinarily impressive way, as part of a collaboration with Dr. Robert Sternberg, a former Choate parent, and at the time head of the PACE (Psychology of Abilities, Competencies, and Expertise) Center. Choate Admission Director Ray Diffley’s leadership of this project, and the lessons he has learned from it, have brought him to the leadership of our Think Tank.

Choate’s work expanding its range of admission assessments has had several iterations. In its earlier, expansive version, which included a wide array of student assessments and tasks, it found “three consistent variables among students best predicted a student’s ability to thrive at Choate….

Read the full post by clicking here.

The Fall issue of the Secondary School Admissions Test Board member newsletter, Memberanda, was just published, with my article introducing its new think tank.   Click here to read the full article; below is the top section.


“Sure she’s smart, but I wish I could tell how creative she is: can she think out of the box?”

“His scores are middling, but he seems pretty motivated and persevering: too bad there isn’t a way to measure that.”

“You know, she had some difficulties on the test, but this teacher says she is a leader in her class – will the committee trust this one recommendation?”

Gather any group of admission directors together and before long, the conversation invariably turns to the issue of testing. While testing is useful and does tell us something about how a student will perform, there is so much more that we want to know about our applicants and so much more that is important about what they can bring to our school. How can we capture more information about our applicants?

SSATB recognizes that in the 21st century the nature of testing and assessment is changing and that its member schools are seeking new ways to assess diverse applicants’ readiness for their academic programs and educational settings. In response, SSATB has convened a Think Tank on innovation in assessment…. Read more. 

One of my main projects this year is serving as a member and consultant/writer for the Secondary School Admissions Testing Board (SSATB) Think Tank on the Future of Admissions Assessment.   More information on the Think Tank is here;  it’s charge is here. As part of my work I am posting a monthly column for the Think Tank; below is a “teaser” that post.  Click the link here or at bottom to read it in full. 

“Creativity,” Dr. Sternberg replied, when asked what addition to admissions assessment he would recommend if he had to limit himself to just one. Coming from the SSATB 2012 Annual Meeting’s keynote speaker, the former President of American Psychological Association, and arguably the world’s foremost scholar of – and experimental practitioner in – expanded admissions assessment, this is compelling counsel for our Think Tank’s work.

Using Sternberg as a framer and guide for the work of the Think Tank on the Future of Admissions Assessment is a no-brainer, and our time with him in Chicago was enormously valuable. In this post, we’ll take a deeper dive into assessing creativity; in future posts we’ll look at other Sternberg recommendations and many other aspects of expanded assessment for admissions.

Sternberg’s recommendation to prioritize creativity is both narrowly pragmatic and broadly idealistic.  Read on….

(originally posted this as SSATB 2013— not sure how/why my brain did that, but apologies)

Very happy to be reporting from this fine conference in the excellent city of Chicago.  Kudos to organizers: this was a great agenda and set of speakers, and I thought the JW Marriot space and the hospitality worked just fine.  I especially loved the availability of Diet Coke and sodas throughout the day!

(nb, full disclosure: I am here as a consultant participating on an SSATB think tank, and am very appreciative of working relationship I have with this fine organization.)

Let me share highlights and some thoughts on 8 of the sessions I attended (note: occasionally my tenses shift in these reports, when I transcribing live I use the present tense, when recollecting afterwards the past tense.  apologies).

1. Executive Director’s Heather Hoerle’s Welcome and Overview:

Priorities for SSAT

Goal: Keep Testing at the Highest possible standard

  • developed all new UL and ML test.
  • we now have independent school admissions tests written by independent schol teachers
  • Writing prompts had grown stale– so we have all new writing prompts
  • created a brand new elementary school admissions test.

Goal:  Strengthening Test Security.

  • Got serious about international testing administrations
  • rebuilt infrastructure with multi-level security
  • re-wrote test management instructions

Goal: Caring about and supporting Admissions professionals.

  • Wrote a report on the independent school professional.
  • worked with the ALC to offer free seminars and monthly blog
  • designed a survey to provide industry wide data on your issues, coming winter 2012
  • Desgined this conference with three themes: Lessons learned from higher ed, diversity and access, and future of assessment.

Goal: Help families and improve customer service.

  • Hired a professional call center and live chat
  • improved current registration system and revamping for next year.
  • redesigned materials for families.

Goal: Become More Transparent.

  • Asked us for help in trustee and award selection
  • Shared information about challenges.
  • Convened a think on the future of assessment
  • appointed a computer based test advisory committee.

There is clearly strong support and enthusiasm in this assembly for Heather’s leadership; one representative tweet from Bill Leahy,

Admissions Director at Phillips Andover: “Feeling fortunate to have a fantastic executive director@HeatherHoerle running @TheAdmissionOrg and #ssatb12


Creativity and Ethical Mindset: These are what we should be assessing in learning, PK-16, in addition to analytic intelligence. So says renowned author and academic, and former President of the American Psychological Association Robert Sternberg.

Important: All slides above are Sternberg’s, from his presentation today at SSATB.

Sternberg matters, and deserves even wider and deeper appreciation and influence in PK-12 education than he already has, and I am sometimes surprised about why he is not more frequently a reference point in 21st century learning. It may be that his work has been primarily in post-secondary that K-12 folks overlook him, but as he said today and as I believe firmly, in almost every way his works is entirely suited for applications in our domain.

Ray Diffley, the trailblazing Director of Admissions for Choate-Rosemary Hall, introduced Sternberg, and labeled him the single most important thinker on expanding and revamping educational assessment in the nation today.

Sternberg, who is in his sixties and has 20 month old triplets (!), couldn’t actually attend in person, due to his airplane’s equipment failure, but his virtual contribution worked just fine.

A few observations about Sternberg, but before you read any further, be sure to view the slides, all of them, if you haven’t seen him present. This session was a very valuable and sweeping overview of his essential themes and thoughts, and the slides convey a very high proportion of what he said in this session.

1. He clearly deeply cares about kids, his own kids and all others, and works always from a foundation of personal experience, his own learning journey.

2. He has walked this talk– he doesn’t just research how assessment can change, or theories of what it could be, but again and again at different universities and schools he has been implementing these assessments. Rarely do you find someone who has done more to blend theorizing and implementation.

3. Expanding assessment is both practical and idealistic.