21k12

–

When we are selecting for admission, do we prioritize selections of those who will succeed inside our halls, or those who will be most successful in using our education to be valuable after leaving our school?

In one of the most fascinating presentations of the USC conference, Sheldon Zadeck, a professor of industrial psychology at UC Berkeley shared the story of his ten plus year, as I understood it, project with the highly selective UC Berkeley Boalt Hall law school.

You can see his lengthy presentation above.

His work dates to the end of affirmative action, on the basis of race or ethnicity, in California by popular referendum.   As he explained, the result was a steep decline in the number of enrolled African Americans and Hispanics– because, he explained, their LSATs, and their combined rating of undergraduate GPA and LSAT scores, their INDEX,  were rarely high enough for admission.

Upon close review, it was also found that that combination could be “justified,” because it was a high predictor for first year law school GPA.

But- maybe first year GPA is not the only, or best, thing we should ask of prospective students applying to law school.   It not good law students we are seeking to add to the world’s ranks, it is high quality lawyers.

Instead, this professor spent years studying very intensely how one might establish the definition of a good attorney– and came up with 26 criteria.

Quoting the conference blog:

The 26 factors were organized into 8 categories including: intellectual and cognitive, research and information gathering, communications, planning and organizing, conflict resolution, client and business relationships- entrepreneurship, working with others (other attorneys), and character (passion and engagement, diligence, etc.).

Next up was collecting a sample set of over 1000 California attornies, Boalt Hall alums, who subjected themselves to rigorous evaluations– by themselves, peers, supervisors, etc– of their effectiveness on each of these 26.

The first question that follows: Did their LSAT scores predict their effectiveness as a lawyer?

For almost half– 12 of these factors, LSAT did correlate.

But for nearly a third, LSAT scores inversely correlated!

Regarding the INDEX of undergraduate GPA and LSATs, the INDEX predicted 9 of the 26 performance dimensions and 4 were negative.

As the blog quotes, Zedeck:

Thus, for some domains, the better you did on the LSAT, the less effective you would be in the workforce in that specific domain.

But could an a carefully selected set of other types of assessments, added into the INDEX mix, improve the predictability of future legal effectiveness?   From the conference blog:

As they developed new predictors, they looked at Snyder’s self-monitoring assessment, carver’s optimism scales, emotional intelligence and a biographical inventory (past experiences) approach, accomplishment and experience records, situational judgment, and a moral responsibility scale.

Zedeck continued to describe the many evaluation tools they used to examine the 26 factors, including those already developed and some that were developed in-house.

What did they find?

From the presentation, slides 45-50:

The LSAT, UGPA, and Index Score were not particularly useful for predicting lawyer performance on the large majority of the 26 Effectiveness Factors identified in our research.

In contrast, the new tests, in particular the SJT, BIO, and several of the personality constructs predicted almost all of the effectiveness factors.

[SJT: Situational Judgment Test, slides 32-34; BIO: Biographical Inventory, slide 31]

In general, race and gender subgroup performance did not substantially differ on the new predictors.

Results showed that the new predictor tests were, for the most part, measuring characteristics that were independent of one another.

As the conference blog reports,

 Zedeck found that their personality assessments showed less differences between minorities and Caucasians- and thus the use of these tools supports the identification of a more diverse student body.

Zedeck suggests that this type of process can also be done with undergrads. He did not promote abandoning the cognitive assessments, but to use them within the battery of tools used to identify students with potential for success.

You can read more about this very interesting work in Zadeck’s journal article here: Predicting Lawyer Effectiveness:Broadening the Basis for Law School Admission Decisions

So what are the implications for K-12?   Clearly we are not professional schools, for which we can evaluate a single profession for its ideal qualifications.

But, this research and these fascinating findings call upon us to broaden our gauge, particularly when we are selecting into selective private, magnet, GIFTED/GATE, and other programs.    Perhaps we do already have cognitive tools, standardized tests, which accurately predict the GPA in the first year of enrollment– but that doesn’t end the conversation.

Look to your mission, look to your alumni, look to your purpose and your school’s view of what our society needs the most: creative and innovative thinkers, people of fine judgment, leaders, collaborators, and a million other possibilities which you can use to more greatly differentiate your school– and then select or design the instruments which will help you make these predictions.