21k12

—

“I told my faculty members when they’re applying for summer grants for PD: only research-based practices will get priority funding for grants; the rest of the applications go to the bottom of the pile.”

—

“When speaking to one group of the faculty, be sure to provide research-based evidence for Project-Based learning; some of these teachers are very particular about that.”

—

“As a caveat I would not accept any data found from a nonacademic non-peer reviewed resource as reason for change or implementing new strategies.”

I’ve encountered each of these messages separately in the past few weeks, and I value them all as good reminders for me to try harder to ground my educational positions and advocacy with evidence from quality research, especially from academic peer-reviewed journal published research.

So what about Project-Based Learning?   PBL is a practice I advocate for frequently here at 21k12, and I do based largely on my own  2008 research, in which I spent five days shadowing students at PBL immersive schools, including High Tech High in San Diego and New Technology High School in Sacramento, and spent about 15 days doing the same at more traditional high schools, and concluded that the PBL schools were far superior in the way they engaged, challenged, and enriched the students and their learning.  It was a bit of an overwhelming recognition, the degree of difference and the degree of superiority I observed. I’ve written about this at length here.

But, in all fairness, this “research” hasn’t been published anywhere other than on the blog, and it hasn’t been peer-reviewed in  by academic researchers.

But there is very good peer-reviewed journal published research available on PBL; let’s review.

Over at Buck Institute for Education they highlight the meta-analysis studies published in the 2008 Interdisciplinary Journal of Problem-based Learning, 3(1), 4-11.

In the journal’s introductory article which I’ve embedded below, by Jason Ravitz, Ph.D. until recently the Research Director for Buck Institute for Education, the research is summarized. (Note: I assisted Jason with editing this piece, for which I’m recognized in the acknowledgements).

The available evidence is promising. Compared to alternative teaching methods, PBL holds its own on standardized tests of concept knowledge and excels on other kinds of outcomes. Walker and Leary’s meta-analysis combined 201 outcomes reported across 82 different studies. They focused on the average effect size of differences in studies comparing students who received a PBL-based curriculum to those who did not.

Walker and Leary conclude that even on standardized tests of basic concepts “PBL is able to hold its own in comparison to lecture-based approaches” (p. 27). Moreover, both Walker and Leary and Strobel and van Barneveld determined that when studies use assessments measuring application of knowledge and principles, the results clearly favor PBL.

PBL is effective for outcomes beyond standardized tests.  For example, students….. felt better prepared to use “self-directed learning skills, problem-solving, information gathering, and self-evaluation techniques” (Albanese & Mitchell, cited in Strobel & van Barneveld, p. 49).

The meta-synthesis conducted by Strobel and van Barneveld is featured in the journal, and I’ve embedded it below.  The nature of this sweeping study is explained as follows:

To answer our research questions about how the differences in the definitions and measurements of learning contribute to the inconclusiveness about the effectiveness of PBL, we conducted a meta-synthesis (Bair, 1999) of existing meta-analyses. The goal was to determine which generalizable value statements about the effectiveness of PBL were supported by the majority of meta-analyses.

The total number of studies included in this paper was eight meta-analyses and systematic reviews.

The research base on the effectiveness of PBL is particularly rich and strong in the field of medicine. Similarly well developed is assessment in the field of medicine, which allows comparisons of different instructional interventions on situated and standardized test environments. Not surprisingly, the meta-analyses dealing with PBL draw heavily from primary studies conducted in medicine, but contain studies from other domains (e.g., economy, computer science) to warrant a rather generalizable statement on the effectiveness of PBL.

As impressive as this research is in looking at so many studies through looking at these eight reviews, (several hundred individual studies in all), it can feel a bit limited for those of us in K-12 education, because so much of it is based upon post-secondary studies, particularly medicine.

The findings though are compelling, and ought to provide considerable confidence to those working in this field, that if our goal is teaching for understanding and transfer, PBL is the pedagogical avenue to take.

several value statements can be made about the effectiveness of PBL that were supported by the majority of the meta-analyses reviewed:

PBL instruction was effective when it came to long-term retention and performance improvement.  PBL students were overall slightly underperforming when it came to short term retention.

Ultimately, the goal of instruction should be performance improvement and long-term retention. Therefore, preference should be given to instructional strategies that focus on students’ performance in authentic situations and their long-term knowledge retention, and not on their performance on tests aimed at short-term retention of knowledge.

–

In the slides at top of the post, I’ve pulled together a set of some of the most pertinent findings on the research based support for the academic achievement efficacy of project-based learning.

The first few slides come, with permission, from a slide deck posted at slideshare by Jason Ravitz, Ph.D., now working as an independent research and evaluation consultant.   Slides 5-13 I threw together, not especially attractively, to showcase the highlights in K-12 education of peer-reviewed journal published findings from the bibliography posted over at edutopia. 

It is very affirming for those of us who advocate to see these positive results.  For instance,  I’m taken with the study, “Learning History in Middle School by Designing Multimedia in a Project-Based Learning Experience” because of its integration of technology, published in Journal of Research of Technology in Education, which found:

Results from content knowledge measures showed significant gains for students in the project-based learning condition as compared to students in the comparison school.

Some studies focus, appropriately, on whether PBL supports students in disadvantaged environments or low performing schools, finding that they do, findings I wholeheartedly endorse based on my observations at schools such as New Tech High School in a disadvantaged section of Sacramento, and CART in depressed Fresno.

But what about the higher performing settings such as the independent schools where I often work?

The Journal for the Education of the Gifted,( v19 n3 p257-75 Spr 1996) published a piece entitled “Content Acquisition in Problem-Based Learning: Depth versus Breadth in American Studies” based on a study comparing gifted students in 10th grade, and found

no differences in content acquisition (as measured by a standardized test) of 167 gifted 10th graders in American Studies classes who received either a problem-based learning approach or traditional instruction.

Results did not support the common assumption that curriculum fostering higher order thinking skills inevitably results in lower content acquisition. 

This is reinforced by a very interesting study in Washington State by researchers at the U. of Washington there, including the John Bransford, author of what I view as the extraordinarily authoritative book, How People Learn, published by the National Research Council and the National Academies Press.

Published in the Journal of Curriculum Studies and entitled “Rethinking advanced high school coursework: tackling the depth/breadth tension in the AP US Government and Politics course,” this looked at an AP Government course in a high performing school, where a full PBL curriculum was delivered, and compared results on the AP test to those of students in a similarly high performing school with a traditional pedagogy.

The PBL students performed as well as or better than traditionally taught students on the AP test 

Not content to settle for success on the AP– as they shouldn’t be, because we know that the AP does a poor job of evaluating the analytic and applied thinking skills so much more important in today’s world–  they went on to compare the two sets of students in a “complex scenario test,” and found that the PBL students, who had spent far more time in class wrestling with this kind of problem, did much better.

Edutopia did a terrific job producing the following video sharing both the process and the findings of this important research.

–

PBL offers students many advantages in learning in the 21st century.   It is almost self-evident that it will demand more of students in collaboration, communication, and creativity, the skills that are so essential.    This was confirmed recently in a major study set in West Virginia by Ravitz and others, reported at the WV Department of Education.

Findings.

Overall, there were substantial and statistically significant effect size differences between teachers who used PBL with extended professional development and other teachers in the sample. Compared with the matching group, the extensively trained PBL-using teachers taught 21st century skills more often and more extensively. This finding applied across the four content areas, in classrooms serving students with a range of performance levels, and whether or not their schools had block scheduling.

Also confirmed in this study is what I strongly believe – we need not fear that in shifting to this approach that we will compromise their mastery of core content and narrowly defined academic achievement as measured in standardized testing.

In summary, there is plenty of evidence for the effectiveness of well-implemented and well-supported PBL.  Instead of talking about whether PBL will work, we should focus on what is needed to make it work for our schools and students.