—

As I wrote about last year, I am greatly enthusiastic about the opportunity Open Computer testing plays in assessing our students in their development of 21st century skills.     I think it is a really exciting way to take assessment into 21st century information environments, and to situate students in much more real-world situations as we prepare them for the contemporary world of work.

The above slideshow displays what I think is a very well designed “Open computer/open internet” exam, and Dr. Scott Morris has added on every slide his annotations of how he expects students to use the internet in answering these questions, and how these questions, with those resources, demand more of his students, particularly the higher order thinking skills of critical thinking and analytic reasoning, and a deeper understanding of the course material. (If the slides are too small for the print to be legible, click on the full screen link at the bottom right).

As I frequently discuss with Dr. Morris, with whom I have prepared this post and with whom I am co-presenting on this topic at the NAIS Annual Conference in March, our rationale is that our students are preparing to work in professional environments where they must tackle and resolve complex problems, and we know that in nearly every envisionable such environment, they will have laptops or other mobile, web-connected, digital tools to address those problems.   Let’s assess their  understanding in situations parallel to those for which we are preparing them.

But it is not just a matter of situating them in real-world environments, but that with open computer testing, the format of exams, (and yes, as much as I am a huge fan of PBL, exhibitions, and portfolios, call me retrograde but I still think there is a place and a role for exams in the range of assessment tools we use,) but the format of exams changes in really meaningful ways.

In his Chemistry class, Dr. Morris recognizes how radically the questions he asks must change if he knows his test-takers have access to the internet, Wolfram-Alpha, and a myriad of other sources on chemical information.   His questions must require his students to genuinely sort out what information they require, get that information, evaluate it, and then apply it to solve his now much more complex and rich questions.    This is assessment for genuine understanding, not assessment of recall and regurgitation.     It is far likely to be assessment of lasting understanding and future applicability than typical memorization based testing.

Open computer testing also benefits students who are organized in arranging their files, book-notes and websites; preparing for an exam like this develops valuable digital proficiencies.    They can upload their notes and organize them in all kinds of easily accessible ways.  Dr. Morris also encourages them to collaborate in advance, urging them to create shared files of notes and strategies for attacking difficult exam questions.   Collaboration during the exam-taking, however, is not permitted.

Dr. Morris shared with me these thoughts about the advantages of laptop usage on exams in his class:

1. Minimizes need for rote memorization (element names, symbols, polyatomic ions, activity series of metals).
2. Takes advantage of powerful interactive information sources (ptable.com)
3. Teaches organization and problem solving approaches
4. Encourages preparation: group study/collaboration/sharing of “discovered” resources
5. Encourages early experimentation and/or confidence with advanced problem solving software (Wolfram Alpha)
6. Reduces frustration (on part of students that have difficulty with memorization tasks)
7. Allows advanced data analysis and graphical output (scatterplots, etcs.)
8. Allows graphical content (from internet) on exams

It is fascinating to watch his students at work during one of these exams.   One of our goals for the NAIS presentation in March is to prepare a video documenting students as they use their tools, scanning information sources and using online calculation programs, to develop their answers over the course of the exam.

Recently I interviewed students discussing their work preparing for open computer examinations.   When I asked how many liked the format, all but one student raised a hand.

Some quotes:

We don’t have to memorize everything- which is basically what is going to happen in actual careers you don’t have to memorize everything you need to know you can look it up on the fly.

By having the internet available you can get a lot more information to use on your exam and get a better understanding.

You still need to memorize and learn the basics.

We are still studying to learn how to apply facts on the test.

And we are studying to determine what the most efficient way to find the information.

There”s more information that is asked for than would be asked for on a test without computers.

This is harder in different ways: It asks for a lot more information.

Instead of thinking about the equations you are thinking about how to get the ideas and really understand the chemistry.

You  have to understand the ideas to find what you actually need because there is a lot of stuff on the internet that doesn’t help you.

I am much more organized on the computer than I am on paper, so this helps me stay organized and use my organization in learning.

This is much more enjoyable a way to learn Chemistry.

I think this is just the tip of the iceberg, and offers a way to rethink and broaden considerably how we assess the skills students most need today, and how we deepen the understandings our students can gain in their course-work.

For more, see Steve Taffee’s great post on the topic: 1:1 programs and Student testing.

Part of being an educated person is knowing which tools to select, whose opinions to seek out, how to analyze and synthesize disparate views into a cogent, persuasive argument. I maintain that this means full access to technology. In 2009 I wrote (Hey Teach! Can I Phone a Friend?) about how some teachers are trying to create “Google proof” test questions. Rather than cast the problem in a negative light, let’s instead think of how we can build into assessments questions that require students to perform actions such as these:

• include the perspective from two personal contacts living in other countries (or representing different age groups, ethnicities, different schools, et al.)
• include links to primary documents to support your argument.
• create a multimedia mashup of text, video, sound and animation to prove your thesis.
• summarize your argument in several different media: a 140 characterTwitter post, a Wordle page, a one stanza original music composition, or a ten second animation.
• post your idea to three selected blogs or web sites and summarize and critique the response you receive.

You can also read about Denmark’s experiments in this direction here:  Danish Pupils Use Web in Exams

At five to nine, the room falls silent. CD-roms and exam papers are handed out together. This is the Danish language exam. One of the teachers stands in front of the class and explains the rules. She tells the candidates they can use the internet to answer any of the four questions. They can access any site they like, even Facebook, but they cannot message each other or email anyone outside the classroom.

The teachers also think the nature of the questions make it harder to cheat in exams. Students are no longer required to regurgitate facts and figures. Instead the emphasis is on their ability to sift through and analyse information.

Minister for education in Denmark, Bertel Haarder, says: “Our exams have to reflect daily life in the classroom and daily life in the classroom has to reflect life in society.  The internet is indispensible, including in the exam situation. I’m sure that is would be a matter of very few years when most European countries will be on the same line.”

And at edutopia they have posed the question and the comments are very illuminating: Should students be allowed to use technology to access informati0n during tests? 

One more:  Shelly Blake Pollock’s excellent example of his ancient history exam: Demonstrating Understanding through Connection and Collaboration.

In the following exam, you are going to be asked to do the work of a historian. Please read the questions carefully as many of them have multiple parts. If you have any difficulty understand concepts or terms, look them up. In real life, historians have the power of the Internet at their fingertips; so too do you on this exam. Further, there will be sections of this exam that assess your ability to collaborate in real-time over the web. This is an essential part of the real work of the 21st century historian and it is something in which you are going to demonstrate fluency.

Lastly, remember that history is as personal as it is public. Think hard about these questions before answering. Don’t just Google yourself into a panic. Use the resources of the Internet History Sourcebook, the BBC History site, National Geographic, the Met Museum, Nova, PBS, Infotrac, Grolier, Biography.com, the Internet Archive, the Library of Congress, the Smithsonian, and all of the resources we have used in class. Also remember to refer back to the Twitter lists we put together during review; they are full of good sources — but beware the occasional not-so-good source: When using a source, ask yourself, “Would Your Teacher Use This Source?”

My post from a year ago: Open Computer Testing: Putting 21st c. learning to the test.