Steve Volk, April 16, 2018

There is a point in every semester when, it being too late to make significant changes in our current courses, we instead begin to think ahead to next semester. So, overwhelmed as you are by finishing up this term, getting in book orders, and planning research and writing for the summer, maybe you can keep this article in mind as you plan for the fall 2018 semester!

In this posting, I provide a few resources and summarize some of the reasoning supporting the argument that student learning is more enhanced by frequent, low-stakes assessments (“retrieval practices” in the technical lingo), with the opportunities they provide for continual feedback, than from infrequent, high-stakes assessments (e.g., a midterm and a final). While the research has suggested that this holds across all disciplines, the impact is noted especially in STEM fields and quantitatively based courses.

Some Research

First, a bit of the research. Where in the classroom cycle (class-studying-examination) does learning most take place? We generally think that students learn when they are studying (reading assigned texts or reviewing their notes), and that testing is the mechanism that tells us what has been learned. The same assumptions also buttress what many instructors think about classroom practice: students are assumed to learn in lectures (learning which is solidified when they study their notes), and tests help us know how much has been learned. The test, that that sense, is considered a “neutral event,” it measures without impacting learning. And yet, a considerable body of research has demonstrated that such an understanding is flawed; many cognitive psychologists now suggest that assessment practices themselves can produce large gains in long-term retention of information and concepts in comparison with studying for an exam. In other words, write  Peter C. Brown, Henry L. Roediger III, and Mark A. McDaniel in Make It Stick: The Science of Successful Learning (Harvard, 2014), using “retrieval practices” (recalling facts, concepts, data, or events from memory) is “a more effective learning strategy than review by rereading.”

To summarize his (and Andrew C. Butler) conclusions from one of Roediger’s many studies,

1. Assessments (“retrieval practices”) often produce superior long-term retention relative to studying for the same amount of time;
2. Repeated assessments are better than one-time testing;
3. Providing feedback produces better results than assessment practices without feedback, although even without feedback, the learning results are better for multiple assessments;
4. Some lag between study and test support retention and learning;
5. The benefits of “retrieval practices” can be transferred from one context (domain) to another.

Cynthia J. Brame and Rachel Biel, both at Vanderbilt, offer a useful chart in CBE Life Sciences Education that summarizes the research on the relationship between testing and learning in undergraduate science courses. Among the findings of previous studies that they consider are the following:

• Testing improved retention significantly more than restudy in delayed tests.
• Multiple tests provided greater benefit than a single test.
• Students in the retrieval-practice condition had greater gains in meaningful learning compared with those who used elaborative concept mapping as a learning tool.
• Both initially correct and incorrect answers were benefited by feedback, but low-confidence answers were most benefited by feedback.
• Testing improved retention and increased transfer of information from one domain to another through test questions that required factual or conceptual recall and inferential questions that required transfer.
• Interim testing improved recall on a final test for information taught before and after the interim test.

But first, a note to the doubters – myself included. One of the central concerns I have always had about testing as a method of assessment, particularly high-stakes mid-terms or finals, or even post-reading quizzes, is whether they only promote rote memory learning (“When did Cortez first arrive in Mexico?” “Describe the wing structure of a bat,” etc.). I questioned whether the memorization of facts that (now) can be looked up instantaneously on a smart phone was necessary, or, in any case, whether what students learned would stick with them, or stick with them in a contextualized fashion, other than the memorization of random facts.

We all know that learning is more than the memorization and the repetition of facts, that higher education learning above all must shift students to higher cognitive levels and outcomes. So, while we know, through a variety of studies, particularly the work carried out by Andrew C. Butler, that “repeated testing produce[s] superior retention and transfer on the final test relative to repeated studying,” what do we know about the impact of frequent assessment such as tests and quizzes on helping to move students to higher cognitive levels? Can what was learned via testing be “transferred,” extended to other domains and contexts? And can attention to “retrieval practices” help students better cope with some of the negative consequences that accompany high-stakes testing? Stick around for some answers.

What to Keep in Mind

1. Frequent, low-stakes testing is better than infrequent, high-stakes testing.

The research cited above suggests the many reasons why this is the case. To put it in practical terms:

a. We know enough about high-stakes testing to know that such assessment practices can have serious negative effects on identity-threatened groups in specific contexts: Blacks and Latinx in the sciences; women in math; etc. Further, low-stakes testing reduces the anxiety associated with high-stakes testing since one’s future isn’t riding on one or two tests. And we know that student anxiety in general has reached frightening levels. In 1985,18% of incoming first-year students agreed that they “felt overwhelmed by all I had to do.” In 2010, that figure increased to 29%. Last year, it surged to 41 percent.

b. Frequent testing allows students to practice remembering, a useful capacity building skill. But, to help students not just retain information but to use retrieval practices to move to higher order thinking, the testing should be “effortful” (e.g., short answer rather than multiple choice), spaced (allowing enough time to elapse for some forgetting to occur) and “interleaved” (one should introduce different topics and problems rather than having students master one topic at a time and then move on to the next). (These are points that Brown, Roediger, and McDaniel stress in Make it Stick.)

c. Frequent testing helps students know where they stand in a course, making it more likely that they will seek help when needed, i.e., when it can actually make a difference. Particularly in STEM courses, if students only realize that they are in a very deep pit at the mid-term point, there is little that they can do to climb out.

2. Feedback is better than no feedback.

While the research suggests that low-stakes testing without feedback is actually better than relying on studying alone in terms of student learning, it also indicates the key role that instructor feedback plays in boosting student learning.

a. Most obviously, feedback from the instructor will let students know where they stand in a course. As faculty, we often have our own ways of intuiting how students are doing in our classes without resorting to formal testing mechanisms. Are they coming to class? Do they look engaged? Are they falling asleep in class? Have they come to office hours when asked? Some of these indicators are fairly accurate; others can be not only wrong but seriously problematic, particularly if they feed off of implicit biases. But the point is that students don’t share these “insights” as to how they are performing in a course. Many will feel that they are doing just fine, when they are actually standing at the edge of a precipice. (A recent study reported on in the New York Times suggests that men, more than women, overestimate their abilities in science classes. Who would have thought!) Concrete feedback from instructors can help students obtain a more objective sense of how they are doing. They may not always “hear” it or respond appropriately to such information, but offering feedback most often is quite welcome and useful.

b. Providing feedback to correct responses as well as incorrect answers is highly useful. As Brame and Biel discuss in the above cited article, feedback on both low-confidence correct answers and incorrect answers may further enhance the testing effect, allowing students to solidify their understanding of concepts about which they are unclear.

c. Feedback can come from instructors on exams or other assessments, or from in-class peers. One research study found that when students answer an in-class conceptual question individually using clickers, discuss it with their neighbors, and then re-vote on the same question, the percentage of correct answers typically increases.

3. Low-stakes, frequent testing with feedback helps students think more intentionally about their own learning.

I have written before about ways to support student metacognition. One way to do this is to encourage students to reflect on their own process of learning by having them respond to feedback on exams, quizzes, or other assignments.

a. Students can “self-test” as part of their studying process, and then measure the outcome versus their expectation, but few will actually do this, and generally those who do are not always the ones who could use additional help. Providing concrete evidence of accomplishments can help students more accurately assess their level of comprehension and (hopefully) seek help to address those areas that need attention.

b. Frequent assignments tied to reflection exercises can help students think about what they did (to prepare for an assignment, for exaple), what the outcome was, and what they will do differently the next time.

c. No-stakes formative assessment (a short quiz at the end of the class or a simple “one-minute paper”) can help students think in a more focused fashion about what they know and they need to focus on.

4. Design questions that can move students to higher order thinking.

While not every exam must be “effortful,” with some thought, one can design multiple choice questions to lead to higher order thinking.

a. Cynthia Brame, from Vanderbilt’s Center for Teaching, provides the following advice: Presenting a problem that requires application of course principles, analysis of a problem, or evaluation of alternatives tests students’ ability to engage in higher-order thinking. Instructors can design problems that require multilogical thinking (“thinking that requires knowledge of more than one fact to logically and systematically apply concepts to a …problem”).

Here are two examples she provides:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

b. In Dynamic Testing: The Nature and Measurement of Learning Potential , Robert Sternberg and Elena Grigorenko suggest, as the title indicates, a more dynamic approach to testing than what is typically provided in static, standardized tests. Essentially, this calls for determining the state of one’s knowledge; refocusing learning on areas that need more attention; and retesting to measure improvement. It is testing that helps the learner focus on new knowledge (i.e., on “learning”), rather than on what has already been accomplished. In this sense, well-designed testing can move the student to higher cognitive levels.

5. Break large assignments/exams into smaller pieces

Designing, for example, eight assessments tools that are effortful, spaced, and interleaved, takes some work, particularly when you are accustomed to giving a midterm and a final. True enough, but think about breaking the high-stake assignments you already give into smaller pieces, as suggested by Sara Jones at Michigan State University’s “Inside Teaching.”

a. Replace high-stakes, large tests into several quizzes.

b. Scaffold large projects (independent research projects, term papers, etc.) into a variety of related steps: topic, preliminary bibliography, outline, final bibliography, draft, etc.

In the end, if there’s one “take-away” from this posting, it is to become more intentional when thinking about the assessments we rely on, particularly in terms of exams or quizzes. The research is quite conclusive that, when well prepared and followed by feedback, “retrieval practices” are more effective in reinforcing student learning than studying in and of itself. Frequent, low-stakes assessments are an important way to promote student learning.