Email Status

Volume 24, Number 4
July/August 2008

What's Wrong with Wrong Answers?

Standardized tests fail to capture this important data source


It may come as no surprise that the people who design large-scale standardized tests focus primarily on right answers. Almost all the technical mumbo-jumbo that goes into determining if state or district tests are valid and reliable focuses on what happens with the correct responses. For instance, to ensure that questions are consistent, test designers expect to see reasonably strong relationships between how a student does on one item and their total score. To avoid bias, they compare correct answers across groups of students. To make sure tests differentiate among students with different skill levels, they compare the probability of students’ getting right answers on items at varying levels of difficulty. The only time they pay attention to wrong answers is when they need to score students’ answers to open-ended test items. For these types of questions, test designers create rubrics or scoring keys so that students can get partial credit. But in general, large-scale standardized tests do not report data that capture the quality of wrong answers and how they differ from the right responses—data that could provide important insights to the development of students’ thinking.

This is an excerpt from the Harvard Education Letter. Subscribers can click here to continue reading this article.


Also by this Author