Standardized Tests

Reliability and validity are two critical ideas for understanding standardized tests. To understand the difference between them, we can use the metaphor of two broken thermometers. An unreliable thermometer gives you a different reading each time you take your temperature, and the reliable but invalid thermometer is consistently ten degrees too hot. Validity also depends on accurate interpretations of results. If people say results of a test mean something they don't, that test may have a validity problem. Just as we wouldn't expect a ruler to tell us how much an elephant weighs, or what it had for breakfast, we can't expect standardized tests alone to reliably tell us how smart someone is, how diplomats will handle a tough situation, or how brave a firefighter might turn out to be. So standardized tests may help us learn a little about a lot of people in a short time, but they usually can't tell us a lot about a single person. Many social scientists worry about test scores resulting in sweeping and often negative changes for test takers, sometimes with long-term life consequences. We can't blame the tests, though. It's up to us to use the right tests for the right jobs, and to interpret results appropriately.