Measuring Up: What Educational Testing Really Tells Us

by Daniel Koretz

The NAEP is an interesting example because the agencies responsible for it have wrestled for years with the problem of adding meaning to scale scores and have devised quite a variety of approaches. One, of course, has been to layer performance standards (“achievement levels”) on top of the scale. But many of the approaches for giving the NAEP scores meaning rely on normative information. For example, normative data are key to making sense of NAEP’s much publicized comparisons among states. How is the commissioner of education in Minnesota to know whether that state’s mean scale score of 290 is good or bad? One way is to compare that average with the distribution of state averages, which the NAEP reports provide. The distribution shows that 290 is pretty damned good, at least so far as achievement in the United States is concerned. NAEP also provides, for every state, the proportion of students falling into each of the four performance “bins” created by the achievement levels—a combination of standards-based and norm-referenced reporting.17