Measuring Up: What Educational Testing Really Tells Us

by Daniel Koretz

In matrix-sampled testing, the test is broken into a number of different parts that comprise different tasks, and these are then distributed randomly within classrooms or schools. Thus the test is not standardized for comparing individual students, but it is standardized for purposes of comparing schools or states. Matrix sampling is now common; it is used, for example, in NAEP, TIMSS, and some state assessments. The significance of this seemingly arcane innovation is that it allows the testing of a broader range of knowledge and skills—a larger sample from the domain—within a given amount of testing time. Initially this approach was seen as advantageous simply because it offers richer information, but it offers an additional, critically important benefit when tests are used for accountability: it changes the incentives for students and teachers. The broader the test, the less incentive there is to narrow instruction inappropriately as a way of “gaming” the system and inflating scores.

.c1