Statistics for People Who (Think They) Hate Statistics

by Neil J. Salkind

Notice that reliability is not the same as validity; it does not reflect whether you are measuring what you want to. Here’s why. True score has nothing to do with whether the construct of interest is really being reflected. Rather, true score is the mean score an individual would get if he or she took a test an infinite number of times, and it represents the theoretical typical level of performance on a given test. Now, one would hope that the typical level of performance would reflect the construct of interest, but that’s another question (a question of validity). The distinction here is that a test is reliable if it consistently produces whatever score a person would get on average, regardless of what the test is measuring. In fact, a perfectly reliable test might not produce a score that has anything to do with the construct of interest, such as “what you really know.” Observed Score = True Score + Error Score Error? Yes—in all its glory. For example, let’s suppose for a moment that someone gets an 89 on a stats test, but his or her true