Noise: A Flaw in Human Judgment

by Daniel Kahneman

You might find that the answer lies in how you used the scale—what we have called level noise. Perhaps you thought a 5 requires something truly extraordinary, whereas the other rater thought that it merely requires something unusually good.