The Undoing Project: A Friendship That Changed Our Minds

by Michael Lewis

They then went on to show that trained scientists—experimental psychologists—were prone to the same mental error. For instance, the psychologists who were asked to guess the mean IQ of the sample of kids, in which the first kid was found to have an IQ of 150, often guessed that it was 100, or the mean of the larger population of eight graders. They assumed that the kid with the high IQ was an outlier who would be offset by an outlier with an extremely low IQ—that every heads would be followed by a tails. But the correct answer—as produced by Bayes’s theorem—was 101. Even people trained in statistics and probability theory failed to intuit how much more variable a small sample could be than the general population—and that the smaller the sample, the lower the likelihood that it would mirror the broader population. They assumed that the sample would correct itself until it mirrored the population from which it was drawn. In very large populations, the law of large numbers did indeed guarantee this result. If you flipped a coin a thousand times, you were more likely to end up with heads or tails roughly half the time than if you flipped it ten times. For some reason human beings did not see it that way. “People’s intuitions about random sampling appear to satisfy the law of small numbers, which asserts that the law of large numbers applies to small numbers as well,” Danny and Amos wrote.