The Data Detective: Ten Easy Rules to Make Sense of Statistics

by Tim Harford

Statisticians are sometimes dismissed as bean counters. The sneering term is misleading as well as unfair. Most of the concepts that matter in policy are not like beans; they are not merely difficult to count, but difficult to define. Once you’re sure what you mean by “bean,” the bean counting itself may come more easily. But if we don’t understand the definition, then there is little point in looking at the numbers. We have fooled ourselves before we have begun. The solution, then: Ask what is being counted, what stories lie behind the statistics. It is natural to think that the skills required to evaluate numbers are numerical—understanding how to compute a percentage, or how to disentangle your millions from your billions from your zillions. It’s a question of mathematics, is it not? What I hope we’ve learned over the past few pages is that the truth is more subtle yet in some ways easier: our confusion often lies less in numbers than in words. Before we figure out whether nurses have had a pay raise, first find out what is meant by “nurse.” Before lamenting the prevalence of self-harm in young people, stop to consider whether you know what “self-harm” is supposed to mean. Before concluding that inequality has soared, ask, “Inequality of what?” Demanding a short, sharp answer to the question “Has inequality risen?” is not only unfair, but strangely incurious. If we are curious instead, and ask the right questions, deeper insight is within easy reach.