The Data Detective: Ten Easy Rules to Make Sense of Statistics

by Tim Harford

Details of the fiasco will, no doubt, be studied for years to come. But some things already seem clear. At the beginning of the crisis, for example, politics seem to have impeded the free flow of honest statistics—a problem we’ll return to in the eighth chapter. Taiwan has complained that in late December 2019 it had given important clues about human-to-human transmission to the World Health Organization—but as late as mid-January, the WHO was reassuringly tweeting that China had found no evidence of human-to-human transmission.

Understandably, when we think about persuasion, we think about people being tricked into believing something that they shouldn’t—and we’ll discuss this problem in the next chapter.

There was a special power in doubt. Doubt is also easy to sell because it is a part of the process of scientific exploration and debate.

Doubt is a powerful weapon, and statistics are a vulnerable target. That target needs defenders. Yes, it’s easy to lie with statistics—but it’s even easier to lie without them.*

All the statistical expertise in the world will not prevent your believing claims you shouldn’t believe and dismissing facts you shouldn’t dismiss. That expertise needs to be complemented by control of your own emotional reactions to the statistical claims you see.

Benjamin Franklin commented, “So convenient a thing it is to be a reasonable creature, since it enables us to find or make a reason for everything one has a mind to do.”

Our emotional reaction to a statistical or scientific claim isn’t a side issue. Our emotions can, and often do, shape our beliefs more than any logic. We are capable of persuading ourselves to believe strange things, and to doubt solid evidence, in service of our political partisanship, our desire to keep drinking coffee, our unwillingness to face up to the reality of our HIV diagnosis, or any other cause that invokes an emotional response.

The Nobel laureate economist Friedrich Hayek had a phrase for the kind of awareness that is hard to capture in metrics and maps: the “knowledge of the particular circumstances of time and place.”

Statisticians are sometimes dismissed as bean counters. The sneering term is misleading as well as unfair. Most of the concepts that matter in policy are not like beans; they are not merely difficult to count, but difficult to define. Once you’re sure what you mean by “bean,” the bean counting itself may come more easily. But if we don’t understand the definition, then there is little point in looking at the numbers. We have fooled ourselves before we have begun.

In 2010, for instance, political scientists Brendan Nyhan and Jason Reifler published a study on what became known as “the backfire effect”—in brief, that people were more likely to believe a false claim if they’d been shown a fact-check that debunked the claim.

But Nyhan and Reifler encouraged further studies, and those studies suggest that the backfire effect is unusual and fact-checking does help.