The Data Detective: Ten Easy Rules to Make Sense of Statistics

by Tim Harford

Stocks are compared with flows. That’s the equivalent of comparing the total cost of buying a house with the annual cost of renting one; it’s not a trivial confusion. Net measures are put alongside gross ones—the equivalent of comparing a firm’s profit with its turnover. The shocking difference between the before-and-after comparisons of the cost of the war in Iraq turns out to be based on an unfair comparison. (Admittedly, a fair comparison might also show a shocking difference.) The prewar number is a narrow estimate: the cost to the US military budget. The postwar number is very broad, including a figure for the cost of lost life, the cost of high oil prices, and a huge sum for the cost of macroeconomic instability, putting part of the blame for the 2008 financial crisis on the war. That broad estimate of cost is not unreasonable, but what is unreasonable is to put it alongside a very different kind of estimate without comment. What seems to be a pure before-and-after contrast is actually narrow-and-before versus broad-and-after, measuring a different thing at a different time. Nobody looking at the Debtris animation would realize that. Debtris was published in 2010, and quickly became my favorite cautionary example—the visualization is so good but the data are so bad. A