How to Make the World Add Up : Ten Rules for Thinking Differently About Numbers

by Tim Harford

easier to argue against positions they disliked than in favour of those they supported. There was a special power in doubt.

‘A learned fool is more foolish than an ignorant one.’

need to be transparent about the data that were gathered but not published, the statistical tests that were performed but then set to one side, the clinical trials that went missing in action, and the studies that produced humdrum results and were rejected by journals or stuffed in a file drawer while researchers got on with something more fruitful.

If the story you’re reading is about health, there’s one place you should be sure to look for a second opinion: the Cochrane Collaboration.

one cure for conformity is to make decisions with a diverse group of people, people who are likely to bring different ideas and assumptions to the table.

any dataset begins with somebody deciding to collect the numbers. What numbers are and aren’t collected, what is and isn’t measured, and who is included or

excluded, are the result of all-too-human assumptions, preconceptions and oversights.

Sampling error is when a randomly chosen sample doesn’t reflect the underlying population purely by chance; sampling bias is when the sample isn’t randomly chosen at all.

Much of the data visualisation that bombards us today is decoration at best, and distraction or even disinformation at worst. The decorative function is surprisingly common, perhaps because the data visualisation teams of many media organisations are part of the art departments. They are led by people whose skills and experience are not in statistics but in illustration or graphic design.4 The emphasis is on the visualisation, not on the data. It is, above all, a picture.

19 – the number of words in the preceding sentence. I suppose this brightens up a page loaded with text, but it’s hardly an insightful use of ink. Also, the correct number is twenty-one. Never let zippy design distract you from the possibility that the underlying numbers simply might be wrong.

visualisation expert Robert Kosara suggests plotting linear data on a spiral. If there’s a periodic pattern to the data – say, repeating every seven days or every three months – that may be concealed by other fluctuations in a conventional plot but will leap out in a spiral plot.

For those of us on the receiving end of beautiful visualisations, everything we’ve learned so far in this book applies. First – and most important, since the visual sense can be so visceral – check your emotional response. Pause for a moment to notice how the graph makes you feel: triumphant, defensive, angry, celebratory? Take that feeling into account. Second, check that you understand the basics behind the graph. What do the axes actually mean? Do you understand what is being measured or counted? Do you have the context to understand, or is the graph showing just a few data points? If the graph reflects complex analysis or the results of an experiment, do you understand what is being done? If you’re not in a position to evaluate that personally, do you trust those who were? (Or have you, perhaps, sought a second opinion?) When you look at data visualisations, you’ll do much better if you recognise that someone may well be trying to persuade you of something. There is nothing wrong with artfully persuasive graphs, any more than with artfully persuasive words.

One of the reasons facts don’t always change our minds is that we are keen to avoid uncomfortable truths.

First, encouragingly for us nerds, it did help to have some training – of a particular kind. Just an hour of training in basic statistics improved the performance of forecasters by helping them turn their expertise about the world into a sensible probabilistic forecast,

The importance of the base rate was made famous by the psychologist Daniel Kahneman, who coined the phrase ‘the outside view and the inside view’. The inside view means looking at the specific case in front of you: this couple. The outside view requires you to look at a more general ‘comparison class’ of cases – here, the comparison class is all married couples. (The outside view needn’t be statistical, but it often will be.)

better to start with the statistical view, the outside view, and then modify it in the light of personal experience than it is to go the other way around. If you start with the inside view you have no real frame of reference,

no sense of scale – and can easily come up with a probability that is ten times too large, or ten times too small.

Second, keeping score was...

simple as remembering whether our earlier forecasts were right or wrong.

Third, superforecasters tended to update their forecasts frequently as new information emerged, which suggests that a receptiveness to new evidence was important.

fourth and perhaps most crucial element: superforecasting is a matter of having an open-minded personality. The superforecasters are what psychologists call ‘actively open-minded thinkers’ – people who don’t cling too tightly to a single approach, are comfortable abandoning an old view in the light of fresh evidence or new arguments, and embrace disagreements with others as an opportunity to learn. ‘For superforecasters, beliefs are hypotheses to be tested, not treasures to be guarded,’ wrote Philip Tetlock after the study had been completed. ‘It would be facile to reduce superforecasting to a bumper-sticker slogan, but if I had to, that would be it.’

This book has argued that it is possible to gather and to analyse numbers in ways that help us understand the world. But it has also argued that very often we make mistakes not because the data aren’t available, but because we refuse to accept what they are telling us. For Irving Fisher, and for many others, the refusal to accept the data was rooted in a refusal to acknowledge that the world had changed.

‘Well-placed trust grows out of active inquiry rather than blind acceptance.’

our desire to belong to a community of like-minded, right-thinking people – can, on certain hot-button issues, lead us to reach the conclusions we wish to reach.

Curiosity is fuelled once we know enough to know that we do not know.

The illusion of explanatory depth is a curiosity-killer and a trap. If we think we already understand, why go deeper? Why ask questions? It is striking that it was so easy to get people to pull back from their earlier confidence: all it took was to get them to reflect on the gaps in their knowledge. And as Loewenstein argued, gaps in knowledge fuel curiosity.

once people are interested they can understand anything in the world.

How to engage people’s interest is neither a new problem nor an intractable one. Novelists, screenwriters and comedians have been figuring out this craft for as long as they have existed. They know that we love mysteries, are drawn in by sympathetic characters, enjoy the arc of a good story, and will stick around for anything that makes us laugh. And scientific evidence suggests that Orson Welles was absolutely right: for example, studies in which people were asked to read narratives and non-narrative texts found that they zipped through the narrative at twice the speed, and recalled twice as much information later.

speaking slowly and clearly will only get you so far. To communicate complex ideas, we needed to spark people’s curiosity – even inspire a sense of wonder.

If we want to make the world add up, we need to ask questions – open-minded, genuine questions. And once we start asking them, we may find it is delightfully difficult to stop.