How to Make the World Add Up : Ten Rules for Thinking Differently About Numbers

by Tim Harford

First, our emotions, our preconceptions and our political affiliations are capable of badly warping the way we interpret the evidence. This problem, central to the argument of the book, is the focus of chapter one.

Emotions

It is easier when you belong to the tribe of bemused outsiders.

Bemused outsiders

confidence to assess information with curiosity and a healthy scepticism.

Curiosity

doubt is a really easy product to make.

Doubt

I worry about a world in which many people will believe anything, but I worry far more about one in which people believe nothing beyond their own preconceptions.

Preconceptions

Correlation is not causation.

Correlation and causation

Many of us refuse to look at statistical evidence because we’re afraid of being tricked.

Refuse to look

The answer to both questions is the same: when it comes to interpreting the world around us, we need to realise that our feelings can trump our expertise.

Feelings and expertese

Sometimes, we want to be fooled.

Want to be fooled

Because there is no emotional response to the claim to trip you up, you can jump straight to trying to understand and evaluate it.

Control emotions

We often find ways to dismiss evidence that we don’t like. And the opposite is true, too: when evidence seems to support our preconceptions, we are less likely to look too closely for flaws. The more extreme the emotional reaction, the harder it is to think straight.

Dismiss evidence we dont like

noticing our emotions and taking them into account may often be enough to improve our judgement.

Notice emotions in judgement

Ask yourself: how does this information make me feel? Do I feel vindicated or smug? Anxious, angry or afraid? Am I in denial, scrambling to find a reason to dismiss the claim?

Ask yourself

letting our reasoning be swayed by our hopes.

Reasoning swayed by hopes

The counterintuitive result is that presenting people with a detailed and balanced account of both sides of the argument may actually push people away from the centre rather than pull them in. If we already have strong opinions, then we’ll seize upon welcome evidence, but we’ll find opposing data or arguments irritating. This ‘biased assimilation’ of new evidence means that the more we know, the more partisan we’re able to be on a fraught issue.

Biased Assimilation

the moral difference between eating beef, eating pork and eating dog. Which of these practices you think is right and which is wrong depends mostly on your culture. Few people will care to discuss the underlying logic of the matter. It’s better to fit in.

Underlying logic of culture

unconscious bias that’s easy to see in others and very hard to see in ourselves.)*

Uunconscious bias

That’s why we need to be calm. And that is also why so much persuasion is designed to arouse us – our lust, our desire, our sympathy or our anger.

Calm

Today’s persuaders don’t want you to stop and think. They want you to hurry up and feel. Don’t be rushed.

Hurry up and feel

The first step, then, is to stop and think when we are being presented with a new piece of information, to examine our emotions and to notice if we’re straining to reach a particular conclusion.

Step 1

In a bird’s eye view you tend to survey everything . . . In a worm’s eye view you don’t have that advantage of looking at everything. You just see whatever is close to you.

Birds eye vs worms eye

Psychologists have a name for our tendency to confuse our own perspective with something more universal: it’s called ‘naive realism’, the sense that we are seeing reality as it truly is, without filters or errors.9

Naive realism

To adapt Kahneman’s terminology, they’re ‘fast statistics’ – immediate, intuitive, visceral and powerful.

Fast statistics

‘We can use photos as data,’ says Rosling Rönnlund.15 What makes them useful data rather than random and potentially misleading is that they’re sortable, comparable, and connected to the numbers.

Photos as data

We should apply the same scrutiny to policy proposals as we do to factual claims about the world. We all know that politicians like to be strategically vague.

Poititians are vague

What the psychologist Steven Pinker calls the ‘curse of knowledge’ is a constant obstacle to clear communication: once you know a subject fairly well, it is enormously difficult to put yourself in the position of someone who doesn’t know it. My

Curse of knowledge

Demanding a short, sharp answer to the question ‘Has inequality risen?’ is not only unfair, but strangely incurious. If we are curious, instead, and ask the right questions, deeper insight is within easy reach.

Right questions

RULE FOUR Step back and enjoy the view

Rule 4

So far I’ve talked about perspective mainly in terms of time. We can get useful context from other kinds of comparison, too.

Perspective

Gini coefficient of zero – there, everyone gets exactly the same income.

Gini

Andrew Elliott, an entrepreneur who likes the question so much he published a book with the title Is That a Big Number?, suggests that we should all carry a few ‘landmark numbers’ in our heads to allow easy comparison.11 A few examples:

Landmark numbers

This isn’t just about Kickstarter, of course. Such bias is everywhere. Most of the books people read are bestsellers – but most books are not bestsellers, and most book projects never become books at all. There’s a similar tale to tell about music, films and business ventures. Even cases of Covid-19 are subject to selective attention: people who feel terrible go to hospital and are tested for the disease; people who feel fine stay at home. As a result, the disease looks even more dangerous than it really is.

Iceberg

survivorship bias

Concentrating on the wrong thing

Scientists sometimes call this practice ‘HARKing’ – HARK is an acronym for Hypothesising After Results Known. To be clear, there’s nothing wrong with gathering data, poking around to find the patterns and then constructing a hypothesis. That’s all part of science. But you then have to get new data to test the hypothesis. Testing a hypothesis using the numbers that helped form the hypothesis in the first place is not OK.15

HARKING

Publication bias, and more generally the garden of forking paths, means that plenty of research that seems rigorous at first sight both to onlookers and often to the researchers themselves may instead be producing spurious conclusions.

Disbeleif

Cochrane summary

Useful

The Asch experiments are endlessly fascinating, and I often find myself discussing them in my writing and talks: they are a great starting point for a conversation about the pressure we all feel to conform, and they provide a memorable window into human nature.

Pressure to conform

Reading her book was less fun, because the incompetence and injustice she described was so depressing – from the makers of protective vests for police officers who forgot that some officers have breasts, to the coders of a ‘comprehensive’ Apple health app who overlooked that some iPhone users menstruate.5 Her book argues that all too often, the people responsible for the products and policies that shape our lives implicitly view the default customer – or citizen – as male. Women are an afterthought.

Gender data gap

Will Moy, the director of the fact-checking organisation Full Fact, points out that in England, the authorities know more about golfers than they do about people who are assaulted, robbed or raped.12

Data gathering

Sample error reflects the risk that, purely by chance, a randomly chosen sample of opinions does not reflect the true views of the population.

Sample error

sampling bias is when the sample isn’t randomly chosen at all. George Gallup took pains to find an unbiased sample because he knew that was far more important than finding a big one.

Sampling bias

Algorithms trained largely on pale faces and male voices, for example, may be confused when they later try to interpret the speech of women or the appearance of darker complexions. This is believed to help explain why Google photo software confused photographs of people with dark skin with photographs of gorillas; Hewlett Packard webcams struggled to activate when pointing at people with dark skin tones; and Nikon cameras, programmed to retake photographs if they thought someone had blinked during the shot, kept retaking shots of people from China, Japan or Korea, mistaking the distinctively east Asian eyelid fold for a blink.

Pale voices and male voices

We must always ask who and what is missing. And

Missing

The ‘winter detector’ problem is common in big data analysis. A literal example, via computer scientist Sameer Singh, is the pattern-recognising algorithm that was shown many photos of wolves in the wild, and many photos of pet husky dogs. The algorithm seemed to be really good at distinguishing the two rather similar canines; it turned out that it was simply labelling any picture with snow as containing a wolf.

Winter detection problem

In 2013, the relatively few people who were paying attention to big data often imagined themselves to be the carpenters; by 2016, many of us had realised that we were nails. Big data went from seeming transformative to seeming sinister.

Big data views changed

We all need to understand what these data are and how they can be exploited. Should big data excite or terrify us? Should we be more inclined to cheer the carpenters or worry about our unwitting role as nails?

Excite or terrify

When the woman contacted Tesco to complain, the company representatives concluded that it wasn’t their job to break the bad news that her husband was cheating on her, and went for the tactful white lie. ‘Indeed, madam? A computer error? You’re quite right, that must be the reason. We are so sorry for the inconvenience.’ Fry tells me that this is now the rule of thumb at Tesco: apologise, and blame the computer.

Blame the computer

Remember the ‘Math is Racist’ headline? I’m fairly confident that maths isn’t racist. Neither is it misogynistic, or homophobic, or biased in other ways. But I’m just as confident that some humans are. And computers trained on our own historical biases will repeat those biases at the very moment we’re trying to leave them behind us.

Humam biases

The state of Illinois introduced just such an algorithm, called Rapid Safety Feedback. It analysed data on each report, compared them to the outcomes of previous cases, and produced a percentage prediction of the child’s risk of death or serious harm. The results were not impressive. The Chicago Tribune reported that the algorithm gave 369 children a 100 per cent chance of serious injury or death. No matter how dire the home environment, that degree of certitude seems unduly pessimistic. It could also have grave implications: a false allegation of child neglect or abuse could have terrible consequences for the accused and the child alike.

Algorithm Failing

So the problem is not the algorithms, or the big datasets. The problem is a lack of scrutiny, transparency and debate. And the solution, I’d argue, goes back a very long time.

Algorithm Problem