How to Make the World Add Up : Ten Rules for Thinking Differently About Numbers

by Tim Harford

Of course, we shouldn’t be credulous, but the antidote to credulity isn’t to believe nothing, but to have the confidence to assess information with curiosity and a healthy scepticism.

But Lord and his colleagues discovered something more surprising: the more detail people were presented with – graphs, research methods, commentary by other fictional academics – the easier they found it to disbelieve unwelcome evidence. If doubt is the weapon, detail is the ammunition.

My own instinct is that statistical significance is an unhelpful concept and we could do better, but others are more cautious. John Ioannidis – he of the ‘Most Published Research Findings Are False’ paper – argues that despite the flaws of the method, it’s ‘a convenient obstacle to unfounded claims’. Unfortunately, there is no single clever statistical technique that would make all these problems evaporate. The journey towards more rigorous science requires many steps, and we at least are taking some of them. I recently had the chance to interview Richard Thaler, a Nobel Memorial Prize winner in economics, who has collaborated with Daniel Kahneman and many other psychologists. He struck me as well placed to evaluate psychology as a sympathetic outsider. ‘I think the replication crisis has been great for psychology,’ he told me. ‘There’s just better hygiene.’36 Brian Nosek, meanwhile, told the BBC: ‘I think if we do another large reproducibility project five years from now, we are going to see a dramatic improvement in reproducibility in the field.’

Economist Shelly Lundberg and colleagues studied what happened in the UK when in 1977, child benefit, a regular subsidy to families, was switched from being a tax credit (usually to the father) to a cash payment to the mother. That shift measurably increased spending on women’s and children’s clothes relative to men’s clothes.

‘For superforecasters, beliefs are hypotheses to be tested, not treasures to be guarded,’ wrote Philip Tetlock after the study had been completed. ‘It would be facile to reduce superforecasting to a bumper-sticker slogan, but if I had to, that would be it.’14 And if even that is too long for the bumper sticker, what about this: superforecasting means being willing to change your mind.

More striking was that when the illusion faded, political polarisation also started to fade. People who would have instinctively described their political opponents as wicked, and who would have gone to the barricades to defend their own ideas, tended to be less strident when forced to admit to themselves that they didn’t fully understand what it was they were so passionate about in the first place. The experiment influenced actions as well as words: researchers found that people became less likely to give money to lobby groups or other organisations which supported the positions they had once favoured.