Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth

by Stuart Ritchie

The size of an effect (representing, for example, how much taller Scottish men are than Scottish women; in our case the effect size is 10 cm) is a different thing from the probability of seeing these results by chance if your hypothesis isn’t true. It’s perfectly possible, for example, that a drug has a very minor impact on an illness, but one that you’re quite sure isn’t a false positive – a small yet statistically significant effect. Back when Fisher was writing, people understood the word ‘significant’ somewhat differently: it implied that the result ‘signified’ that something was happening in the data, not that whatever was happening was necessarily noteworthy.