Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth

by Stuart Ritchie

Outside of exceptions such as the Higgs Boson, though, the 0.05 threshold remains, through conformity, tradition and inertia, the most widely used criterion today. It has scientists feverishly rifling through their statistical tables, checking for p-values lower than 0.05 so that they can report their results as being statistically significant. It’s easy to forget the arbitrariness. Richard Dawkins has bemoaned the ‘discontinuous mind’: our human tendency to think in terms of distinct, sharply defined categories rather than the messy, blurry, ambiguous way the world really is.25 One example is the debate around abortion, which often fixates on the question of when an embryo or foetus becomes ‘a person’, as if there could ever be a bright line by which we could make that decision. In Dawkins’s own field of evolutionary biology, pinpointing the exact moment when one species evolves into another is likewise a fool’s errand, no matter how satisfied we might feel if we could do so. It’s the same for p-values: the 0.05 cut-off for statistical significance encourages researchers to think of results below it as somehow ‘real’, and those above it as hopelessly ‘null’. But 0.05 is as much a convention as the 17-degree taps-aff rule – or, slightly more seriously, the societal decision that someone legally becomes an adult precisely on a particular birthday.