Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth

by Stuart Ritchie

Similarly disheartening findings come from surveys in other fields. A survey of biomedical statisticians in 2018 found that 30 per cent had been asked by their scientist clients to ‘interpret the statistical findings on the basis of expectations, not the actual results’ while 55 per cent had been asked to ‘stress only significant findings but underreport nonsignificant ones’.67 32 per cent of economists in another survey admitted to ‘present[ing] empirical findings selectively so that they confirm[ed their] argument’, while 37 per cent said they had ‘stopped statistical analysis when they had [their] result’ – that is, whenever p happened to drop below 0.05, even if it might’ve done so by chance.68 If you collect together all the p-values from published papers and graph them by size, you see a strange, sudden bump just under 0.05 – there are a lot more 0.04s, 0.045s, 0.049s, and so on, than you’d expect by chance. This isn’t knockdown proof, but it’s a red flag for p-hacking: scientists appear to tweak studies just enough to limbo their results under the 0.05 line, then send them off for publication.69