Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth

by Stuart Ritchie

Running a study with low statistical power is like setting out to look for distant galaxies with a pair of binoculars: even if what you’re looking for is definitely out there, you have essentially no chance of seeing it. Sadly, this point seems to have passed many scientists by, not least in Macleod’s chosen field of animal research. A 2013 review looked across a variety of neuroscientific studies, including, for example, research on sex differences in the ability of mice to navigate mazes.43 To have enough statistical power to detect the typically expected sex effect in maze-navigating performance, a study would require 134 mice; it’s a much subtler effect, in other words, than ‘men weigh more than women’. But the average study the researchers looked at included only twenty-two mice. This isn’t specific to mice in mazes: it seems to be a problem across most types of neuroscience.44 Large-scale reviews have also found that underpowered research is rife in medical trials, biomedical research more generally, economics, brain imaging, nursing research, behavioural ecology and – quelle surprise – psychology.45