The problems are obvious. 5 per cent is an arbitrary cut-off point – why not 6 per cent, or 4 per cent? – and it encourages us to think in black-and-white, pass-or-fail terms, instead of embracing degrees of uncertainty. And if you found the previous paragraph confusing, I don’t blame you. Conceptually, statistical significance is baffling, almost backwards: it tells us the chance of observing the data given a particular theory, the theory that there is no effect. Really, we’d like to know the opposite, the probability of a particular theory being true, given the data. My own instinct is that
...more
