even when there’s nothing going on, you’ll still regularly get ‘significant’ p-values, especially if you run lots of statistical tests.52 It’s analogous to the ‘psychics’ who make thousands of predictions about what’ll happen in the next year, then at the end of that year highlight only the ones they got correct, making it look as if they have magical forecasting skills.53 Roll the statistical dice enough times and something will show up as significant, even if it’s just a freak accident in your data.54 Hide away all the times it wasn’t significant and you have the perfect recipe to convince
...more
