The Skeptics' Guide to the Universe: How To Know What's Really Real in a World Increasingly Full of Fake

by Steven Novella

choosing when to stop recording data, what variables to follow, which comparisons to make, and which statistical methods to use—all decisions that researchers have to make about every study. If, however, they monitor the data or the outcomes in any way while making these decisions, they can consciously or unconsciously exploit their “degrees of freedom” to reach the magic p-value of 0.05. In fact, Simmons showed you can reach a p-value of 0.05 60 percent of the time with completely negative data. Simonsohn points out that p-values in published papers cluster suspiciously around the 0.05 level—implying that researchers were engaging in p-hacking until they reached this minimal publishable threshold.

P hacking