How to Measure Anything in Cybersecurity Risk

by Douglas W. Hubbard

There is no single, universal sample size required to be “statistically significant.” To compute it correctly, statistical significance is a function of not only sample size, but also the variance within a sample and the hypothesis being tested. These would be used to compute something called a “P-value.” This result is then compared to a stated “significance level.” Lacking those steps, the declaration of what is statistically significant cannot be trusted. Once you know not only how to compute statistical significance but also how to understand what it means, then you will find out that it isn’t even what you wanted to know in the first place. Statistical significance does not mean you learned something and the lack of statistical significance does not mean you learned nothing.