USE A 99 PERCENT STATISTICAL CONFIDENCE LEVEL: Many tools automatically set, or let you set for yourself, the confidence levels for tests. Common levels are 95 percent and 99 percent. While the difference of four percentage points may not seem like much, it is actually quite significant from a statistical point of view. A 95 percent confidence level means that a “winning” test can still be wrong 5 percent of the time. That means that 1 out of 20 tests that come back as winners could actually be losers. At 99 percent confidence, however, that number of false-positive tests drops to 1 out of
...more
