The Signal and the Noise: Why So Many Predictions Fail-but Some Don't

by Nate Silver

This is not to suggest that our priors always dominate the new evidence, however, or that Bayes’s theorem inherently produces counterintuitive results.

researcher John P. A. Ioannidis. In 2005, Ioannidis published an influential paper, “Why Most Published Research Findings Are False,”40 in which he cited a variety of statistical and theoretical arguments to claim that (as his title implies) the majority of hypotheses deemed to be true in journals in medicine and most other academic and scientific professions are, in fact, false.

The idea behind frequentism is that uncertainty in a statistical problem results exclusively from collecting data among just a sample of the population rather than the whole population.

But perhaps the bigger problem is the way that Fisher’s statistical philosophy tends to conceive of the world. It emphasizes the objective purity of the experiment—every hypothesis could be tested to a perfect conclusion if only enough data were collected.

The viewer was led to believe that poker is easy to learn, easy to profit from, and incredibly action-packed—none of which is true.

If the most important technical skill in poker is learning how to forecast your opponent’s hand range, the next-most-important one is making your own play unpredictable.

One interesting thing about poker is that the very best players and the very worst ones both play quite randomly, although for different

This relationship also holds in many other disciplines in which prediction is vital. The first 20 percent often begins with having the right data, the right technology, and the right incentives. You need to have some information—more of it rather than less, ideally—and you need to make sure that it is quality-controlled.

But there is no reason to allege a conspiracy when an explanation based on rational self-interest will suffice:

Scott Armstrong, a professor at the Wharton School at the University of Pennsylvania, is such a skeptic. He is also among the small group of people who have devoted their lives to studying forecasting. His book, Principles of Forecasting, should be considered canonical to anybody who is seriously interested in the field.

The goal of any predictive model is to capture as much signal as possible and as little noise as possible.

An admonition like “The more complex you make the model the worse the forecast gets” is equivalent to saying “Never add too much salt to the recipe.” How much complexity—how much salt—did you begin with?

the Maunder Minimum, a period of about seventy years during the late seventeenth and early eighteenth centuries when there was very little sunspot activity, may have triggered cooler temperatures in Europe and North America.

The eruption of Mount Pinatubo in 1991 reduced global temperatures by about 0.2°C for a period of two years, equivalent to a decade’s worth of greenhouse warming.

As we found, there is about a 15 percent chance that there will be no net warming over a decade even if the global warming hypothesis is true because of the variability in the climate.