The Signal and the Noise: Why So Many Predictions Fail-but Some Don't

by Nate Silver

The instinctual shortcut that we take when we have “too much information” is to engage with it selectively, picking out the parts we like and ignoring the remainder, making allies with those who have made the same choices and enemies of the rest.

We face danger whenever information growth outpaces our understanding of how to process it.

Risk greases the wheels of a free-market economy; uncertainty grinds them to a halt.

that in a market plagued by asymmetries of information, the quality of goods will decrease and the market will come to be dominated by crooked sellers and gullible or desperate buyers.

When there is an excess of greed in the system, there is a bubble. When there is an excess of fear, there is a panic.

Most commercial weather forecasts are biased, and probably deliberately so. In particular, they are biased toward forecasting more precipitation than will actually occur43—what meteorologists call a “wet bias.”

A related doctrine known as Goodhart’s law, after the London School of Economics professor who proposed it,38 holds that once policy makers begin to target a particular variable, it may begin to lose its value as an economic indicator.

the economic forecasts produced by the White House, for instance, have historically been among the least accurate of all,69 regardless of whether it’s a Democrat or a Republican in charge.)

Under some circumstances, in fact, it may be quite rational for traders to take positions that lose money for their firms and their investors if it allows them to stay with the herd and reduces their chance of getting fired.70 There is significant theoretical and empirical evidence71 for herding behavior among mutual funds and other institutional investors.

As John Maynard Keynes said, “The market can stay irrational longer than you can stay solvent.”

When a possibility is unfamiliar to us, we do not even think about it.

An unknown unknown is a contingency that we have not even considered.

The more eagerly we commit to scrutinizing and testing our theories, the more readily we accept that our knowledge of the world is uncertain, the more willingly we acknowledge that perfect prediction is impossible, the less we will live in fear of our failures, and the more liberty we will have to let our minds flow freely. By knowing more about what we don’t know, we may get a few more predictions right.