The Signal and the Noise: Why So Many Predictions Fail-but Some Don't

by Nate Silver

The cognitive shortcuts that our mind takes—our heuristics—are what get investors into trouble. The idea that something going up will continue to go up couldn’t be any more instinctive. It just happens to be completely wrong when it comes to the stock market. Our instincts related to herding may be an even more fundamental problem. Oftentimes, it will absolutely be right to do what everyone else is doing, or at least to pay some attention to it. If you travel to a strange city and need to pick a restaurant for dinner, you probably want to select the one that has more customers, other things being equal. But occasionally it will backfire: you wind up at the tourist trap. Likewise, when we were making forecasts in Bayesland, it usually behooved us to pay some attention to our neighbors and adjust our beliefs accordingly, rather than adopt the stubborn and often implausible notion that we knew better than everyone else. I pay quite a bit of attention to what the consensus view is—what a market like Intrade is saying—when I make a forecast. It is never an absolute constraint. But the further I move away from that consensus, the stronger my evidence has to be before I come to the view that I have things right and everyone else has it wrong.