The Signal and the Noise: The Art and Science of Prediction

by Nate Silver

The equilibrium proposed by Stiglitz is one in which some minimal profits are available to some investors. Efficient-market hypothesis can’t literally be true. Although some studies (like mine of mutual funds on E*Trade) seem to provide evidence for Fama’s view that no investor can beat the market at all, others are more equivocal,87 and a few88 identify fairly tangible evidence of trading skill and excess profits.

“Investors need to learn how to do exactly the reverse of what their fight-or-flight mechanism is telling them to do,” Blodget told me. “When the market crashes, that is the time to get excited and put your money into it. It’s not the time to get scared and pull money out. What you see instead is the more the market drops, the more money comes out of it. Normal investors are obliterated, because they continuously do exactly the wrong thing.”

The cognitive shortcuts that our mind takes—our heuristics—are what get investors into trouble. The idea that something going up will continue to go up couldn’t be any more instinctive. It just happens to be completely wrong when it comes to the stock market.

Our instincts related to herding may be an even more fundamental problem.

Fisher Black estimated that markets are basically rational 90 percent of the time. The other 10 percent of the time, the noise traders dominate—and they can go a little haywire.97 One way to look at this is that markets are usually very right

My view on trading markets (and toward free-market capitalism more generally) is the same as Winston Churchill’s attitude toward democracy.100 I think it’s the worst economic system ever invented—except for all the other ones. Markets do a good job most of the time, but I don’t think we’ll ever be rid of bubbles.

And tests like Shiller’s P/E ratio have been quite reliable indicators of bubbles in the past.

Whatever range of abilities we have acquired, there will always be tasks sitting right at the edge of them. If we judge ourselves by what is hardest for us, we may take for granted those things that we do easily and routinely.

Bayes’s theorem begins and ends with a probabilistic expression of the likelihood of a real-world event. It does not require you to believe that the world is intrinsically uncertain.

It does require you to accept, however, that your subjective perceptions of the world are approximations of the truth.

Bayes’s theorem encourages us to be disciplined about how we weigh new information. If our ideas are worthwhile, we ought to be willing to test them by establishing falsifiable hypotheses and subjecting them to a prediction.

The more often you are willing to test your ideas, the sooner you can begin to avoid these problems and learn from your mistakes. Staring at the ocean and waiting for a flash of insight is how ideas are generated in the movies.