Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets (Incerto, #1)

by Nassim Nicholas Taleb

Generate a long series of coin flips producing heads and tails with 50% odds each and fill up sheets of paper. If the series is long enough you may get eight heads or eight tails in a row, perhaps even ten of each. Yet you know that in spite of these wins the conditional odds of getting a head or a tail is still 50%.

The deviation from the norm as seen in excess heads or excess tails is here entirely attributable to luck, in other words, to variance, not to the skills of the hypothetical player (since there is an even probability of getting either).

A result is that in real life, the larger the deviation from the norm, the larger the probability of it comin...

survive? They maximize their odds of staying in the game by taking black-swan risks (like John and Carlos)—those that fare well most of the time, but incur a risk of blowing up.

A similar misconception of probabilities arises from the random encounters one may have with relatives or friends in highly unexpected places.

Rather, we are simply testing for any encounter, with any person we have ever met in the past, and in any place we will visit during the period concerned. The probability of the latter is considerably higher, perhaps several thousand times the magnitude of the former.

When the statistician looks at the data to test a given relationship, say, to ferret out the correlation between the occurrence of a given event, like a political announcement, and stock market volatility, odds are that the results can be taken seriously.

The same mechanism is behind the formation of conspiracy theories. Like The Bible Code they can seem perfect in their logic and can cause otherwise intelligent people to fall for them.

I would then concoct a conspiratorial theory around a secret message shared by these paintings. This is seemingly what the author of the bestselling The Da Vinci Code did.

One such example came from one Paul Wilmott (an English financial mathematician of rare brilliance and irreverence) who managed to announce that I gave him his “first bad review,” yet used excerpts from it as praise on the book jacket

I am fitting the rule on the data. This activity is called data snooping.

The more I try, the more I am likely, by mere luck, to find a rule that worked on past data.

pattern. I am convinced that there exists a tradable security in the Western world that would be 100% correlated with the changes in...

Suppose that, over time, investors have experimented with technical trading rules drawn from a very wide universe—in principle thousands of parameterizations of a variety of types of rules.

. If enough trading rules are considered over time, some rules are bound by pure luck, even in a very large sample, to produce superior performance even if they do not genuinely possess predictive power over asset returns.

surviving trading rules

I have to decry some excesses in backtesting that I have closely witnessed in my private career. There

Many medical researchers are not even remotely aware of this data mining bias. True, it may only play a small role, but it is certainly present. One recent medical study links cigarette smoking to a reduction in breast cancer, thus conflicting with all previous studies.

When a company shows an increase in earnings once, it draws no immediate attention. Twice, and the name starts showing up on computer screens. Three times, and the company will merit some buy recommendation.

The twist: Life is unfair in a nonlinear way. This chapter is about how a small advantage in life can translate into a highly disproportionate payoff, or, more viciously, how no advantage at all,

What we are witnessing here is a nonlinear effect resulting from a linear force exerted on an object. A very small additional input, here the grain of sand, caused a disproportionate result, namely the destruction of my starter Tower of Babel. Popular wisdom has integrated many such phenomena, as witnessed by such expressions as “the straw that broke the camel’s back” or “the drop that caused the water to spill.”

These nonlinear dynamics have a bookstore name, “chaos theory,” which is a misnomer because it has nothing to do with chaos.

Chaos theory concerns itself primarily with functions in which a small input can lead to a...

The arrangement of the letters on a typewriter is an example of the success of the least deserving method. For our typewriters have the order of the letters on their keyboard arranged in a nonoptimal manner, as a matter of fact in such a nonoptimal manner as to slow down the typing rather than make the job easy, in order to avoid jamming the ribbons as they were designed for less electronic days.

several attempts were made to rationalize the computer keyboard, to no avail. People were trained on a QWERTY keyboard and their habits were too sticky for change. Just like the helical propulsion of an actor into stardom, people patronize what other people like to do.

This is called a path dependent outcome, and has thwarted many mathematical attem...

While early economic models excluded randomness, Arthur explained how “unexpected orders, chance meetings with lawyers, managerial whims . . . would help determine which ones achieved early sales and, over time, which firms dominated.”

winning increases after past wins, that of losing increases after past losses. Simulating such a process, one can see a huge variance of outcomes, with astonishing successes and a large number of failures (what we called skewness).

Say you played roulette and won. Would this increase your chances of winning again? No. In a Polya process case, it does.

Because the notion of independence (i.e., when the next draw does not depend on past outcomes) is violated.

Independence is a requirement for working with the (known) m...

Answer: There was a bunch of intelligent people who felt compelled to use mathematics just to tell themselves that they were rigorous in their thinking, that theirs was a science.

Studies of the dynamics of networks have mushroomed recently. They became popular with Malcolm Gladwell’s book The Tipping Point, in which he shows how some of the behaviors of variables such as epidemics spread extremely fast beyond some unspecified critical level.

Once one exits the conventional models of randomness (the bell curve family of charted randomness), something acute can happen. Why does the Internet hub Google get so many hits as compared to that of the National Association of Retired Veteran Chemical Engineers? The more connected a network, the higher the probability of someone hitting it

Our brain is not cut out for nonlinearities. People think that if, say, two variables are causally linked, then a steady input in one variable should always yield a result in the other one. Our emotional apparatus is designed for linear causality.

For instance, you study every day and learn something in proportion to your studies. If you do not feel that you are going anywhere, your emotions will cause you to become demoralized.

up, something will come to you in a flash. My partner Mark Spitznagel summarizes it as follows: Imagine yourself practicing the piano every day for a long time, barely being able to perform “Chopsticks,” then suddenly finding yourself capable of playing Rachmaninov.

event. This summarizes why there are routes to success that are nonrandom, but few, very few, people have the mental stamina to follow them. Those

react at all until some critical point. Most people give up before the rewards.

The reader no doubt has played a version of Buridan’s donkey, by “flipping a coin” to break some of the minor stalemates in life where one lets randomness help with the decision process. Let Lady Fortuna make the decision and gladly submit. I often use Buridan’s donkey (under its mathematical name) when my computer goes into a freeze between two possibilities (to be technical, these “randomizations” are frequently done during optimization problems, when one needs to perturbate a function).

I am also realizing the nonlinear effect behind success in anything: It is better to have a handful of enthusiastic advocates than hordes of people who appreciate your work—better to be loved by a dozen than liked by the hundreds.

This applies to the sales of books, the spread of ideas, and success in general and runs counter to conventional logic. The information age is worsening this effect.

You know that the two states are mutually exclusive (you can only be in one place at one time), but exhaustive (there is a 100% probability that you will be in one of them). They are equiprobable, with, in your opinion, 50% probability assigned to each. You derive great pleasure thinking about your vacation; it motivates you and makes your daily commute more bearable. But the adequate way to visualize yourself, according to rational behavior under uncertainty, is 50% in one of the vacation spots and 50% in the other—what is mathematically called a linear combination of the two states. Can your brain handle that?

Our brain can properly handle one and only one state at once—unless you have personality troubles of a deeply pathological nature. Now try to imagine an 85%/15% combination. Any luck?

Accordingly, rules have their value. We just follow them not because they are the best but because they are useful and they save time and effort. Consider that those who started theorizing upon seeing a tiger on whether the tiger was of this or that taxonomic variety, and the degree of danger it represented, ended up being eaten by it.

Clearly he got his intuitions from computer science—he spent his entire career at Carnegie-Mellon University in Pittsburgh, which has a reputation as a computer science center. “Satisficing” was his idea (the melding together of satisfy and suffice): You stop when you get a near-satisfactory solution. Otherwise it may take you an eternity to reach the smallest conclusion or perform the smallest act. We are therefore rational, but in a limited way: “boundedly rational.” He believed that our brains were a large optimizing machine that had built-in rules to stop somewhere.

Kahneman and Tversky went in a completely different direction than Simon and started figuring out rules in humans that did not make them rational—but things went beyond the shortcut. For them, these rules, which are called heuristics, were not merely a simplification of rational models, but were different in methodology and category. They called them “quick and dirty” heuristics. There is a dirty part: These shortcuts came with side effects, these effects being the biases, most of which I discussed previously throughout the text (such as the inability to accept anything abstract as risk).

A normative science (clearly a self-contradictory concept) offers prescriptive teachings; it studies how things should be. Some economists, for example those of the efficient-market religion, believe that our studies should be based on the hypothesis that humans are rational and act rationally because it is the best thing for them to do (it is mathematically “optimal”). The opposite is a positive science, which is based on how people actually are observed to behave.

physics is an inherently positive science while economics, particularly microeconomics and financial economics, is predominantly a normative one.

If your mind operates by series of different disconnected rules, these may not be necessarily consistent with each other, and if they may still do the job locally, they will not necessarily do so globally. Consider them stored as a rulebook of sorts. Your reaction will depend on which page of the book you open to at any point in time. I will illustrate it with another socialist example.