Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets (Incerto, #1)

by Nassim Nicholas Taleb

Nero believes that risk-conscious hard work and discipline can lead someone to achieve a comfortable life with a very high probability.

Heroes are heroes because they are heroic in behavior, not because they won or lost.

The way they introduced rigor into intellectual life is by declaring that a statement could fall only into two categories: deductive, like “2 +2 =4,” i.e., incontrovertibly flowing from a precisely defined axiomatic framework (here the rules of arithmetic), or inductive, i.e., verifiable in some manner (experience, statistics, etc.), like “it rains in Spain” or “New Yorkers are generally rude.” Anything else was plain unadulterated hogwash (music could be a far better replacement to metaphysics).

Fashionable Nonsense

Selfish Gene

Turing’s test of artificial intelligence, except in reverse. What is the Turing test? The brilliant British mathematician, eccentric, and computer pioneer Alan Turing came up with the following test: A computer can be said to be intelligent if it can (on average) fool a human into mistaking it for another human. The converse should be true. A human can be said to be unintelligent if we can replicate his speech by a computer, which we know is unintelligent, and fool a human into believing that it was written by a human. Can one produce a piece of work that can be largely mistaken for Derrida entirely randomly? The answer seems to be yes.

Note that the economist Robert Lucas dealt a blow to econometrics by arguing that if people were rational then their rationality would cause them to figure out predictable patterns from the past and adapt, so that past information would be completely useless for predicting the future (the argument, phrased in a very mathematical form, earned him the Swedish Central Bank Prize in honor of Alfred Nobel).

effect. There is no point searching for patterns that are available to everyone with a brokerage account; once detected, they would be self-canceling.

Pseudoscience came with a collection of idealistic nerds who tried to create a tailor-made society, the epitome of which is the central planner.

The problem of induction, to which we will turn in the next chapter.

Treatise on Human Nature,

(as rephrased in the now famous black swan problem by John Stuart Mill): No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion.

The problem is that, without a proper method, empirical observations can lead you astray. Hume came to warn us against such knowledge, and to stress the need for some rigor in the gathering and interpretation of knowledge—what is called epistemology (from episteme, Greek for learning). Hume is the first modern epistemologist (epistemologists operating in the applied sciences are often called methodologists or philosophers of science).

You can more safely use the data to reject than to confirm hypotheses. Why? Consider the following statements: Statement A: No swan is black, because I looked at four thousand swans and found none. Statement B: Not all swans are white. I cannot logically make statement A, no matter how many successive white swans I may have observed in my life and may observe in the

swans). It is, however, possible to make Statement B merely by finding one single counterexample. Indeed, Statement A was disproved by the discovery of Australia, as it led to the sighting of the Cygnus atratus, a swan variety that was jet black!

inference. The following inductive statement illustrates the problem of interpreting past data literally, without methodology or logic: I have just completed a thorough statistical examination of the life of President Bush. For fifty-eight years, close to 21,000 observations, he did not die once. I can hence pronounce him as immortal, with a high degree of statistical significance.

Niederhoffer’s publicized hiccup came from his selling naked options based on his testing and assuming that what he saw in the past was an exact generalization about what could happen in the future. He relied on the statement “The market has never done this before,” so he sold puts that made a small income if the statement was true and lost hugely in the event of it turning out to be wrong. When he blew up, close to a couple of decades of performance were overshadowed by a single event that only lasted a few minutes.

case? If the past, by bringing surprises, did not resemble the past previous to it (what I call the past’s past), then why should our future resemble our current past?

Maximizing the probability of winning does not lead to maximizing the expectation from the game when one’s strategy may include skewness, i.e., a small chance of large loss and a large chance of a small win. If you engaged in a Russian roulette–type strategy with a low probability of large loss, one that bankrupts you every several years, you are likely to show up as the winner in almost all samples—except in the year when you are dead.

There are only two types of theories:

1. Theories that are known to be wrong, as they were tested and adequately rejected (he calls them falsified). 2. Theories that have not yet been known to be wrong, not falsified yet, but are exposed to be proved wrong.

Indeed the difference between Newtonian physics, which was falsified by Einstein’s relativity, and astrology lies in the following irony. Newtonian physics is scientific because it allowed us to falsify it, as we know that it is wrong, while astrology is not because it does not offer conditions under which we could reject it.

Accordingly, I will use statistics and inductive methods to make aggressive bets, but I will not use them to manage my risks and exposure. Surprisingly, all the surviving traders I know seem to have done the same. They trade on ideas based on some observation (that includes past history) but, like the Popperian scientists, they make sure that the costs of being wrong are limited (and their probability is not derived from past data). Unlike Carlos and John, they know before getting involved in the trading strategy which events would prove their conjecture wrong and allow for it (recall that Carlos and John used past history both to make their bets and to measure their risk). They would then terminate their trade. This is called a stop loss, a predetermined exit point, a protection from the black swan. I find it rarely practiced.

This problem enters the business world more viciously than other walks of life, owing to the high dependence on randomness (we have already belabored the contrast between randomness-dependent business with dentistry). The greater the number of businessmen, the greater the likelihood of one of them performing in a stellar manner just by luck. I have rarely seen anyone count the monkeys. In the same vein, few count the investors in the market in order to calculate, instead of the probability of success, the conditional probability of successful runs given the number of investors in operation over a given market history.

These biases can be outlined as follows: (a) The survivorship biases (a.k.a. monkeys on a typewriter) arising from the fact that we see only winners and get a distorted view of the odds (Chapters 8 and 9, “Too Many Millionaires” and “Fry an Egg”), (b) the fact that luck is most frequently the reason for extreme success (Chapter 10, “Loser Takes All”), and (c) the biological handicap of our inability to understand probability (Chapter 11, “Randomness and Our Brain”).

We will not get too involved in the Chekhovian dilemmas in the private lives of Marc and Janet, but their case provides a very common illustration of the emotional effect of survivorship bias.

As compared to the general U.S. population, Marc has done very well, better than 99.5% of his compatriots. As compared to his high school friends, he did extremely well, a fact that he could have verified had he had time to attend the periodic reunions, and he would come at the top. As compared to the other people at Harvard, he did better than 90% of them (financially, of course).

Why? Because he chose to live among the people who have been successful, in an area that excludes failure. In other words, those who have failed do not show up in the sample, thus making him look as if he were not doing well at all. By living on Park Avenue, one does not have exposure to the losers, one only sees the winners.

habitat. In the case of Marc and Janet, this leads to considerable emotional distress; here we have a woman who married an extremely successful man but all she can see is comparative failure, for she cannot emotionally compare him to a sample that would do him justice.

Aside from the misperception of one’s performance, there is a social treadmill effect: You get rich, move to rich neighborhoods, then become poor again.

try to infer some attributes that are common to rich people.

They call such people the accumulators; persons ready to postpone consumption in order to amass funds.

The moral of the book is that the wealthiest are to be found among those less suspected to be wealthy. On the other hand, those who act and look wealthy subject their net worth to such a drain that they inflict considerable and irreversible damage to their brokerage account.

I will set aside the point that I see no special heroism in accumulating money, particularly if, in addition, the person is foolish enough to not even try to derive any tangible benefit from the wealth (aside from the pleasure of regularly counting the beans).

Something about the praise lavished upon him for living in austerity while being so rich escapes me; if austerity is the end, he should become a monk or a social worker—we should remember that becoming rich is a purely selfish act, not a social one.

The first bias comes from the fact that the rich people selected for their sample are among the lucky monkeys on typewriters. The authors made no attempt to correct their statistics with the fact that they saw only the winners.

They make no mention of the “accumulators” who have accumulated the wrong things (members of my family are experts on that; those who accumulated managed to accumulate currencies about to be devalued and stocks of companies that later went bust). Nowhere do we see a mention of the fact that some people were lucky enough to have invested in the winners; these people no doubt would make their way into the book.

Virtually all of the subjects became rich from asset price inflation, in other words from the recent inflation in financial paper and assets that started in 1982.

An investor who engaged in the same strategy during less august days for the market would certainly have a different story to tell.

Consider the fate of those who, in place of spending their money buying expensive toys and paying for ski trips, bought Lebanese lira denominated Treasury bills (as my grandfather did), or junk bonds from Michael Milken (as many of my colleagues in the 1980s did). Go back in history and imagine the accumulator buying Russian Imperial bonds bearing the signature of Czar Nicholas II and trying to accumulate further by cashing them from the Soviet government, or Argentine real estate in the 1930s (as my great-grandfather did).

The mistake of ignoring the survivorship bias is chronic, even (or perhaps especially) among professionals. How? Because we are trained to take advantage of the information that is lying in front of our eyes, ignoring the information that we do not see.

A brief summing up at this point: I showed how we tend to mistake one realization among all possible random histories as the most representative one, forgetting that there may be others. In a nutshell, the survivorship bias implies that the highest performing realization will be the most visible. Why? Because the losers do not show up.

I can practically make the same statement about anyone operating in the physical world, or in a business in which the degree of randomness is low.

Well . . . the fact that he made money in the past may have some relevance, but not terribly so. This is not to say that it is always the case; there are some instances in which one can trust a track record, but,

randomness. I will probably bombard him with questions that he may not be prepared to answer, blinded by his past results. I will probably lecture him that Machiavelli ascribed to luck at least a 50% role in life (the rest was cunning and bravura), and that was before the creation of modern markets.

The concept presented here is well-known for some of its variations under the names survivorship bias, data mining, data snooping, over-fitting, regression to the mean, etc., basically situations where the performance is exaggerated by the observer, owing to a misperception of the importance of randomness.

randomness. Here we take a far simpler situation where we know the structure of randomness; the first such exercise is a finessing of the old popular saying that even a broken clock is right twice a day.

Now we run the game a second year. Again, we can expect 2,500 managers to be up two years in a row; another year, 1,250; a fourth one, 625; a fifth, 313. We have now, simply in a fair game, 313 managers who made money for five years in a row. Out of pure luck. Meanwhile if we throw one of these successful traders into the real world we would get very interesting and helpful comments on his remarkable style, his incisive mind, and the influences that helped him achieve such success. Some analysts may attribute his achievement to precise elements among his childhood experiences. His biographer will dwell on the wonderful role models provided by his parents;

The first counterintuitive point is that a population entirely composed of bad managers will produce a small amount of great track records. As a matter of fact, assuming the manager shows up unsolicited at your door, it will be practically impossible to figure out whether he is good or bad. The results would not markedly change even if the population were composed entirely of managers who are expected in the long run to lose money. Why? Because owing to volatility, some of them will make money. We can see here that volatility actually helps bad investment decisions.

Why do I use the notion of expectation of the maximum? Because I am not concerned at all with the average track record. I will get to see only the best of the managers, not all of the managers. This means that we would see more “excellent managers” in 2006 than in 1998, provided the cohort of beginners was greater in 2001 than it was in 1993—I can safely say that it was.