The Black Swan: The Impact of the Highly Improbable (Incerto, #2)

by Nassim Nicholas Taleb

I had been told in high school that the planets are in something called equilibrium, so we did not have to worry about the stars hitting us unexpectedly. To me, that eerily resembled the stories we were also told about the “unique historical stability” of Lebanon. The very idea of assumed equilibrium bothered me. I looked at the constellations in the sky and did not know what to believe.

What I call the generator of historical events is different from the events themselves, much as the minds of the gods cannot be read just by witnessing their deeds. You are very likely to be fooled about their intentions.

The human mind suffers from three ailments as it comes into contact with history, what I call the triplet of opacity. They are: the illusion of understanding, or how everyone thinks he knows what is going on in a world that is more complicated (or random) than they realize; the retrospective distortion, or how we can assess matters only after the fact, as if they were in a rearview mirror (history seems clearer and more organized in history books than in empirical reality); and the overvaluation of factual information and the handicap of authoritative and learned people, particularly when they create categories—when they “Platonify.”

Much of what took place would have been deemed completely crazy with respect to the past. Yet it did not seem that crazy after the events. This retrospective plausibility causes a discounting of the rarity and conceivability of the event. I later saw the exact same illusion of understanding in business success and the financial markets.

History Does Not Crawl, It Jumps

These events were unexplainable, but intelligent people thought

they were capable of providing convincing explanations for them—after the fact. Furthermore, the more intelligent the person, the better sounding the explanation. What’s more worrisome is that all these beliefs and accounts appeared to be logically coherent and devoid of inconsistencies.

How about the competing religion that emerged seven centuries later; who forecast that a collection of horsemen would spread their empire and Islamic law from the Indian subcontinent to Spain in just a few years? Even more than the rise of Christianity, it was the spread of Islam (the third edition, so to speak) that carried full unpredictability; many historians looking at the record have been taken aback by the swiftness of the change. Georges Duby, for one, expressed his amazement about how quickly close to ten centuries of Levantine Hellenism were blotted out “with a strike of a sword.”

History and societies do not crawl. They make jumps. They go from fracture to fracture, with a few vibrations in between. Yet we (and historians) like to believe in the predictable, small incremental progression.

While we have a highly unstable memory, a

diary provides indelible facts recorded more or less immediately; it thus allows the fixation of an unrevised perception and enables us to later study events in their own context.

Cabdrivers did not believe that they understood as much as learned people—really, they were not the experts and they knew it. Nobody knew anything, but elite thinkers thought that they knew more than the rest because they were elite thinkers, and if you’re a member of the elite, you automatically know more than the nonelite.

Categorizing is necessary for humans, but it becomes pathological when the category is seen as definitive, preventing people from considering the fuzziness of boundaries, let alone revising their categories.

What I saw was that in one of the most prestigious business schools in the world, in the most potent country in the history of the world, the executives of the most powerful corporations were coming to describe what they did for a living, and it was possible that they too did not know what was going on.

But I realized then and there that I did not give a hoot about the money. I experienced the strangest feeling I have ever had in my life, this deafening trumpet signaling to me that I was right, so loudly that it made my bones vibrate. I have never had it since and will never be able to explain it to those who have never experienced it. It was a physical sensation, perhaps a mixture of joy, pride, and terror.

And numbers are easy to process on computers. Studying historical data makes you conscious that history runs forward, not backward, and that it is messier than narrated accounts. Epistemology, the philosophy of history, and statistics aim at understanding truths, investigating the mechanisms that generate them, and separating regularity from the coincidental in historical matters.

Bonuses at the time were a fraction of what they are today, but if my employer, First Boston, and the financial system survived until year-end, I would get the equivalent of a fellowship. This is sometimes called “f*** you money,” which, in spite of its coarseness, means that it allows you to act like a Victorian gentleman, free from slavery. It is a psychological buffer: the capital is not so large as to make you spoiled-rich, but large enough to give you the freedom to choose a new occupation without excessive consideration of the financial rewards. It shields you from prostituting your mind and frees you from outside authority—any outside authority. (Independence is person-specific: I have always been taken aback at the high number of people in whom an astonishingly high income led to additional sycophancy as they became more dependent on their clients and employers and more addicted to making even more money.) While not substantial by some standards, it literally cured me of all financial ambition—it made me feel ashamed whenever I diverted time away from study for the pursuit of material wealth.

So I stayed in the quant and trading businesses (I’m still there), but organized myself to do minimal but intense (and entertaining) work, focus only on the most technical aspects, never attend business “meetings,” avoid the company of “achievers” and people in suits who don’t read books, and take a sabbatical year for every three on average to fill up gaps in my scientific and philosophical culture. To slowly distill my single idea, I wanted to become a flâneur, a professional meditator, sit in cafés, lounge, unglued to desks and organization structures, sleep as long as I needed, read voraciously, and not owe any explanation to anybody.

When your sample is large, no single instance will significantly change the aggregate or the total. The largest observation will remain impressive, but eventually insignificant, to the sum.

In Extremistan, inequalities are such that one single observation can disproportionately impact the aggregate, or the total. So while weight, height, and calorie consumption are from Mediocristan, wealth is not. Almost all social matters are from Extremistan.

Note that before the advent of modern technology, wars used to belong to Mediocristan. It is hard to kill many people if you need to slaughter them one at the time. Today, with tools of mass destruction, all it takes is a button, a nutcase, or a small error to wipe out the planet. Look at the implication for the Black Swan. Extremistan can produce Black Swans, and does, since a few occurrences have had huge influences on history. This is the main idea of this book.

Another way to rephrase the general distinction is as follows: Mediocristan is where we must endure the tyranny of the collective, the routine, the obvious, and the predicted; Extremistan is where we are subjected to the tyranny of the singular, the accidental, the unseen, and the unpredicted. As hard as you try, you will never lose a lot of weight in a single day; you need the collective effect of many days, weeks, even months.

In fact, the entire knowledge-seeking enterprise is based on taking conventional wisdom and accepted scientific beliefs and shattering them into pieces with new counterintuitive evidence, whether at a micro scale (every scientific discovery is an attempt to produce a micro–Black Swan) or at a larger one (as with Poincaré’s and Einstein’s relativity). Scientists may be in the business of laughing at their predecessors, but owing to an array of human mental dispositions, few realize that someone will laugh at their beliefs in the (disappointingly near) future. In this case, my readers and I are laughing at the present state of social knowledge.

Consider a turkey that is fed every day. Every single feeding will firm up the bird’s belief that it is the general rule of life to be fed every day by friendly members of the human race “looking out for its best interests,” as a politician would say. On the afternoon of the Wednesday before Thanksgiving, something unexpected will happen to the turkey. It will incur a revision of belief.*

This turkey problem (a.k.a. the problem of induction) is a very old one, but for some reason it is likely to be called “Hume’s problem” by your local philosophy professor.

Like many of the skeptics I hang around with, Hume was jovial and a bon vivant, eager for literary fame, salon company, and pleasant conversation. His life was not devoid of anecdotes. He once fell into a swamp near the house he was building in Edinburgh. Owing to his reputation among the locals as an atheist, a woman refused to pull him out of it until he recited the Lord’s Prayer and the Belief, which, being practical-minded, he did. But not before he argued with her about whether Christians were obligated to help their enemies. Hume looked unprepossessing. “He exhibited that preoccupied stare of the thoughtful scholar that so commonly impresses the undiscerning as imbecile,” writes a biographer.

Doubting the consequences of an outcome will allow you to remain imperturbable. The Pyrrhonian skeptics were docile citizens who followed customs and traditions whenever possible, but taught themselves to systematically doubt everything, and thus attain a level of serenity. But while conservative in their habits, they were rabid in their fight against dogma.

That awareness of a problem does not mean much—particularly when you have special interests and self-serving institutions in play.

This idea of relying on faith, not reason, was known as fideism.

Pierre-Daniel Huet wrote his Philosophical Treatise on the Weaknesses of the Human Mind in 1690, a remarkable book that tears through dogmas and questions human perception. Huet presents arguments against causality that are quite potent—he states, for instance, that any event can have an infinity of possible causes.

Above all, an erudite can be dissatisfied with his own knowledge, and such dissatisfaction is a wonderful shield against Platonicity, the simplifications of the five-minute manager, or the philistinism of the overspecialized scholar. Indeed, scholarship without erudition can lead to disasters.

In September 2006, a fund called Amaranth, ironically named after a flower that “never dies,” had to shut down after it lost close to $7 billion in a few days, the most impressive loss in trading history (another irony: I shared office space with the traders). A few days prior to the event, the company made a statement to the effect that investors should not worry because they had twelve risk managers—people who use models of the past to produce risk measures on the odds of such an event. Even if they had one hundred and twelve risk managers, there would be no meaningful difference; they still would have blown up. Clearly you cannot manufacture more information than the past can deliver; if you buy one hundred copies of The New York Times, I am not too certain that it would help you gain incremental knowledge of the future. We just don’t know how much information there is in the past.

“I never meant to say that the Conservatives are generally stupid. I meant to say that stupid people are generally Conservative,” John Stuart Mill once complained. This problem is chronic: if you tell people that the key to success is not always skills, they think that you are telling them that it is never skills, always luck.

Zoogles Are Not All Boogles All zoogles are boogles. You saw a boogle. Is it a zoogle? Not necessarily, since not all boogles are zoogles; adolescents who make a mistake in answering this kind of question on their SAT test might not make it to college. Yet another person can get very high scores on the SATs and still feel a chill of fear when someone from the wrong side of town steps into the elevator. This inability to automatically transfer knowledge and sophistication from one situation to another, or from theory to practice, is a quite disturbing attribute of human nature.

In 1971, the psychologists Danny Kahneman and Amos Tversky plied professors of statistics with statistical questions not phrased as statistical questions. One was similar to the following (changing the example for clarity): Assume that you live in a town with two hospitals—one large, the other small. On a given day 60 percent of those born in one of the two hospitals are boys. Which hospital is it likely to be? Many statisticians made the equivalent of the mistake (during a casual conversation) of choosing the larger hospital, when in fact the very basis of statistics is that large samples are more stable and should fluctuate less from the long-term average—here, 50 percent for each of the sexes—than smaller samples. These statisticians would have flunked their own exams. During my days as a quant I counted hundreds of such severe inferential mistakes made by statisticians who forgot that they were statisticians.

For another illustration of the way we can be ludicrously domain-specific in daily life, go to the luxury Reebok Sports Club in New York City, and look at the number of people who, after riding the escalator for a couple of floors, head directly to the StairMasters.

It is not feasible, in the present state of technology, to examine every single one of the patient’s cells to see if all of them are nonmalignant, so the doctor takes a sample by scanning the body with as much precision as possible. Then she makes an assumption about what she did not see. I was once taken aback when a doctor told me after a routine cancer checkup, “Stop worrying, we have evidence of cure.” “Why?” I asked. “There is evidence of no cancer” was the reply. “How do you know?” I asked. He replied, “The scan is negative.” Yet he went around calling himself doctor!

An acronym used in the medical literature is NED, which stands for No Evidence of Disease. There is no such thing as END, Evidence of No Disease.

Evidence By a mental mechanism I call naïve empiricism, we have a natural tendency to look for instances that confirm our story and our vision of the world—these instances are always easy to find.

I am saying that a series of corroborative facts is not necessarily evidence. Seeing white swans does not confirm the nonexistence of black swans. There is an exception, however: I know what statement is wrong, but not necessarily what statement is correct. If I see a black swan I can certify that not all swans are white! If I see someone kill, then I can be practically certain that he is a criminal. If I don’t see him kill, I cannot be certain that he is innocent. The same applies to cancer detection: the finding of a single malignant tumor proves that you have cancer, but the absence of such a finding cannot allow you to say with certainty that you are cancer-free.

We can get closer to the truth by negative instances, not by verification! It is misleading to build a general rule from observed facts. Contrary to conventional wisdom, our body of knowledge does not increase from a series of confirmatory observations, like the turkey’s. But there are some things I can remain skeptical about, and others I can safely consider certain. This makes the consequences of observations one-sided. It is not much more difficult than that.

It is true that a thousand days cannot prove you right, but one day can prove you to be wrong.

Popper generated a large-scale theory around this asymmetry, based on a technique called “falsification” (to falsify is to prove wrong) meant to distinguish between science and nonscience, and people immediately started splitting hairs about its technicalities, even though it is not the most interesting, or the most original, of Popper’s ideas. This idea about the asymmetry of knowledge is so liked by practitioners, because it is obvious to them; it is the way they run their business. The philosopher maudit Charles Sanders Peirce, who, like an artist, got only posthumous respect, also came up with a version of this Black Swan solution when Popper was wearing diapers—some people even called it the Peirce-Popper approach. Popper’s far more powerful and original idea is the “open” society, one that relies on skepticism as a modus operandi, refusing and resisting definitive truths. He accused Plato of closing our minds, according to the arguments I described in the Prologue. But Popper’s biggest idea was his insight concerning the fundamental, severe, and incurable unpredictability of the world, and that I will leave for the chapter on prediction.*

But it remains the case that you know what is wrong with a lot more confidence than you know what is right. All pieces of information are not equal in importance.

Popper introduced the mechanism of conjectures and refutations, which works as follows: you formulate a (bold) conjecture and you start looking for the observation that would prove you wrong. This is the alternative to our search for confirmatory instances. If you think the task is easy, you will be disappointed—few humans have a natural ability to do this. I confess that I am not one of them; it does not come naturally to me.*

As we saw earlier, disconfirming instances are far more powerful in establishing truth.

The first experiment I know of concerning this phenomenon was done by the psychologist P. C. Wason. He presented subjects with the three-number sequence 2, 4, 6, and asked them to try to guess the rule generating it. Their method of guessing was to produce other three-number sequences, to which the experimenter would respond “yes” or “no” depending on whether the new sequences were consistent with the rule. Once confident with their answers, the subjects would formulate the rule. (Note the similarity of this experiment to the discussion in Chapter 1 of the way history presents itself to us: assuming history is generated according to some logic, we see only the events, never the rules, but need to guess how it works.) The correct rule was “numbers in ascending order,” nothing more. Very few subjects discovered it because in order to do so they had to offer a series in descending order (that the experimenter would say “no” to). Wason noticed that the subjects had a rule in mind, but gave him examples aimed at confirming it instead of trying to supply series that were inconsistent with their hypothesis. Subjects tenaciously kept trying to confirm the rules that they had made up.

Similarly, the speculator George Soros, when making a financial bet, keeps looking for instances that would prove his initial theory wrong. This, perhaps, is true self-confidence: the ability to look at the world without the need to find signs that stroke one’s ego.*

The following point further illustrates the absurdity of confirmation. If you believe that witnessing an additional white swan will bring confirmation that there are no black swans, then you should also accept the statement, on purely logical grounds, that the sighting of a red Mini Cooper should confirm that there are no black swans.

Contrary to the opinion held by the great David Hume, and that of the British empiricist tradition, that belief arises from custom, as they assumed that we learn generalizations solely from experience and empirical observations, it was shown from studies of infant behavior that we come equipped with mental machinery that causes us to selectively generalize from experiences (i.e., to selectively acquire inductive learning in some domains but remain skeptical in others). By doing so, we are not learning from a mere thousand days, but benefiting, thanks to evolution, from the learning of our ancestors—which found its way into our biology.