The Undoing Project: A Friendship That Changed Our Minds

by Michael Lewis

“In making predictions and judgments under uncertainty,” they wrote, “people do not appear to follow the calculus of chance or the statistical theory of prediction. Instead, they rely on a limited number of heuristics which sometimes yield reasonable judgments and sometimes lead to severe and systematic error.”

Man’s inability to see the power of regression to the mean leaves him blind to the nature of the world around him. We are exposed to a lifetime schedule in which we are most often rewarded for punishing others, and punished for rewarding.

When Richard Nixon announced his surprising intention to visit China and Russia, Fischhoff asked people to assign odds to a list of possible outcomes—say, that Nixon would meet Chairman Mao at least once, that the United States and the Soviet Union would create a joint space program, that a group of Soviet Jews would be arrested for attempting to speak with Nixon, and so on. After the trip, Fischhoff went back and asked the same people to recall the odds they had assigned to each outcome. Their memories of the odds they had assigned to various outcomes were badly distorted. They all believed that they had assigned higher probabilities to what happened than they actually had. They greatly overestimated the odds that they had assigned to what had actually happened. That is, once they knew the outcome, they thought it had been far more predictable than they had found it to be before, when they had tried to predict it. A few years after Amos described the work to his Buffalo audience, Fischhoff named the phenomenon “hindsight bias.”

Effective example: American about-face regarding Iraq war sentiment.

All too often, we feel like kicking ourselves for failing to foresee that which later appears inevitable. For all we know, the handwriting might have been on the wall all along. The question is: was the ink visible?

“Creeping determinism,” he called it—and jotted in his notes one of its many costs: “He who sees the past as surprise-free is bound to have a future full of surprises.”

“It was a classic case of the representativeness heuristic,” said Redelmeier. “You need to be so careful when there is one simple diagnosis that instantly pops into your mind that beautifully explains everything all at once. That’s when you need to stop and check your thinking.” It wasn’t that what first came to mind was always wrong; it was that its existence in your mind led you to feel more certain than you should be that it was correct.

An immune system responding as it should generated fever, coughs, chills, sputum—and a faster than normal heartbeat. A body fighting an infection required blood to be pumped through it at a faster than normal rate. “The heart rate of an old man with pneumonia is not supposed to be normal!” said Redelmeier. “It’s supposed to be ripping along!” An old man with pneumonia whose heart rate appears normal is an old man whose heart may well have a serious problem. But the normal reading on the heart rate monitor created a false sense in doctors’ minds that all was well. And it was precisely when all seemed well that medical experts “failed to check themselves.”

When Redelmeier entered medical school in 1980, for instance, the conventional wisdom held that if a heart attack victim suffered from some subsequent arrhythmia, you gave him drugs to suppress it. By the end of Redelmeier’s medical training, seven years later, researchers had shown that heart attack patients whose arrhythmia was suppressed died more often than the ones whose condition went untreated. No one explained why doctors, for years, had opted for a treatment that systematically killed patients—though proponents of evidence-based medicine were beginning to look to the work of Kahneman and Tversky for possible explanations.

Later Redelmeier said as much to one of his fellow students, an American. What is it with you freedom-loving Americans? he asked. Live free or die. I don’t get it. I say, “Regulate me gently. I’d rather live.” His fellow student replied, Not only do a lot of Americans not share your view; other physicians don’t share your view. Redelmeier’s fellow student told him about Stanford’s famous head of cardiac surgery, Norm Shumway, who had actively lobbied against the creation of a law that would require motorcyclists to wear helmets. “It dropped my jaw,”

Lung cancer proved to be a handy example. Lung cancer doctors and patients in the early 1980s faced two unequally unpleasant options: surgery or radiation. Surgery was more likely to extend your life, but, unlike radiation, it came with the small risk of instant death. When you told people that they had a 90 percent chance of surviving surgery, 82 percent of patients opted for surgery. But when you told them that they had a 10 percent chance of dying from the surgery—which was of course just a different way of putting the same odds—only 54 percent chose the surgery. People facing a life-and-death decision responded not to the odds but to the way the odds were described to them.

What did it mean if people’s prediction of the misery that might be caused by some event was different from the misery they actually experienced when the event occurred, or if people’s memory of an experience turned out to be meaningfully different from the experience as it had actually played out? A lot, thought Danny. People had a miserable time for most of their vacation and then returned home and remembered it fondly; people enjoyed a wonderful romance but, because it ended badly, looked back on it mainly with bitterness. They didn’t simply experience fixed levels of happiness or unhappiness. They experienced one thing and remembered something else.

Danny wanted Redelmeier to find him a real-world medical example of what he was calling the “peak-end rule.” It didn’t take long for Redelmeier to come up with a bunch, but they settled on colonoscopies. In the late 1980s, colonoscopies were painful, and not merely dreaded. The discomfort of the procedure dissuaded people from returning for another one. By 1990, colon cancer was killing sixty thousand people every year in the United States alone. Many of its victims would have survived had their cancer been detected at an early stage. One of the big reasons colon cancer went undetected was that people found their first colonoscopy so unpleasant that they elected not to return for a second one. Was it possible to alter their memory of the experience so that they might forget how unpleasant it was?

“No one ever made a decision because of a number. They need a story.”

What was the point of laying out the odds of a gamble, if the person taking it either didn’t believe the numbers or didn’t want to know them? The trouble, Danny suspected, was that “the understanding of numbers is so weak that they don’t communicate anything. Everyone feels that those probabilities are not real—that they are just something on somebody’s mind.”

The marginal value of the dollars you give up to buy fire insurance on your house is less than the marginal value of the dollars you lose if your house burns down—which is why even though the insurance is, strictly speaking, a stupid bet, you buy it.

When they made decisions, people did not seek to maximize utility. They sought to minimize regret.

One rule was that the emotion was closely linked to the feeling of “coming close” and failing. The nearer you came to achieving a thing, the greater the regret you experienced if you failed to achieve it.

A second rule: Regret was closely linked to feelings of responsibility. The more control you felt you had over the outcome of a gamble, the greater the regret you experienced if the gamble turned out badly.

Today Jack and Jill each have a wealth of 5 million. Yesterday, Jack had 1 million and Jill had 9 million. Are they equally happy? (Do they have the same utility?)

Two decades later, in 1995, the American psychologist Thomas Gilovich, who collaborated in turn with Danny and Amos, coauthored a study that examined the relative happiness of silver and bronze medal winners at the 1992 Summer Olympics. From video footage, subjects judged the bronze medal winners to be happier than the silver medal winners. The silver medalists, the authors suggested, dealt with the regret of not having won gold, while the bronze medalists were just happy to be on a podium.

Amos and Danny already knew, from their thinking about regret, that in gambles that offered a certain outcome, people would pay dearly for that certainty. Now they saw that people reacted differently to different degrees of uncertainty. When you gave them one bet with a 90 percent chance of working out and another with a 10 percent chance of working out, they did not behave as if the first was nine times as likely to work out as the second. They made some internal adjustment, and acted as if a 90 percent chance was actually slightly less than a 90 percent chance, and a 10 percent chance was slightly more than a 10 percent chance. They responded to probabilities not just with reason but with emotion.

But when you framed the sure thing as a loss, people chose the gamble. When you framed it as a gain, people picked the sure thing. The reference point—the point that enabled you to distinguish between a gain and a loss— wasn’t some fixed number. It was a psychological state.

If you could calculate what people needed to be paid to accept a 1 percent chance of being killed on the job, you could, in theory, work out what you’d need to pay them to accept a 100 percent chance of being killed on the job. (The number he came up with was $1.4 million, in 2016 dollars.)

Instead, Thaler began to keep a list. On the list were a lot of irrational things people do that economists claim that they don’t do, because economists presume that people are rational. At the top of the list was their willingness to pay 100 times more to avoid a one-in-a-thousand chance of being infected with an incurable disease than they were for the cure for that same disease, after they already had a one-in-a-thousand chance of having it.

A lot of the items on it fell into a bucket that he eventually would label “The Endowment Effect.” The endowment effect was a psychological idea with economic consequences. People attached some strange extra value to whatever they happened to own, simply because they owned it, and so proved surprisingly reluctant to part with their possessions, or endowments, even when trading them made economic sense.

Danny now had an idea that there might be a fourth heuristic—to add to availability, representativeness, and anchoring. “The simulation heuristic,” he’d eventually call it, and it was all about the power of unrealized possibilities to contaminate people’s minds. As they moved through the world, people ran simulations of the future.

flights, at the same time. They traveled from town in the same limousine, were caught in the same traffic jam, and arrived at the airport thirty minutes after the scheduled departure time of their flights. Mr. Crane is told that his flight left on time. Mr. Tees is told that his flight was delayed, and just left five minutes ago. Who is more upset? The situation of the two men was identical. Both expected to miss their planes and both had. And yet 96 percent of the subjects to whom Danny put the question said that Mr. Tees was more upset. Everyone seemed to understand that reality wasn’t the only source of frustration. The emotion was also fed by its proximity to another reality—how “close” Mr. Tees came to making his flight.

counterfactual emotions,” or the feelings that spurred people’s minds to spin alternative realities in order to avoid the pain of the emotion. Regret was the most obvious counterfactual emotion, but frustration and envy shared regret’s essential trait.

A middle-aged banker takes the same route to work every day. One day he takes a different route and is killed when a drugged-out kid in a pickup truck runs a red light and sideswipes his car. Ask people to undo the tragedy, and their minds drift to the route the banker took that day. If only he had gone the usual way! But put that same man back on his normal route, and let him be killed by the same drugged-out boy in the same truck, running a different stoplight, and no one thought: If only he had taken a different route that day! The distance the mind needed to travel from the usual way of doing things to some less ordinary way of doing things felt further than the trip made from the other direction.

“The older guys had this idea that we were going to rescue the image of psychology in the Soviet Union,” recalled Wandell. “Psychology flew in the face of Marxism. It was on the list of things that didn’t need to exist.”

It took about a day to realize why Marxism might feel that way. These particular Soviet psychologists were charlatans. “We were thinking there really were going to be scientists on the Soviet end,” said Wandell. “There weren’t.” The Soviets and the Americans took turns giving presentations. An American would give a learned talk about decision theory. His Soviet counterpart would rise and offer a talk that sounded completely insane—one guy spent his allotted time on his theory about how the brain waves caused by beer canceled the brain waves caused by vodka. “We’d get up and give a paper, and you know, it was okay,” said Wandell. “Then some Russian guy would get up and talk and we’d say, ‘Wow, that was weird.’ One was about how the meaning of life could be put into a formula and the formula might have some variable labeled E in it.”

“The major point of shadow theory,” Amos wrote to himself, “is that the context of alternatives or the possibility set determines our expectations, our interpretations, our recollection and our attribution of reality, as well as the affective states which it induces.” Toward the end of his thinking on the subject, he summed up a lot in a single sentence: “Reality is a cloud of possibility, not a point.”

“Because metaphors are vivid and memorable, and because they are not readily subjected to critical analysis, they can have considerable impact on human judgment even when they are inappropriate, useless, or misleading,” said Amos. “They replace genuine uncertainty about the world with semantic ambiguity. A metaphor is a cover-up.”

People were blind to logic when it was embedded in a story. Describe a very sick old man and ask people: Which is more probable, that he will die within a week or die within a year? More often than not, they will say, “He’ll die within a week.” Their mind latches onto a story of imminent death and the story masks the logic of the situation.

Amos created a lovely example. He asked people: Which is more likely to happen in the next year, that a thousand Americans will die in a flood, or that an earthquake in California will trigger a massive flood that will drown a thousand Americans? People went with the earthquake.

Eighty-five percent still insisted that Linda was more likely to be a bank teller in the feminist movement than she was to be a bank teller. The Linda problem resembled a Venn diagram of two circles, but with one of the circles wholly contained by the other. But people didn’t see the circles. Danny was actually stunned.

After all, what is a marriage if not an agreement to distort one’s perception of another, in relation to everyone else?

He also downplayed or ignored most of their evidence, and all of their strongest evidence. He did what critics sometimes do: He described the object of his scorn as he wished it to be rather than as it was. Then he debunked his description.

The mind was more like a coping mechanism than it was a perfectly designed tool. “The brain appears to be programmed, loosely speaking, to provide as much certainty as it can,” he once said, in a talk to a group of Wall Street executives. “It is apparently designed to make the best possible case for a given interpretation rather than to represent all the uncertainty about a given situation.” The mind, when it dealt with uncertain situations, was like a Swiss Army knife. It was a good enough tool for most jobs required of it, but not exactly suited to anything—and certainly not fully “evolved.” “Listen to evolutionary psychologists long enough,” Amos said, “and you’ll stop believing in evolution.”

“Prospect Theory,” scarcely cited in the first decade after its publication, would become, by 2010, the second most cited paper in all of economics.

A paper Thaler had titled in his mind “Stupid Shit That People Do” he’d finally published as “Toward a Positive Theory of Consumer Choice.”

In the school year after he left the White House, about 40 percent more poor kids ate free school lunches than had done so before, back when they or some adult acting on their behalf had to take action and make choices to get them.

On May 29, Israel held an election for prime minister, and a militaristic Benjamin Netanyahu defeated Shimon Peres. “So I won’t see peace in my lifetime,” Amos said, upon hearing the news. “But I was never going to see peace in my lifetime.”