The Undoing Project: A Friendship That Changed Our Minds

by Michael Lewis

From his stint as a consultant he learned something valuable, however. It seemed to him that a big part of a consultant’s job was to feign total certainty about uncertain things. In a job interview with McKinsey, they told him that he was not certain enough in his opinions. “And I said it was because I wasn’t certain. And they said, ‘We’re billing clients five hundred grand a year, so you have to be sure of what you are saying.’” The consulting firm that eventually hired him was forever asking him to exhibit confidence when, in his view, confidence was a sign of fraudulence. They’d asked him to forecast the price of oil for clients, for instance. “And then we would go to our clients and tell them we could predict the price of oil. No one can predict the price of oil. It was basically nonsense.”

There were a great many interesting questions in the world to which the only honest answer was, “It’s impossible to know for sure.”

Later, when basketball scouts came to him looking for jobs, the trait he looked for was some awareness that they were seeking answers to questions with no certain answers—that they were inherently fallible. “I always ask them, ‘Who did you miss?’” he said. Which future superstar had they written off, or which future bust had they fallen in love with? “If they don’t give me a good one, I’m like, ‘Fuck ’em.’”

He suggested a new definition of the nerd: a person who knows his own mind well enough to mistrust it.

“Almost everything we looked at was nonpredictive,” says Morey. But not everything.

The allure of behaviorism was that the science appeared clean: the stimuli could be observed, the responses could be recorded. It seemed “objective.”

There was a joke that captured the antiseptic spirit of behaviorism that Skinner himself liked to tell: A couple makes love. Afterward, one of them turns to the other and says, “It was good for you. How was it for me?”

All the leading behaviorists were WASPs—a fact that didn’t go unnoticed by young people entering psychology in the 1950s. Looking back, a casual observer of the field at that time couldn’t help but wonder if there shouldn’t be two entirely unrelated disciplines: “WASP Psychology” and “Jewish Psychology.” The WASPs marched around in white lab coats carrying clipboards and thinking up new ways to torture rats and all the while avoided the great wet mess of human experience. The Jews embraced the mess—even the Jews who disdained Freud’s methods and longed for “objectivity” and wished to search for the kinds of truth that might be tested according to the rules of science.

Danny was especially struck by the way that the Gestalt psychologists, in their writings, put their readers through an experience, so that they might feel for themselves the mysterious inner workings of their own minds:

The central question posed by Gestalt psychologists was the question the behaviorists had elected to ignore: How does the brain create meaning? How does it turn the fragments collected by the senses into a coherent picture of reality? Why does that picture so often seem to be imposed by the mind upon the world around it, rather than by the world upon the mind? How does a person turn the shards of memory into a coherent life story? Why does a person’s understanding of what he sees change with the context in which he sees it? Why—to speak a bit loosely—when a regime bent on the destruction of the Jews rises to power in Europe, do some Jews see it for what it is, and flee, and others stay to be slaughtered? These questions, or ones like them, had led Danny into psychology. They weren’t the sort to be answered by even the most gifted rat. Their answers, if they existed, could be found only in the human mind.

The claim turned out to be false, but Danny was so taken by it in that moment that he decided to chuck psychology to pursue a medical degree—so that he’d be allowed to poke around the human brain and see what other effects he might generate.

Meehl’s book, called Clinical versus Statistical Prediction, had shown that psychoanalysts who tried to predict what would become of their neurotic patients fared poorly compared to simple algorithms.

What he did was teach the army interviewers—young women, mainly—how to put a list of questions to each recruit to minimize the halo effect. He told them to pose very specific questions, designed to determine not how a person thought of himself but how the person had actually behaved. The questions were not just fact-seeking but designed to disguise the facts being sought. And at the end of each section, before moving on to the next, the interviewer was to assign a rating from 1 to 5 that corresponded with choices ranging from “never displays this kind of behavior” to “always displays this kind of behavior.” So, for example, when evaluating a recruit’s sociability, they’d give a 5 to a person who “forms close social relationships and identifies completely with the whole group” and a 1 to “a person who was “completely isolated.” Even Danny could see that there were all kinds of problems with his methods, but he didn’t have the time to worry too much about them. For instance, he briefly agonized over how to define a 3—was it someone who was extremely sociable on occasion, or someone who was moderately sociable all the time? Both, he basically decided. The big thing was that the judge was to keep her private opinions to herself. The question was not “What do I think of him?” but “What has he done?” The judgment of who went where in the Israeli army was to be made by Danny’s algorithm. “The interviewers hated it,” he recalled. “I had a mutiny on my hands. I still remember one o...

Later, when he was a university professor, Danny would tell students, “When someone says something, don’t ask yourself if it is true. Ask what it might be true of.” That was his intellectual instinct, his natural first step to the mental hoop: to take whatever someone had just said to him and try not to tear it down but to make sense of it.

“The difference between Danny and the next nine hundred and ninety-nine thousand nine hundred and ninety-nine psychologists is his ability to find the phenomenon and then explain it in a way that applies to other situations,” said Dale Griffin, a psychologist at the University of British Columbia. “It looks like luck but he keeps doing it.”

What all those who came to know Amos eventually realized was that the man had a preternatural gift for doing only precisely what he wanted to do.

“The nice thing about things that are urgent,” he liked to say, “is that if you wait long enough they aren’t urgent anymore.” “I would say to Amos I have to do this or I have to do that,” recalled his old friend Yeshu Kolodny. “And he would say, ‘No. You don’t.’ And I thought: lucky man!”

The trouble with philosophy, Amos thought, was that it didn’t play by the rules of science. The philosopher tested his theories of human nature on a sample size of one—himself. Psychology at least pretended to be a science. It kept at least one hand at all times on hard data. A psychologist might test whatever theory he devised on a representative sample of humanity. His theories might be tested by others, and his findings reproduced, or falsified. If a psychologist stumbled upon a truth he might make it stick.

To his assigned work he paid shockingly little attention.

By changing the context in which two things are compared, you submerge certain features and force others to the surface.

He still did only what he wanted to do. Even his new interest in wearing a suit was more peculiarly Amos than it was bourgeois. Amos chose his suits only by the number and size of the jacket pockets. Along with an interest in pockets, he had what amounted to a fetish for briefcases, and acquired dozens of them. He’d returned from five years in the most materialistic culture on the face of the earth with a desire only for objects that might help him impose order on the world around him.

Here was another way Israel was different from the United States: Its wars were short, and someone always won.

“He was like Woody Allen, without the humor.”

That was another thing colleagues and students noticed about Danny: how quickly he moved on from his enthusiasms, how easily he accepted failure. It was as if he expected it. But he wasn’t afraid of it. He’d try anything. He thought of himself as someone who enjoyed, more than most, changing his mind. “I get a sense of movement and discovery whenever I find a flaw in my thinking,” he said.

“He had no ability to see what is a waste of time and what is not,” said Dalia Etzion. “He was willing to accept anything as possibly interesting.”

“Someone once said that education was knowing what to do when you don’t know,” said one of his students. “Danny took that idea and ran with it.”

“The idea that everyone is entitled to his/her opinion was a California thing—that’s not how we did things in Jerusalem.”

Theories for Amos were like mental pockets or briefcases, places to put the ideas you wanted to keep. Until you could replace a theory with a better theory—a theory that better predicted what actually happened—you didn’t chuck a theory out. Theories ordered knowledge, and allowed for better prediction.

The new studies being made about how people’s minds worked when rendering dispassionate judgments had ignored what was known about how the mind worked when it was doing other things.

Amos was a one-man wrecking ball for illogical arguments; when Danny heard an illogical argument, he asked, What might that be true of?

Danny was a pessimist. Amos was not merely an optimist; Amos willed himself to be optimistic, because he had decided pessimism was stupid. When you are a pessimist and the bad thing happens, you live it twice, Amos liked to say. Once when you worry about it, and the second time when it happens.

“Belief in the Law of Small Numbers” teased out the implications of a single mental error that people commonly made—even when those people were trained statisticians. People mistook even a very small part of a thing for the whole. Even statisticians tended to leap to conclusions from inconclusively small amounts of evidence. They did this, Amos and Danny argued, because they believed—even if they did not acknowledge the belief—that any given sample of a large population was more representative of that population than it actually was.

“Even the fairest coin, however, given the limitations of its memory and moral sense, cannot be as fair as the gambler expects it to be,” they wrote. In an academic journal that line counted as a splendid joke.

If the mind, when it was making probabilistic judgments about an uncertain world, was not an intuitive statistician, what was it?

A lot of psychologists at the time, including Danny, were using sample sizes of 40 subjects, which gave them only a 50 percent chance of accurately reflecting the population. To have a 90 percent chance of capturing the traits of the larger population, the sample size needed to be at least 130. To gather a larger sample of course required a lot more work, and thus slowed a research career.

Psychology had long been an intellectual garbage bin for problems and questions that for whatever reason were not welcome in other academic disciplines.

Amos liked to say that if you are asked to do anything—go to a party, give a speech, lift a finger—you should never answer right away, even if you are sure that you want to do it. Wait a day, Amos said, and you’ll be amazed how many of those invitations you would have accepted yesterday you’ll refuse after you have had a day to think it over. A corollary to his rule for dealing with demands upon his time was his approach to situations from which he wished to extract himself. A human being who finds himself stuck at some boring meeting or cocktail party often finds it difficult to invent an excuse to flee. Amos’s rule, whenever he wanted to leave any gathering, was to just get up and leave. Just start walking and you’ll be surprised how creative you will become and how fast you’ll find the words for your excuse, he said.

A prediction is a judgment that involves uncertainty.

Here was a useful way of thinking about base rates: They were what you would predict if you had no information at all.

“Evidently, people respond differently when given no specific evidence and when given worthless evidence,” wrote Danny and Amos. “When no specific evidence is given, the prior probabilities are properly utilized; when worthless specific evidence is given, prior probabilities are ignored.”*

That is, once they knew the outcome, they thought it had been far more predictable than they had found it to be before, when they had tried to predict it. A few years after Amos described the work to his Buffalo audience, Fischhoff named the phenomenon “hindsight bias.”†

The problem was not what they knew, or didn’t know. It was their need for certainty or, at least, the appearance of certainty. Standing beside the slide projector, many of them did not so much teach as preach. “There was a generalized mood of arrogance,” said Redelmeier. “ ‘What do you mean you didn’t give steroids!!????’” To Redelmeier the very idea that there was a great deal of uncertainty in medicine went largely unacknowledged by its authorities. There was a reason for this: To acknowledge uncertainty was to admit the possibility of error. The entire profession had arranged itself as if to confirm the wisdom of its decisions. Whenever a patient recovered, for instance, the doctor typically attributed the recovery to the treatment he had prescribed, without any solid evidence that the treatment was responsible. Just because the patient is better after I treated him doesn’t mean he got better because I treated him, Redelmeier thought. “So many diseases are self-limiting,” he said. “They will cure themselves. People who are in distress seek care. When they seek care, physicians feel the need to do something. You put leeches on; the condition improves. And that can propel a lifetime of leeches. A lifetime of overprescribing antibiotics. A lifetime of giving tonsillectomies to people with ear infections. You try it and they get better the next day and it is so compelling. You go to see a psychiatrist and your depression improves—you are convinced of the efficacy of psychia...

The secret to doing good research is always to be a little underemployed. You waste years by not being able to waste hours.

It is sometimes easier to make the world a better place than to prove you have made the world a better place.

“He needs the concrete examples to test his general theories,” said Redelmeier. “Some of the principles were just extremely robust, and I was supposed to find examples and give voice to them in a particular domain, medicine.”

For arthritis, selective matching leads people to look for changes in the weather when they experience increased pain, and pay little attention to the weather when their pain is stable. . . . [A] single day of severe pain and extreme weather might sustain a lifetime of belief in a relation between them.”

When they made decisions, people did not seek to maximize utility. They sought to minimize regret.

“Always keep one hand firmly on data,” Amos liked to say. Data was what set psychology apart from philosophy, and physics from metaphysics.

For most people, the happiness involved in receiving a desirable object is smaller than the unhappiness involved in losing the same object.”

Amos liked to call good ideas “raisins.” There were three raisins in the new theory. The first was the realization that people responded to changes rather than absolute levels. The second was the discovery that people approached risk very differently when it involved losses than when it involved gains. Exploring people’s responses to specific gambles, they found a third raisin: People did not respond to probability in a straightforward manner. Amos and Danny already knew, from their thinking about regret, that in gambles that offered a certain outcome, people would pay dearly for that certainty. Now they saw that people reacted differently to different degrees of uncertainty.