Superforecasting: The Art and Science of Prediction

by Philip E. Tetlock

Only when the probabilities were closer to even did they easily grasp that the outcome may or may not happen, Rubin said. “If you say something is 60/40, people kind of get the idea.”13

“Uncertainty is real,” Byers writes. “It is the dream of total certainty that is an illusion.”15

If nothing is certain, it follows that the two- and three-setting mental dials are fatally flawed. Yes and no express certainty. They have to go. The only setting that remains is maybe—the one setting people intuitively want to avoid.

As the legendary investor Charlie Munger sagely observed, “If you don’t get this elementary, but mildly unnatural, mathematics of elementary probability into your repertoire, then you go through a long life like a one-legged man in an ass-kicking contest.”23

Winfrey made essentially the same point in explicitly religious language: “I understand the manifestation of grace and God, so I know there are no coincidences. There are none. Only divine order here.”

Meaning is a basic human need.

Science doesn’t tackle “why” questions about the purpose of life. It sticks to “how” questions that focus on causation and probabilities.

And to the extent that we allow our thoughts to move in the direction of fate, we undermine our ability to think probabilistically.

A probabilistic thinker will be less distracted by “why” questions and focus on “how.” This is no semantic quibble. “Why?” directs us to metaphysics; “How?” sticks with physics.

“You tend to believe that history played out in a logical sort of sense, that people ought to have foreseen, but it’s not like that,” he told me. “It’s an illusion of hindsight.”30

For both the superforecasters and the regulars, we also compared individual fate scores with Brier scores and found a significant correlation

Is misery the price of accuracy?

Social psychologists have long known that getting people to publicly commit to a belief is a great way to freeze it in place, making it resistant to change. The stronger the commitment, the greater the resistance.8

Psycho-logic trumps logic.

That belief block is holding up a lot of others. Take it out and you risk chaos, so many people refuse to even imagine it. When a block is at the very base of the tower, there’s no way to remove it without bringing everything crashing down. This extreme commitment leads to extreme reluctance to admit error,

the dilution effect remains a bias.

They sway in the wind, at the mercy of the next random gust of irrelevant information. Such swaying is overreaction, a common and costly mistake.

Even John Maynard Keynes—he may not have said those famous words but he really did urge people to change their minds in light of changing facts—felt that “day-to-day fluctuations in the profits of existing investments, which are obviously of an ephemeral and nonsignificant character, tend to have an altogether excessive, and even an absurd, influence on the market.”12

And yet, superforecasters often manage to avoid both errors. They wouldn’t be superforecasters if they didn’t. So how do they do it? In the nineteenth century, when prose was never complete without a sage aside to Greek mythology, any discussion of two opposing dangers called for Scylla and Charybdis. Scylla was a rock shoal off the coast of Italy. Charybdis was a whirlpool on the coast of Sicily, not far away. Sailors knew they would be doomed if they strayed too far in either direction.

Forecasters should feel the same about under- and overreaction to new information, the Scylla and Charybdis of forecasting. Good updating is all about finding the middle passage.

In his famous essay “Politics and the English Language,” George Orwell concluded with six emphatic rules, including “never use a long word where a short one will do” and “never use the passive where you can use the active.” But the sixth rule was the key: “Break any of these rules sooner than saying anything outright barbarous.”

And they would rather break the rules than make a barbarous forecast.

The psychologist Carol Dweck would say Simpson has a “growth mindset,” which Dweck defines as believing that your abilities are largely the product of effort—that you can “grow” to the extent that you are willing to work hard and learn.2 Some

Many people have what she calls a “fixed mindset”—the belief that we are who we are, and abilities can only be revealed, not created and developed.

“There is no harm in being sometimes wrong, especially if one is promptly found out,” he wrote in 1933.6

For Keynes, failure was an opportunity to learn—to identify mistakes, spot new alternatives, and try again.

Keynes operated on a higher plane than most of us, but that process—try, fail, analyze, adjust, try again—is fundamental to how all of us learn, almost from the moment we are born.

“Managing is like holding a dove in your hand. If you hold it too tightly you kill it, but if you hold it too loosely, you lose it.”4