Superforecasting: The Art and Science of Prediction

by Philip E. Tetlock

I call them superforecasters because that is what they are. Reliable evidence proves it. Explaining why they’re so good, and how others can learn to do what they do, is my goal in this book.

The one undeniable talent that talking heads have is their skill at telling a compelling story with conviction, and that is enough.

easiest to beat chance on the shortest-range questions that only required looking one year out, and accuracy fell off the further out experts tried to forecast—

it is possible to see into the future, at least in some situations and to some extent, and that any intelligent, open-minded, and hardworking person can cultivate the requisite skills.

kleptocracy.

Kleptocracy is a form of government where those in power, typically corrupt leaders, use their positions to systematically steal from the public and enrich themselves. This is often achieved through embezzlement, bribery, and abuse of public funds, ultimately harming the wider population.

How predictable something is depends on what we are trying to predict, how far into the future, and under what circumstances.

skill at judging how high-stakes events are likely to unfold three months, six months, a year, or a year and a half in advance.

Foresight isn’t a mysterious gift bestowed at birth. It is the product of particular ways of thinking, of gathering information, of updating beliefs.

if it’s possible to improve foresight simply by measuring, and if the rewards of improved foresight are substantial, why isn’t measuring standard practice?

the psychology that convinces us we know things we really don’t

For centuries, it hobbled progress in medicine.

physicians finally accepted that their experience and perceptions were not reliable means of determining whether a treatment works, they turned to scientific testing

what it takes to test forecasting as rigorously as modern medicine

what they thought didn’t matter. It was how they thought.

superforecasting demands thinking that is open-minded, careful, curious, and—above all—self-critical. It also demands focus. The kind of thinking that produces superior judgment does not come effortlessly. Only the determined can deliver it reasonably consistently, which is why our analyses have consistently found commitment to self-improvement to be the strongest predictor of performance.

in most cases statistical algorithms beat subjective judgment, and in the handful of studies where they don’t, they usually tie.

we will need to blend computer-based forecasting and subjective judgment in the future.

We have all been too quick to make up our minds and too slow to change them.

never conducted anything resembling a modern experiment. Why should he? Experiments are what people do when they aren’t sure what the truth is.

“All who drink of this treatment recover in a short time, except those whom it does not help, who all die,” he wrote. “It is obvious, therefore, that it fails only in incurable cases.”

Snap judgments are sometimes essential.

A defining feature of intuitive judgment is its insensitivity to the quality of the evidence on which the judgment is based.

stock market closes and a journalist says something like “The Dow rose ninety-five points today on news that…” A quick check will often reveal that the news that supposedly drove the market came out well after the market had risen. But that minimal level of scrutiny is seldom applied.

The problem is that we move too fast from confusion and uncertainty

to a clear and confident conclusion

Scientists must be able to answer the question “What would convince me I am wrong?” If they can’t, it’s a sign they have grown too attached to their beliefs.

scientific caution runs against the grain of human nature.

our natural inclination is to grab on to the first plausible explanation and happily gather supportive evidence without checking its reliability.

confirmation bias. We rarely seek out evidence that undercuts our first explanation, and when that evidence is shoved under our noses we become motivated skeptics—finding reasons, howeve...

declarations of high confidence mainly tell you that an individual has constructed a coherent story in his mind, not necessarily that the story is true.”

another mental process was likely at work, as well. Formally, it’s called attribute substitution, but I call it bait and switch:

pattern recognition. With training or experience, people can encode patterns deep in their memories in vast number and intricate detail—such as the estimated fifty thousand to one hundred thousand chess positions that top players have in their repertoire.

But there is a catch. As Kahneman and Klein noted, it’s often hard to know when there are enough valid cues to make intuition work. And even where it clearly can, caution is advisable.

But then the train of history hit a curve, and as Karl Marx once quipped, when that happens, the intellectuals fall off.

a forecast without a time frame is absurd. And yet, forecasters routinely make them,