Superforecasting Quotes

“With training or experience, people can encode patterns deep in their memories in vast number and intricate detail—such as the estimated fifty thousand to one hundred thousand chess positions that top players have in their repertoire.20 If something doesn’t fit a pattern—like a kitchen fire giving off more heat than a kitchen fire should—a competent expert senses it immediately. But as we see every time someone spots the Virgin Mary in burnt toast or in mold on a church wall, our pattern-recognition ability comes at the cost of susceptibility to false positives. This, plus the many other ways in which the tip-of-your-nose perspective can generate perceptions that are clear, compelling, and wrong, means intuition can fail as spectacularly as it can work.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“The explanatory urge is mostly a good thing. Indeed, it is the propulsive force behind all human efforts to comprehend reality. The problem is that we move too fast from confusion and uncertainty (“I have no idea why my hand is pointed at a picture of a shovel”) to a clear and confident conclusion (“Oh, that’s simple”) without spending any time in between (“This is one possible explanation but there are others”).”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“consider a young Tunisian man pushing a wooden handcart loaded with fruits and vegetables down a dusty road to a market in the Tunisian town of Sidi Bouzid. When the man was three, his father died. He supports his family by borrowing money to fill his cart, hoping to earn enough selling the produce to pay off the debt and have a little left over. It’s the same grind every day. But this morning, the police approach the man and say they’re going to take his scales because he has violated some regulation. He knows it’s a lie. They’re shaking him down. But he has no money. A policewoman slaps him and insults his dead father. They take his scales and his cart. The man goes to a town office to complain. He is told the official is busy in a meeting. Humiliated, furious, powerless, the man leaves. He returns with fuel. Outside the town office he douses himself, lights a match, and burns. Only the conclusion of this story is unusual. There are countless poor street vendors in Tunisia and across the Arab world. Police corruption is rife, and humiliations like those inflicted on this man are a daily occurrence. They matter to no one aside from the police and their victims. But this particular humiliation, on December 17, 2010, caused Mohamed Bouazizi, aged twenty-six, to set himself on fire, and Bouazizi’s self-immolation sparked protests. The police responded with typical brutality. The protests spread. Hoping to assuage the public, the dictator of Tunisia, President Zine el-Abidine Ben Ali, visited Bouazizi in the hospital. Bouazizi died on January 4, 2011. The unrest grew. On January 14, Ben Ali fled to a cushy exile in Saudi Arabia, ending his twenty-three-year kleptocracy. The Arab world watched, stunned. Then protests erupted in Egypt, Libya, Syria, Jordan, Kuwait, and Bahrain. After three decades in power, the Egyptian dictator Hosni Mubarak was driven from office. Elsewhere, protests swelled into rebellions, rebellions into civil wars. This was the Arab Spring—and it started with one poor man, no different from countless others, being harassed by police, as so many have been, before and since, with no apparent ripple effects. It is one thing to look backward and sketch a narrative arc, as I did here, connecting Mohamed Bouazizi to all the events that flowed out of his lonely protest. Tom Friedman, like many elite pundits, is skilled at that sort of reconstruction, particularly in the Middle East, which he knows so well, having made his name in journalism as a New York Times correspondent in Lebanon. But could even Tom Friedman, if he had been present that fatal morning, have peered into the future and foreseen the self-immolation, the unrest, the toppling of the Tunisian dictator, and all that followed? Of course not. No one could. Maybe, given how much Friedman knew about the region, he would have mused that poverty and unemployment were high, the number of desperate young people was growing, corruption was rampant, repression was relentless, and therefore Tunisia and other Arab countries were powder kegs waiting to blow. But an observer could have drawn exactly the same conclusion the year before. And the year before that. Indeed, you could have said that about Tunisia, Egypt, and several other countries for decades. They may have been powder kegs but they never blew—until December 17, 2010, when the police pushed that one poor man too far.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“The one undeniable talent that talking heads have is their skill at telling a compelling story with conviction, and that is enough. Many have become wealthy peddling forecasting of untested value to corporate executives, government officials, and ordinary people who would never think of swallowing medicine of unknown efficacy and safety but who routinely pay for forecasts that are as dubious as elixirs sold from the back of a wagon.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“the more famous an expert was, the less accurate he was.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Doubt is not a fearful thing,” Feynman observed, “but a thing of very great value.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“We have all been too quick to make up our minds and too slow to change them.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“I have been struck by how important measurement is to improving the human condition,” Bill Gates wrote. “You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal. … This may seem basic, but it is amazing how often it is not done and how hard it is to get right.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“it turns out that forecasting is not a “you have it or you don’t” talent. It is a skill that can be cultivated.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“forecast Steve Ballmer made in 2007, when he was CEO of Microsoft: “There’s no chance that the iPhone is going to get any significant market share. No chance.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“what makes these superforecasters so good. It’s not really who they are. It is what they do. Foresight isn’t a mysterious gift bestowed at birth. It is the product of particular ways of thinking, of gathering information, of updating beliefs. These habits of thought can be learned and cultivated”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Flash back to early 2012. How likely is the Assad regime to fall? Arguments against a fall include (1) the regime has well-armed core supporters; (2) it has powerful regional allies. Arguments in favor of a fall include (1) the Syrian army is suffering massive defections; (2) the rebels have some momentum, with fighting reaching the capital. Suppose you weight the strength of these arguments, they feel roughly equal, and you settle on a probability of roughly 50%. But notice what’s missing? The time frame. It obviously matters. To use an extreme illustration, the probability of the regime falling in the next twenty-four hours must be less—likely a lot less—than the probability that it will fall in the next twenty-four months. To put this in Kahneman’s terms, the time frame is the “scope” of the forecast. So we asked one randomly selected group of superforecasters, “How likely is it that the Assad regime will fall in the next three months?” Another group was asked how likely it was in the next six months. We did the same experiment with regular forecasters. Kahneman predicted widespread “scope insensitivity.” Unconsciously, they would do a bait and switch, ducking the hard question that requires calibrating the probability to the time frame and tackling the easier question about the relative weight of the arguments for and against the regime’s downfall. The time frame would make no difference to the final answers, just as it made no difference whether 2,000, 20,000, or 200,000 migratory birds died. Mellers ran several studies and found that, exactly as Kahneman expected, the vast majority of forecasters were scope insensitive. Regular forecasters said there was a 40% chance Assad’s regime would fall over three months and a 41% chance it would fall over six months.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“we showed them how to tactfully dissect the vague claims people often make. Suppose someone says, “Unfortunately, the popularity of soccer, the world’s favorite pastime, is starting to decline.” You suspect he is wrong. How do you question the claim? Don’t even think of taking a personal shot like “You’re silly.” That only adds heat, not light. “I don’t think so” only expresses disagreement without delving into why you disagree. “What do you mean?” lowers the emotional temperature with a question but it’s much too vague. Zero in. You might say, “What do you mean by ‘pastime’?” or “What evidence is there that soccer’s popularity is declining? Over what time frame?” The answers to these precise questions won’t settle the matter, but they will reveal the thinking behind the conclusion so it can be probed and tested.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Once we know the outcome of something, that knowledge skews our perception of what we thought before we knew the outcome: that’s hindsight bias. Baruch Fischhoff was the first to document the phenomenon in a set of elegant experiments. One had people estimate the likelihood of major world events at the time of Fischhoff’s research—Will Nixon personally meet with Mao?—then recall their estimate after the event did or did not happen. Knowing the outcome consistently slanted the estimate, even when people tried not to let it sway their judgment. The effect can be subtle, but it can also be quite big. In 1988, when the Soviet Union was implementing major reforms that had people wondering about its future, I asked experts to estimate how likely it was that the Communist Party would lose its monopoly on power in the Soviet Union in the next five years. In 1991 the world watched in shock as the Soviet Union disintegrated. So in 1992–93 I returned to the experts, reminded them of the question in 1988, and asked them to recall their estimates. On average, the experts recalled a number 31 percentage points higher than the correct figure. So an expert who thought there was only a 10% chance might remember herself thinking there was a 40% or 50% chance. There was even a case in which an expert who pegged the probability at 20% recalled it as 70%—which illustrates why hindsight bias is sometimes known as the “I knew it all along” effect.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“The Yale professor Dan Kahan has done much research showing that our judgments about risks—Does gun control make us safer or put us in danger?—are driven less by a careful weighing of evidence than by our identities, which is why people’s views on gun control often correlate with their views on climate change, even though the two issues have no logical connection to each other. Psycho-logic trumps logic. And when Kahan asks people who feel strongly that gun control increases risk, or diminishes it, to imagine conclusive evidence that shows they are wrong, and then asks if they would change their position if that evidence were handed to them, they typically say no. That belief block is holding up a lot of others. Take it out and you risk chaos, so many people refuse to even imagine it.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Natural as such thinking may be, it is problematic. Lay out the tangled chain of reasoning in a straight line and you see this: “The probability that I would meet the love of my life was tiny. But it happened. So it was meant to be. Therefore the probability that it would happen was 100%.” This is beyond dubious. It’s incoherent. Logic and psycho-logic are in tension.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“I’d rather be a bookie than a goddamn poet,” was his legendary response.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Politics and the English Language,” George Orwell concluded with six emphatic rules, including “never use a long word where a short one will do” and “never use the passive where you can use the active.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Keynes is always ready to contradict not only his colleagues but also himself whenever circumstances make this seem appropriate,” reported a 1945 profile of the “consistently inconsistent” economist. “So far from feeling guilty about such reversals of position, he utilizes them as pretexts for rebukes to those he saw as less nimble-minded. Legend says that while conferring with Roosevelt at Quebec, Churchill sent Keynes a cable reading, ‘Am coming around to your point of view.’ His Lordship replied, ‘Sorry to hear it. Have started to change my mind.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“To illustrate, imagine”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“WE ARE ALL forecasters. When we think about changing jobs, getting married, buying a home, making an investment, launching a product, or retiring, we decide based on how we expect the future will unfold. These expectations are forecasts. Often we do our own forecasting.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“day-to-day fluctuations in the profits of existing investments, which are obviously of an ephemeral and nonsignificant character, tend to have an altogether excessive, and even an absurd, influence on the market.”12”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“do. In 1814 the French mathematician and astronomer Pierre-Simon Laplace took this dream to its logical extreme: We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“there is no such thing as failure. Failure is just life trying to move us in another direction….Learn from every mistake because every experience, encounter, and particularly your mistakes are there to teach you and force you into being who you are.” Everything happens for a reason. Everything has a purpose.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“produce forecast-wrecking”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“we are serious about measuring and improving, this won’t do. Forecasts must have clearly defined terms and timelines. They must use numbers. And one more thing is essential: we must have lots of forecasts.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“So finding meaning in events is positively correlated with well-being but negatively correlated with foresight. That sets up a depressing possibility: Is misery the price of accuracy?”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“superforecasting demands thinking that is open-minded, careful, curious, and—above all—self-critical. It also demands focus. The kind of thinking that produces superior judgment does not come effortlessly. Only the determined can deliver it reasonably consistently, which is why our analyses have consistently found commitment to self-improvement to be the strongest predictor of performance.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Superforecasting isn’t a paint-by-numbers method but superforecasters often tackle questions in a roughly similar way—one that any of us can follow: Unpack the question into components. Distinguish as sharply as you can between the known and unknown and leave no assumptions unscrutinized. Adopt the outside view and put the problem into a comparative perspective that downplays its uniqueness and treats it as a special case of a wider class of phenomena. Then adopt the inside view that plays up the uniqueness of the problem. Also explore the similarities and differences between your views and those of others—and pay special attention to prediction markets and other methods of extracting wisdom from crowds. Synthesize all these different views into a single vision as acute as that of a dragonfly. Finally, express your judgment as precisely as you can, using a finely grained scale of probability.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“For those who hope that we can become collectively wiser, it was a bewildering fracas that looked less like a debate between great minds and more like a food fight between rival fraternities.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction