Superforecasting Quotes

“Novice forecasters often ask why not just say 0.5, coin toss, whenever they “know nothing” about a problem. There are several reasons why not. One is the risk of being ensnared in self-contradictions. Imagine you are asked whether the Nikkei stock index will close above 20,000 by June 30, 2015. Knowing nothing, you say 0.5 chance. Now suppose you are asked whether it will close above 22,000—and you again say 0.5—or between 20,000 and 22,000, and you again say 0.5. The more possibilities the questioner unpacks, the more obvious it becomes that the casual user of 0.5 is assigning incoherent probabilities that far exceed 1.0. See Amos Tversky and Derek Koehler, “Support Theory: A Nonextensional Representation of Subjective Probability,” Psychological Review”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“For instance, people trust more confident financial advisers over those who are less confident even when their track records are identical. And people equate confidence and competence, which makes the forecaster who says something has a middling probability of happening less worthy of respect. As one study noted, people “took such judgments as indications the forecasters were either generally incompetent, ignorant of the facts in a given case, or lazy, unwilling to expend the effort required to gather information that would justify greater confidence.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“A woman living in a Kansas City suburb may think Tunisia is another planet, and her life has no connection to it, but if she were married to an air force navigator who flies out of the nearby Whiteman Air Force Base, she might be surprised to learn that one obscure Tunisian’s actions led to protests, that led to riots, that led to the toppling of a dictator, that led to protests in Libya, that led to a civil war, that led to the 2012 NATO intervention, that led to her husband dodging antiaircraft fire over Tripoli.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“More often forecasts are made and then…nothing. Accuracy is seldom determined after the fact and is almost never done with sufficient regularity and rigor that conclusions can be drawn. The reason? Mostly it’s a demand-side problem: The consumers of forecasting—governments, business, and the public—don’t demand evidence of accuracy. So there is no measurement. Which means no revision. And without revision, there can be no improvement.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“We don’t want intelligence analysts to assume jihadist groups must be inept or that vicious regimes can’t be creatively vicious.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Forecasters who see illusory correlations and assume that moral and cognitive weakness run together will fail when we need them most.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Forecasters who can’t cope with the dissonance risk making the most serious possible forecasting error in a conflict: underestimating your opponent.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“With firmness in the right, as God gives us to see the right, let us strive on to finish the work we are in,” Abraham Lincoln”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“When the facts change, I change my mind,” the legendary British economist John Maynard Keynes declared.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“Archilochus: “The fox knows many things but the hedgehog knows one big thing.”
― Philip Tetlock, Superforecasting: The Art and Science of Prediction

“disagreeable). Wise leaders know how fine the line can be between a helpful suggestion and micromanagerial meddling or between a rigid group and a decisive one or between a scatterbrained group and an open-minded one.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“Intellectual humility compels the careful reflection necessary for good judgment; confidence in one’s abilities inspires determined action.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“A forecast that is updated to reflect the latest available information is likely to be closer to the truth than a forecast that isn’t so informed.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“When the scientist tells you he does not know the answer, he is an ignorant man. When he tells you he has a hunch about how it is going to work, he is uncertain about it. When he is pretty sure of how it is going to work, and he tells you, “This is the way it’s going to work, I’ll bet,” he still is in some doubt. And it is of paramount importance, in order to make progress, that we recognize this ignorance and this doubt. Because we have the doubt, we then propose looking in new directions for new ideas. The rate of the development of science is not the rate at which you make observations alone but, much more important, the rate at which you create new things to test.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“P(H|D)/P(-H|D) = P(D|H)/P(D|-H) • P(H)/P(-H) Posterior Odds = Likelihood Ratio • Prior Odds The Bayesian belief-updating equation”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“It’s human nature. We have all been too quick to make up our minds and too slow to change them. And if we don’t examine how we make these mistakes, we will keep making them.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“bafflegab.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“point of making forecasts is not to tick all the boxes on the “how to make forecasts” checklist. It is to foresee what’s coming. To have accountability for process but not accuracy is like ensuring that physicians wash their hands, examine the patient, and consider all the symptoms, but never checking to see whether the treatment works.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“As Moltke observed, “It shakes the trust of subordinates and gives the units a feeling of uncertainty if things happen entirely differently from what orders from higher headquarters had presumed.”9”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“Everybody has said, ‘I want push-back from you if you see something I don’t,’ ” said Rosenthal. That made a difference. So did offering thanks for constructive criticism.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“People take things differently. What one person would consider a helpful inquiry another might take as an aggressive criticism.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“on average, teams were 23% more accurate than individuals.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“If forecasters can keep questioning themselves and their teammates, and welcome vigorous debate, the group can become more than the sum of its parts.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“But groups also let people share information and perspectives. That’s good. It helps make dragonfly eye work, and aggregation is critical to accuracy.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“the CIA was training Cuban exiles to land in Cuba and launch a guerrilla war against the new government of Fidel Castro.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“readers of the New York Times opened the newspaper on the kitchen table and read the front-page headline: U.S. HELPS TRAIN AN ANTI-CASTRO FORCE AT SECRET GUATEMALAN AIR-GROUND BASE”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“Special counsel Theodore Sorensen and the president’s brother Bobby were designated “intellectual watchdogs,” whose job was to “pursue relentlessly every bone of contention in order to prevent errors arising from too superficial an analysis of the issues,” Janis noted.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“After the fiasco, Kennedy ordered an inquiry to figure out how his people could have botched it so badly. It identified cozy unanimity as the key problem and recommended changes to the decision-making process to ensure it could never develop again.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“In Janis’s hypothesis, “members of any small cohesive group tend to maintain esprit de corps by unconsciously developing a number of shared illusions and related norms that interfere with critical thinking and reality testing.”3 Groups that get along too well don’t question assumptions or confront uncomfortable facts. So everyone agrees, which is pleasant, and the fact that everyone agrees is tacitly taken to be proof the group is on the right track.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction

“In his 1972 classic, Victims of Groupthink, the psychologist Irving Janis—one of my PhD advisers at Yale long ago—explored the decision making that went into both the Bay of Pigs invasion and the Cuban missile crisis.”
― Philip E. Tetlock, Superforecasting: The Art and Science of Prediction