Superforecasting: The Art and Science of Prediction

by Philip E. Tetlock

No, what made it gutsy was what it could reveal. Here’s one possible revelation: Imagine you get a couple of hundred ordinary people to forecast geopolitical events. You see how often they revise their forecasts and how accurate those forecasts prove to be and use that information to identify the forty or so who are the best. Then you have everyone make lots more forecasts. This time, you calculate the average forecast of the whole group—“the wisdom of the crowd”—but with extra weight given to those forty top forecasters. Then you give the forecast a final tweak: You “extremize” it, meaning you push it closer to 100% or zero. If the forecast is 70% you might bump it up to, say, 85%. If it’s 30%, you might reduce it to 15%.9 Now imagine that the forecasts you produce this way beat those of every other group and method available, often by large margins. Your forecasts even beat those of professional intelligence analysts inside the government who have access to classified information—by margins that remain classified. Think how shocking it would be to the intelligence professionals who have spent their lives forecasting geopolitical events—to be beaten by a few hundred ordinary people and some simple algorithms. It actually happened. What I’ve described is the method we used to win IARPA’s tournament.