Superforecasting: The Art and Science of Prediction

by Philip E. Tetlock

Important as calibration is, it’s not the whole story because “perfect calibration” isn’t what we think of when we imagine perfect forecasting accuracy. Perfection is godlike omniscience. It’s saying “this will happen” and it does, or “this won’t happen” and it doesn’t. The technical term for this is “resolution.” The two figures here show how calibration and resolution capture distinct facets of good judgment. The figure on top represents perfect calibration but poor resolution. It’s perfect calibration because when the forecaster says there is a 40% chance something will happen, it happens 40% of the time, and when she says there is a 60% chance something will happen, it happens 60% of the time. But it’s poor resolution because the forecaster never strays out of the minor-shades-of-maybe zone between 40% and 60%.