Superforecasting: The Art and Science of Prediction

by Philip E. Tetlock

were “like blind men arguing over the colors of the rainbow.”7

SUPERFORECASTING ISN’T A paint-by-numbers method but superforecasters often tackle questions in a roughly similar way—one that any of us can follow: Unpack the question into components. Distinguish as sharply as you can between the known and unknown and leave no assumptions unscrutinized. Adopt the outside view and put the problem into a comparative perspective that downplays its uniqueness and treats it as a special case of a wider class of phenomena. Then adopt the inside view that plays up the uniqueness of the problem. Also explore the similarities and differences between your views and those of others—and pay special attention to prediction markets and other methods of extracting wisdom from crowds. Synthesize all these different views into a single vision as acute as that of a dragonfly. Finally, express your judgment as precisely as you can, using a finely grained scale of probability.

I can’t imagine a better description of the “try, fail, analyze, adjust, try again” cycle—and of the grit to keep at it and keep improving.

Ten Commandments for Aspiring Superforecasters

(1) Triage.

(2) Break seemingly intractable problems into tractable sub-problems.

(3) Strike the right balance between inside and outside views.

(4) Strike the right balance between under- and overreacting to evidence.

(5) Look for the clashing causal forces at work in each problem.

(6) Strive to distinguish as many degrees of doubt as the problem permits but no more.

(7) Strike the right balance between under- and overconfidence, between prudence and decisiveness.

(8) Look for the errors behind your mistakes but beware of rearview-mirror hindsight biases.

(9) Bring out the best in others and let others bring out the best in you.

(10) Master the error-balancing bicycle.

(11) Don’t treat commandments as commandments.