Tristan Caplan's Tetlockian Glossary, by Bryan Caplan

This summer, I assigned Tetlock and Gardner's Superforecasting (see here, here, and here) to my homeschoolers.  They took to this masterpiece of political psychology like fish to water.  I'd already taught them about human beings' wacky mapping from language to quantitative probabilities, but these passages have really stuck with them:

In March 1951 National Intelligence Estimate (NIE) 29-51 was published. 
"Although it is impossible to determine which course of action the
Kremlin is likely to adopt," the report concluded, "we believe that the
extent of [Eastern European] military and propaganda preparations
indicate that an attack on Yugoslavia in 1951 should be considered a
serious possibility." ...But a few days later, [Sherman] Kent was
chatting with a senior State Department official who casually asked, "By
the way, what did you people mean by the expression 'serious
possibility'?  What kind of odds did you have in mind?"  Kent said he
was pessimistic.  He felt the odds were about 65 to 35 in favor of an
attack.  The official was started.  He and his colleagues had taken
"serious possibility" to mean much lower odds.

Disturbed, Kent
went back to his team.  They had all agreed to use "serious possibility"
in the NIE so Kent asked each person, in turn, what he thought it
meant.  One analyst said it meant odds of about 80 to 20, or four times
more likely than not that there would be an invasion.  Another thought
it meant odds of 20 to 80 - exactly the opposite.  Other answers were
scattered between these extremes.  Kent was floored.

Ideally, analysts would adopt standard conventions on the correct way to translate from English to math.  But since that won't happen anytime soon, my son Tristan proposed a simple, effective half-measure.  Namely: every serious book (and perhaps every article) should include a Probability Glossary in the opening pages, alongside the List of Illustrations and List of Tables.  In these glossaries, each author would explicitly state what probabilities (or probability ranges) he assigns to probability-relevant English words.  For example, here are roughly the probabilities I have in mind when writing this blog:

English Term

My Probability

Absolutely certain

100%

Certain

>99%

Almost Certain

���95%

Highly Likely

>80%

Probable

>60%

Likely

>51%

Toss-up

45-55%

Unlikely

Possible

1-35%

Highly Unlikely

Almost impossible

���5%

Impossible

Absolutely impossible

0%

Filling out this table makes me self-conscious of the inadequacies of my current usage.  But it's still a big step forward in the War for Clarity.  Indeed, reading this table might make you realize you and I suffer from illusory disagreement - or illusory agreement.  And even if publishers are loathe to add a Probability Glossary, there's no reason why every thinker who wants to signal seriousness couldn't publicly post one on his blog or webpage.

When you do, please credit my son for the idea...

(12 COMMENTS)