Goodreads helps you keep track of books you want to read.

Start by marking “Superforecasting: The Art and Science of Prediction” as Want to Read:

Other editions

Enlarge cover

Rate this book

Clear rating

1 of 5 stars 2 of 5 stars 3 of 5 stars 4 of 5 stars 5 of 5 stars

Open Preview

Superforecasting: The Art and Science of Prediction

by Philip E. Tetlock, Dan Gardner

4.14 · Rating details · 5,611 Ratings · 511 Reviews

A New York Times Bestseller

An Economist Best Book of 2015

"The most important book on decision making since Daniel Kahneman's Thinking, Fast and Slow."
—Jason Zweig, The Wall Street Journal

Everyone would benefit from seeing further into the future, whether buying stocks, crafting policy, launching a new product, or simply planning the week’s meals. Unfortunately, people A New York Times Bestseller

An Economist Best Book of 2015

"The most important book on decision making since Daniel Kahneman's Thinking, Fast and Slow."
—Jason Zweig, The Wall Street Journal

Everyone would benefit from seeing further into the future, whether buying stocks, crafting policy, launching a new product, or simply planning the week’s meals. Unfortunately, people tend to be terrible forecasters. As Wharton professor Philip Tetlock showed in a landmark 2005 study, even experts’ predictions are only slightly better than chance. However, an important and underreported conclusion of that study was that some experts do have real foresight, and Tetlock has spent the past decade trying to figure out why. What makes some people so good? And can this talent be taught?

In Superforecasting, Tetlock and coauthor Dan Gardner offer a masterwork on prediction, drawing on decades of research and the results of a massive, government-funded forecasting tournament. The Good Judgment Project involves tens of thousands of ordinary people—including a Brooklyn filmmaker, a retired pipe installer, and a former ballroom dancer—who set out to forecast global events. Some of the volunteers have turned out to be astonishingly good. They’ve beaten other benchmarks, competitors, and prediction markets. They’ve even beaten the collective judgment of intelligence analysts with access to classified information. They are "superforecasters."

In this groundbreaking and accessible book, Tetlock and Gardner show us how we can learn from this elite group. Weaving together stories of forecasting successes (the raid on Osama bin Laden’s compound) and failures (the Bay of Pigs) and interviews with a range of high-level decision makers, from David Petraeus to Robert Rubin, they show that good forecasting doesn’t require powerful computers or arcane methods. It involves gathering evidence from a variety of sources, thinking probabilistically, working in teams, keeping score, and being willing to admit error and change course. Superforecasting offers the first demonstrably effective way to improve our ability to predict the future—whether in business, finance, politics, international affairs, or daily life—and is destined to become a modern classic. ...more

Get A Copy

Or buy for

Hardcover, 352 pages

Published September 29th 2015 by Crown (first published 2015)

More Details... edit details

Friend Reviews

To see what your friends thought of this book, please sign up.

Reader Q&A

To ask other readers questions about Superforecasting, please sign up.

Recent Questions

How much overlap does this book have with Nate Silver's book?

See 1 question about Superforecasting…

Lists with This Book

Between the World and Me by Ta-Nehisi Coates

The Economist - Books of the Year 2015

46 books — 34 voters

Thinking, Fast and Slow by Daniel Kahneman

The Better Angels of Our Nature by Steven Pinker

Thoughts on the Self

111 books — 13 voters

More lists with this book...

Community Reviews

(showing 1-30)

filter | sort: default (?) | Rating Details

Aug 03, 2015 Yannick Serres rated it really liked it · review of another edition

During the first hundred pages, I was sure to give the book a perfect score. It totally caught my attention and made me want more and more. The book made me feel like it had been written for me, someone that don't know much about predictions and forecasts, but feels like he could be good at it.

Then, after the half of the book, you get a little bored because it always come back to the same thing: Use number to make your predictions in a well established timeframe, always question your predictions ...more

flag 30 likes · Like · see review

Oct 14, 2015 Elizabeth Theiss rated it it was amazing

When it comes to forecasting, most pundits and professionals do little better than chimps with dartboards, according to Phillip Tetlock, who ought to know because he has spent a good deal of his life keeping track. Tetlock has partnered with Dan Gardner, an excellent science journalist, to write this engaging book about the 2 percent of forecasters who manage to consistently outperform their peers.

Oddly, consumers of forecasts generally do not require evidence of accuracy. Few television networ When it comes to forecasting, most pundits and professionals do little better than chimps with dartboards, according to Phillip Tetlock, who ought to know because he has spent a good deal of his life keeping track. Tetlock has partnered with Dan Gardner, an excellent science journalist, to write this engaging book about the 2 percent of forecasters who manage to consistently outperform their peers.

Oddly, consumers of forecasts generally do not require evidence of accuracy. Few television networks or web sites score the accuracy of forecasts. Years ago, as a stockbroker I gave very little weight to the forecasts of my firm's experts; the stocks they recommended were as likely to go down as they were to go up. Today, as an occasional television pundit, I'm often asked to forecast electoral outcomes, so I was very curious about Tetlock's 2 percent that managed "superforecasting."

"How predictable something is depends on what we are trying to predict, how far into the future, and under what circumstances," according to Tetlock and Gardner. It makes no sense to try to predict the economy ten years from now, for example. But he wanted to understand how the best forecasters manage to maintain accuracy over the course of many predictions. In order to find out, he launched the Good Judgement Project, which involved 2800 volunteer forecasters who worked on a series of prediction problems over several years. After the first year, he identified the best forecasters and put them on teams to answer questions like whether Arafat was poisoned by polonium, whether WMDs were in Iraq and whether Osama bin Laden was in Abbottabad. His findings shed light on the kind of evidence-based, probabilistic, logical thought processes that go into the best predictions. A section on group think is nicely illustrated by the Bay of Pigs disaster; the ability of JFK's team to learn from their mistakes is demonstrated by the same group's more skillful response to the Cuban missile crisis.

Written in an engaging and accessible style, Superforecasting illustrates every concept with a good story, often featuring national surprises like 9/11 and the lack of WMDs in Iraq with explanations of why forecasters missed what looks obvious in hindsight. Ultimately, this is a book about critical thinking that challenges the reader to bring more rigor to his or her own thought processes. Tetlock and Gardner have made a valuable contribution to a world of internet factoids and snap judgments.
...more

flag 12 likes · Like · see review

Mar 16, 2016 Pavlo Illashenko rated it it was amazing

Harry Truman famously said
: Give me a one-handed economist! All my economics say, ''On the one hand? on the other.''

Philip Tetlock combines three major findings from different areas of research:

1) People don't like experts who are context specific and could not provide us with clear simple answers regarding complex phenomena in a probabilistic world. People don't like if an expert sounds not 100% confident. They reason, that confidence represents skills.

2) Experts who employ publicly acceptable ...more

flag 8 likes · Like · see review

Jan 11, 2016 Michael rated it liked it · review of another edition

Shelves: non-fiction, audiobook

Philip E. Tetlock feels a bit too polite. Sometimes it seems he is excusing wrong predictions by finding weasel words in them or interpreting them kindly instead of using the intended assertion.
Just say, Thomas Friedman is a bad forecaster.
Instead of reading this book I recommend reading the books he references:
Thinking, Fast and Slow The Black Swan: The Impact of the Highly Improbable and The Signal and the Noise: Why So Many Predictions Fail - But Some Don't
This books feels like a (superficial ...more

flag 7 likes · Like · see review

Dec 29, 2016 Atila Iamarino rated it it was amazing

Shelves: mente, comportamento, previsao

Me surpreendeu positivamente. Tinha comprado este livro em 2015 e nem lembrava o que me motivou. Não me arrependi.

O livro começa com a explicação de como e porque a maioria dos especialistas em previsão política e similares geralmente estão errados. Muitas vezes, mais errados do que tentativas aleatórias de prever o futuro (o anedótico chimpanzé com um dardo).

Philip E. Tetlock fez parte de um longo estudo chamado The Good Judgment Project, onde os participantes passaram anos fazendo predições ...more

flag 7 likes · Like · see review

Dec 02, 2015 John Kaufmann rated it liked it

This book was solid, though perhaps not quite as good as I hoped/expected. It was interesting reading, full of interesting stories and examples. The author doesn't prescribe a particular method - superforecasting, it appears, is more about a toolbox or set of guidelines that must be used and adapted based on the particular circumstances. At a result, at times I felt the author's thread was being lost or scattered; however, upon reflection I realized it was part of the nature of making prediction ...more

flag 7 likes · Like · see review

Jun 24, 2015 Frank Ruscica rated it it was amazing · review of another edition

Just finished reading an advance copy. The signal-to-noise ratio of this book: maximum.

flag 5 likes · Like · see review

Sep 30, 2015 Allen Adams rated it it was amazing

Shelves: published-reviews, nonfiction, science

http://www.themaineedge.com/style/fut...

Ever since mankind has grasped the concept of time, we have been trying to predict the future. Whole cottage industries have sprung up around the process of prediction. Knowing what is coming next is a need that borders on the obsessive within our culture.

But is it even possible to predict what has yet to happen?

According to “Superforecasters: The Art and Science of Prediction”, the answer is yes…sort of. Social scientist Philip Tetlock and journalist Dan http://www.themaineedge.com/style/fut...

Ever since mankind has grasped the concept of time, we have been trying to predict the future. Whole cottage industries have sprung up around the process of prediction. Knowing what is coming next is a need that borders on the obsessive within our culture.

But is it even possible to predict what has yet to happen?

According to “Superforecasters: The Art and Science of Prediction”, the answer is yes…sort of. Social scientist Philip Tetlock and journalist Dan Gardner have teamed up to offer a treatise on the nature of prognostication. Not only do they discuss the many pitfalls of prediction, but they also offer up some thoughts on that small percentage of the population who, for a variety of reasons, are very, VERY good at it.

Tetlock has spent decades researching the power of prediction. Basically, people are pretty terrible at it. He himself uses the oft-offered analogy of a chimpanzee throwing darts – the implication is that random chance is at least as good at predicting future outcomes as the average forecaster. Even the well-known pundits, the newspaper columnists and talking heads – even they struggle to outperform the proverbial dart-tossing simian.

But over the course of Tetlock’s years of study by way of his ongoing Good Judgment Project, he uncovered an astonishing truth. Yes, most people have no real notion of how to predict the outcome of future events. However, there are some who can outperform the chimp. They can outperform the famous names. They can outperform think tanks and universities and national intelligence agencies and algorithms.

Tetlock calls these people “superforecasters.”

These superforecasters were among the tens of thousands that volunteered to be a part of the Good Judgment Project. They were part of a massive, government-funded forecasting tournament. These people – folks from all walks of life, filmmakers and retirees and ballroom dancers and you name it – were asked to predict the outcome of future events. And so they did. These people soon separated themselves from the pack, offering predictions about a vast and varied assemblage of global events with a degree of unmatched accuracy.

In “Superforecasters,” we get a chance to look a little closer at some of these remarkably gifted individuals. Tetlock offers analysis of some past predictions that were successful and others that were failures. We also get insight from prominent figures in the intelligence community and from people ensconced in the public and private sectors alike. And as Tetlock and company dig deeper, it becomes clear that the key to forecasting accuracy – to becoming a superforecaster – isn’t about computing power or complex algorithms or secret formulas. Instead, it’s about a mindset, a desire to devote one’s intellectual powers to a flexibility of thought. The accuracy of these superforecasters springs from their ability to understand probability and to work as a team, as well as an acceptance of the possibility of error and a willingness to change one’s mind.

There’s something inherently fascinating about predicting the future. One might think that Tetlock’s findings are a bit complex – and they undoubtedly are. However, what he and co-author Gardner have done is condense the myriad complications of his decades of research into something digestible. A wealth of information has been distilled into a compelling and fascinating work. It’s not quite pop science – it’s a bit denser than that – but it’s still perfectly comprehensible to the layman. In essence, this book gives us a clear and demonstrable way to improve the way we predict the future.

“Superforecasting” is a fascinating and compelling exploration of something to which many of us may not have given much thought. It’s not all haphazard chance – there are actually ways to improve your ability to predict the future, some of which are laid out right here for you. If nothing else, you’ll never look at a pundit’s prediction the same way again. ...more

flag 4 likes · Like · see review

Jun 30, 2016 Andy rated it it was ok · review of another edition

There is some interesting trivia about "Super-forecasters" but when it comes to elucidating evidence-based practice, this book was Super-disappointing. It starts off well with a discussion of Archie Cochrane and evidence-based medicine (EBM), but then it bizarrely ignores the core concepts of EBM.

-In EBM, you look up what works and then use that info to help people instead of killing them. But when Tetlock talks about social philanthropy he implies that it's evidence-based as long as you rigoro There is some interesting trivia about "Super-forecasters" but when it comes to elucidating evidence-based practice, this book was Super-disappointing. It starts off well with a discussion of Archie Cochrane and evidence-based medicine (EBM), but then it bizarrely ignores the core concepts of EBM.

-In EBM, you look up what works and then use that info to help people instead of killing them. But when Tetlock talks about social philanthropy he implies that it's evidence-based as long as you rigorously evaluate what you're doing. NO! If your doctor gives you arsenic instead of antibiotics for your bacterial infection, that's not OK even if he does lots of lab tests afterwards to see how you're progressing.

-In EBM, you focus on the best available evidence. There's a difference between what some drug rep told you vs. the conclusions of a randomized clinical trial. But when Tetlock reviews the Iraq War fiasco, he argues there was a really big pile of evidence so it made sense to go to war. He doesn't seem to get that a really big pile of crap is still just crap. Some elements of the pro-war narrative were known to be bogus before the war. Others turned out to be bogus after (Curveball, etc.), so those were not investigated and confirmed as solid evidence beforehand either.

-In EBM, the point of a diagnostic test is to get a predictive value. This number tells you how likely a test result is to be true, based on its track record. Instead, Tetlock praises his forecasters for making up percentages that reflect their subjective degree of certitude. And he calls those "probabilities" but that is very misleading, because in science a probability is something like the chance of drawing a royal flush in poker, i.e. it's an objectively calculated number based on reality.

-In EBM, the big issue is whether the treatment works for the main relevant outcome. So for the Iraq War example, the question for the CIA was whether an invasion would A) spread democracy in the Middle East after preventing an imminent nuclear attack on the USA, or B) not prevent anything (because there was no secret new nukes program) and increase regional chaos as well as global terrorism (think ISIS). This decision tree is absent from the book, and that omission violates Tetlock's own rule about asking the meaningful hard question.

This book has good content on cognitive biases. But I would recommend going directly to the source on that topic.

...more

flag 4 likes · Like · see review

Nov 19, 2015 Asif rated it it was amazing

Possibly the best book I read in 2015. Couldn't have read at a better time as the year nears an end. I could relate with a lot of things as I work as an equity analyst trying to do the seemingly impossible thing of forecasting stock prices. In particular, the examples of how superforecasters go about doing their jobs were pretty inspiring. Examples of taking the outside view and creating a tree of various outcomes and breaking down that tree into branches are something I could benefit from.

As a ...more

flag 4 likes · Like · see review

Apr 03, 2016 Ivan rated it it was amazing · review of another edition

Shelves: smart-thinking

Среди плотно заставленных полок в разделе “Smart Thinking” как-то мне на глаза попалась книга “Superforecasting”. Интригующая тема, подумал я, много упоминаний в элитной прессе – нужно читать! Если кратко, то эта книга о том, насколько хорошо люди могут предсказывать результаты важных глобальных событий, как это измерить и чему можно научиться у тех, кто предсказывает лучше.

Каждый из нас регулярно занимается прогнозированием: мы размышляем над тем, на сколько нам повысят зарплату, упадёт ли долл ...more

flag 3 likes · Like · see review

Aug 16, 2015 Leland Beaumont rated it it was amazing

Summarizing 20 years of research on forecasting accuracy conducted from 1984 through 2004, Philip Tetlock concluded “the average expert was roughly as accurate as a dart-throwing chimpanzee.” More worrisome is the inverse correlation between fame and accuracy—the more famous a forecasting expert was, the less accurate he was. This book describes what was learned as Tetlock set out to improve forecasting accuracy with the Good Judgement Project.

Largely in response to colossal US intelligence erro Summarizing 20 years of research on forecasting accuracy conducted from 1984 through 2004, Philip Tetlock concluded “the average expert was roughly as accurate as a dart-throwing chimpanzee.” More worrisome is the inverse correlation between fame and accuracy—the more famous a forecasting expert was, the less accurate he was. This book describes what was learned as Tetlock set out to improve forecasting accuracy with the Good Judgement Project.

Largely in response to colossal US intelligence errors, the Intelligence Advanced Research Projects Activity (IARPA) was created in 2006. The goal was to fund cutting-edge research with the potential to make the intelligence community smarter and more effective. Acting on recommendations of a research report the IARPA sponsored a massive tournament to see who could invent the best methods of making the sorts of forecasts that intelligence analysis make every day. This tournament provided the experimental basis for rigorously testing the effectiveness of many diverse approaches to forecasting.

And learn they did! Thousands of ordinary citizen volunteers applied, approximately 3,200 were invited to participate, and 2,800 eventually joined the project. “Over four years, nearly five hundred questions about international affairs were asked of thousands of Good Judgment Project’s forecasters, generating well over one million judgments about the future.” Because fuzzy thinking can never be proven wrong, questions and forecasts were specific enough that the correctness of each forecast could be clearly judged. These results were used to compute a Brier score—a quantitative assessment of the accuracy of each forecast— for each forecaster.

In the first year 58 forecasters scored extraordinary well; they outperformed regular forecasters in the tournament by 60%. Remarkably these amateur superforecasters “performed about 30 percent better than the average for intelligence community analysts who could read intercepts and other secret data.” This is not just luck; the superforecasters as a whole increased their lead over all other forecasters in subsequent years.

Superforecasters share several traits that set them apart, but more importantly they use many techniques that we can all learn. Superforecasters have above average intelligence, are numerically literate, pay attention to emerging world events, and continually learn from their successes and failures. But perhaps more importantly, they approach forecasting problems using a particular philosophic outlook, thinking style, and methods, combined with a growth mindset and grit. The specific skills they apply can be taught and learned by anyone who wants to improve their forecasting accuracy.

This is an important book. Forecasting accuracy matters and the track record has been miserable. Public policy, diplomacy, military action, and financial decisions often depend on forecast accuracy. Getting it wrong, as so often happens, is very costly. The detailed results presented in this book can improve intelligence forecasts, economic forecasts, and other consequential forecasts if we are willing to learn from them.

This is as close to a page-turner as a nonfiction book can get. The book is well-written and clearly presented. The many rigorous arguments presented throughout the book are remarkably accessible. Sophisticated quantitative reasoning is well presented using examples, diagrams, and only a bare minimum of elementary mathematical formulas. Representative evidence from the tournament results support the clearly-argued conclusions presented. Personal accounts of individual superforecasters add interest and help create an entertaining narrative. An appendix summarizes “Ten Commandments for Aspiring Superforecasters”. Extensive notes allow further investigation, however the advanced reader edition lacks an index.

Applying the insights presented in this book can help anyone evaluate and improve forecast accuracy. “Evidence-based policy is a movement modeled on evidence-based medicine.” The book ends with simple advice and a call to action: “All we have to do is get serious about keeping score.”
...more

flag 4 likes · Like · see review

Feb 12, 2016 Andrew rated it it was ok

Shelves: data-information-tech, math, united-states-of-america, psychology-behaviour

Superforcasting: The Art of Science and Prediction, by Philip E. Tetlock, is a book about the art and science of statistical prediction, and its everyday uses. Except it isn't really, that is just what they are selling it as. The book starts off really strong, analyzing skepticism, illusions of statistical knowledge, and various types of bias. However, the majority of the book focuses on a US government intelligence project called IARPA, designed to use everyday citizens to make statistical pred ...more

flag 3 likes · Like · see review

Sep 29, 2015 Aloke rated it it was amazing

Shelves: favorites, borrowed, money, non-fiction

Social scientist Philip Tetlock talks about his multi-year experiment to measure and identify ways to improve our forecasts. A lot of his findings appear to be common sense, i.e. many forecasts are very nebulous and difficult to disprove and so we can't really tell if we are improving, big problems can be forecast by breaking them into more manageable pieces (Fermi-izing in his terminology), our judgement is affected by various biases, etc. The contribution is that Tetlock has brought these idea ...more

flag 3 likes · Like · see review

Dec 09, 2015 Daniel rated it it was amazing

Un libro interesante acerca de cómo realizar pronósticos.

Uno de los puntos principales para saber cómo se comportará un fenómeno en el futuro depende en gran medida de la información con la que contamos y los posibles escenarios que podrían suceder.

Aunque siempre existen sucesos inesperados el conocimiento de las situaciones disminuyen la incertidumbre.

flag 3 likes · Like · see review

Oct 18, 2015 Marco rated it liked it · review of another edition

TL;DR: a more practical companion to Expert Political Judgment, useful for forecasts in politics, economics, and other areas of human activity, if not quite as compelling as its predecessor. 3 1/2 stars.

Superforecasting is a book that presents the results of Philip Tetlock's research on prediction and avoiding the biases that afflict even experts' forecasts within their domain of expertise. The book is something of a sequel to Expert Political Judgment, which shows that, on average, the politica TL;DR: a more practical companion to Expert Political Judgment, useful for forecasts in politics, economics, and other areas of human activity, if not quite as compelling as its predecessor. 3 1/2 stars.

Superforecasting is a book that presents the results of Philip Tetlock's research on prediction and avoiding the biases that afflict even experts' forecasts within their domain of expertise. The book is something of a sequel to Expert Political Judgment, which shows that, on average, the political forecasts made by experts don't fare much better than random, when one takes into account both accuracy and calibration (that is, events which were attributed high probabilities occur with higher frequency, and the converse for low-probability events). This dismal scenario is alleviated by the discovery that some experts had a significantly better performance than that, and that those experts shared certain traits.

That silver lining led to the creation of the Good Judgment Project led by the author, which studied the behavior and results of those overperforming predictors. The book describes some of the findings of that study: despite the (practical?) impossibility of perfect prediction, some heuristics and habits can improve the accuracy of forecasts in significant and persistent ways.

The author claims, supported by the project's evidence, that testability plays a central role on that improvement: if there are metrics that allow the verification of how right or how wrong a given prediction (or predictor) was, then the higher level of scrutiny will push forecasters towards "good judgment" as described by the Good Judgment Project.

This programme is open to some critiques that the author acknowledges but -- at least not in my opinion -- does not fully address, especially about the relevance and scope of the kind of predictions which have results that can be easily metrified. Yet, as the author claims, those doubts don't mean that a programme for improving forecast, even if somewhat narrow on its outputs, would not be beneficial for the public debate on the relevant subject matters.

While, in general, I think that Expert Political Judgment is a more interesting book, Superforecasting has a much wider and more general scope. As such, it is a good read for those interested in practical forecast at whatever level or topic. ...more

flag 2 likes · Like · see review

Sep 04, 2017 Vincent Li rated it it was amazing

A great read, probably the best light read this summer. A book I would recommend to whole-heartedly to anyone, since we are all forecasters. Usually, I am not a fan of the Predictably Irrational/Freakonomics model of turning an academic paper into a full length book but exceptions must be made. Tetlock was famous for producing the chimp throwing darts study (interestingly, fame was inversely proportional to accuracy, and there's many examples of pundits who project forward a vague forecast and t A great read, probably the best light read this summer. A book I would recommend to whole-heartedly to anyone, since we are all forecasters. Usually, I am not a fan of the Predictably Irrational/Freakonomics model of turning an academic paper into a full length book but exceptions must be made. Tetlock was famous for producing the chimp throwing darts study (interestingly, fame was inversely proportional to accuracy, and there's many examples of pundits who project forward a vague forecast and then rationalize their forecast ex post), but the work that produced this book is far more interesting and meaningful. Tetlock starts with a digression into the history of medicine, which consistently spurned randomized trials and empiricism to the harm of patients and the practice of medicine (reminds me particularly of Taleb's work as well as the general narrative of Emperor of All Maladies). Tetlock argues that without randomized testing, our folk wisdom are essentially shots in the dark. As a faint hearted-empiricist, I cannot agree with him more. Tetlock discusses how many forecasts are unfalsifiable because the time frame is too vague, the event is ill defined and language can be ambiguous (maybe, significant, probably). Tetlock argues fairly that sometimes this vagueness comes from a common understanding that is quickly forgotten, a general discomfort with turning probabilities into numbers, or more cynically a method to cover one's own forecasts (this is not totally irrational, as people have trouble understanding probabilistic forecasts, and are quick to pin blame). A point that Tetlock makes that I thought was important is that not all forecasts have the purpose of being accurate. Forecasts have other purposes, such as entertainment, encouraging action, and bringing comfort.

The book goes through Tetlock's entry into the IAPRA competition, which tried to measure the accuracy of clearly defined questions with time frames (Tetlock praises IAPRA for daring to run an experiment that could cost them their jobs). The book goes on to quantify the accuracy of forecasts by explaining the concept of a brier score (0-2, the lower the score, the more accurate), calibration (over many forecasts, the match between a predicted probability and the percentage occurrence of an event, i.e. when a forecaster claims that something is likely to occur with 60% confidence, do those events overall actually occur 60% of the time?), and resolution (to adjust for forecasters who just consistently forecast in the midrange). According to Tetlock, his methods for aggregating the wisdom of the forecasters (the wisdom of the crowd idea, that random errors cancel each other out but real information point the same way) with heavier weights for "superforecasters" and extremizing the probability was the superior forecasting method to the other entries in the competition. According to one source, the method was 30% more accurate than analysts with access to classified information (Tetlock argues that the superforecasters as teams are superior to even prediction markets, but with intellectual candor admits that his markets might have been too inliquid and had no monetary incentive). The core of the book discusses these superforecasters. I was impressed by the factors that Tetlock tries to control for in order to find the secret to their success. Tetlock systemically eliminates superintelligence (the superforecasters on the whole are a little above average but nowhere near genius level) and luck (Tetlock notes the lack of regression to the mean in time series data) as the driving factors behind the superforecasters' success. Instead, tantalizingly, Tetlock argues that their success is driven by learnable habits that is accessible to nearly everyone. Tetlock's superforecasters tend to be foxes, who are intellectually humble, constantly learn through feedback, use base rates, break down problems into pieces like a consulting interview problem ("Fermize"), update their forecasts in a Bayesian manner (though not strictly mathematically), use fine gradations of probability (an analysis demonstrated that rounding to the nearest 10% of these forecasts actually reduced their accuracy), avoided cognitive errors (like confirmation bias, by taking a different position and arguing it to the earnest, and avoiding hindsight bias), had grit, typically had a growth mindset instead of a fixed mindset, did not believe that things necessarily happened for a reason but instead believed in chance, and practiced a lot(Tetlock does not think there is a magic bullet). Tetlock also studied the effect of putting these superforecasters in teams, and observed the general avoidance of groupthink by these teams who had improved accuracy (more ground covered, diverse opinions, and respectful clarification and pushback). Tetlock also suggests creating psychological safety in order to avoid group think by having dedicated devil advocates, suspending hierarchy, and bringing in fresh perspectives. The team members were also generally givers, which maximized accuracy.

Tetlock then responds to a two criticisms from Kahneman and Taleb. Tetlock characterizes Kahneman's objection as an inability for humans to adjust to their system 1 mistakes. Tetlock claims that certain adjustments can be repeated so often that they become reactionary and part of a person's system 1. Tetlock offers potential scope sensitivity as evidence that superforecasters can perhaps overcome their innate biases. Tetlock argues that many of the black swans that people discuss are actually to some degree predictable (though Tetlock admits that extremely rare events would be hard to accurately measure the forecast of), and it is still necessary for forecasts to be used to appropriately allocate resources for different contingencies. Tetlock argues that a leader can be both decisive when decisions are made and still have the good qualities of a forecaster. In particular, Tetlock argues for a decentralized decision making process that only sets out goals, allowing subordinates to achieve this goal through the means they see fit, which he claims was the method used by the German army, US army and several successful businesses.

Tetlock spends some time setting out potential implications of his research. He argues that there might be a place for pundits in setting the questions for superforecasters to answer, and the hope that these forecasts will be imported into political discourse in order to bring empirical rigor to economic and political debates. ...more

flag 1 like · Like · see review

Aug 11, 2015 Itsadog rated it really liked it · review of another edition

Shelves: first-reads

I received an ARC in exchange for an honest review through Goodreads First Reads. Thank you, Random House of Canada!!

Old joke about American Intelligence and oxymorons aside, Mr Tetlock draws heavy inspiration from American politics in the latest book detailing his life's research. This is the person behind the quip about American forecasters being worse than "chimps with darts", a line he's sick of hearing but that's not inaccurate.

This book was about half guilty pleasure for nerds *raises hand ...more

flag 2 likes · Like · see review

Sep 19, 2015 Josh Url rated it it was amazing

Superforecasting: The Art and Science of Prediction is a wonderful read on a fascinating subject which we all do to some degree every single day of our lives - forecasting. Tetlock and Gardner paint the picture of why we struggle at predicting uncertain things and illustrate several ways we can improve our abilities to improve in this area. I work in basketball and one of the core elements is forecasting how players may perform in a different offensive system, defensive system, with different te ...more

flag 2 likes · Like · see review

Aug 14, 2015 Grumpus rated it liked it

Shelves: business

A Christmas gift from my daughters with a note that said, “Now you can predict the lottery numbers.” No pressure.

This book is not that type of book, but it is still worth the read to those of us who feel the need to quantify everything. The book documents the inaccuracy of TV pundits and, through a website in which anyone can make quantifiable forecasts, tracks the accuracy of forecasters on key economic and world issues. Through a scoring system, the super forecasters are identified and studied ...more

flag 2 likes · Like · see review

Sep 23, 2015 Pat rated it it was amazing

I was one-hundred percent, absolutely blown away by this book. I think it is because it contained so many truths that I, a a project-based learning facilitator in a public high school, preach to my students. Students have to be able to accurately forecast whether their project will be successful and make changes if it won't be. How do they do that? They need to know what is known and unknown. They also need grit, determination, and a growth mindset. They also need to be able to work together wit ...more

flag 2 likes · Like · see review

Jul 19, 2015 Lindley rated it liked it

I didn't know much about forecasting before reading this book, and found the explanations easy to follow. The ease with which the superforecasters' predictions were explained, however, took some of the magic out of the ability to consistently beat others' forecasts. Yet tips not to over- or under-predict simultaneously make the superforecasters' feats seem even more out of reach.

This book will appeal to readers who enjoyed Nate Silver's The Signal and the Noise.

flag 3 likes · Like · see review

Jan 04, 2016 Nick Pascucci rated it really liked it

This is really a book on epistemology: How do you manage uncertainty in a world with tons of it? How can we update our beliefs while taking into account the usefulness of new evidence? In the forecasting arena these questions and their answers are key to producing accurate predictions. I would recommend this book to anyone interested in the art of Bayesian epistemology and probabilistic thinking, and doubly so if one has a predilection for prediction.

flag 2 likes · Like · see review

Mar 25, 2016 Michael Litton rated it it was amazing

Jag kan varmt rekommendera denna bok för alla som någon gång har försökt att förutsäga någonting om framtiden och vill bli bättre på att göra det.

Goda tankar kring mål och mätning för den typen av arbete. Många nya och återupptäckta idéer för framtida äventyr :)

flag 2 likes · Like · see review

Oct 18, 2015 Yifan Wu rated it it was ok

It was mildly interesting. I wouldn't really recommend if you have taken stats and know basic ideas behind thinking fast and slow.

flag 2 likes · Like · see review

Jan 04, 2017 Michael Cestas rated it really liked it

Super.

flag 2 likes · Like · see review

Nov 26, 2016 Travis Scher rated it really liked it

Shelves: best-nonfiction

An excellent book on how to think clearly. The short-short version of the forecasting formula set forth by author Philip Tetlock is "use probabilities and check your ego." The book elaborates on this, and will teach you how to use probabilities and when and how to check your ego. Perhaps more importantly, it will help you spot others' errors and see through their bs.

The book also highlights that much of the "forecasts" by professional and amateur pundits are no sincere attempts to accurately pr ...more

flag 1 like · Like · see review

Mar 31, 2017 Kate rated it really liked it

Shelves: non-fiction, canadian-author, dude-writer, american-author, forecasting, psychology

This was a fascinating look at "superforecasters" - basically people who are the best at predicting the (near) future. They're not always the people you think though - none of the guys who make their living doing forecasts for the news wanted to be tested so who knows how well they're doing! But a lot of the superforecasters described in this book are regular joes who do this for fun on the side. There's a lot of good advice in here on how you too can become a superforecaster - it's a learnable ...more

flag 1 like · Like · see review

Jun 02, 2017 Honza Marcinek rated it liked it

Shelves: marketing

Kdysi jsem měl problém s knihou Myšlení, rychlé a pomalé a doufal jsem, že tato kniha mě nadchne víc. Jenže bohužel asi nejsem na tento typ knih stavěnej. Ani nevím, komu bych tu knihu mohl doporučit. Možná těm, kteří sázejí v různých sázkařských společnostech, jenž nabízí možnost si vsadit i na určité společenské situace a ne jen sportovní utkání. Při čtení jsem si totiž vzpomněl na muže, který si vsadil na vítězství Trumpa a nakonec vyhrál 1,445 milionu.

flag 1 like · Like · see review

Jun 17, 2017 Lorena rated it liked it

Este libro trata de las técnicas y forma de pensamiento necesarias para lograr hacer predicciones lo más asertadas posibles.

La primera parte me pareció bastante aburrida y obvia, describe porqué los pronosticadores famosos no son buenos en lo que hacen.

La segunda parte analiza, en base a un concurso de pronósticos que se realizó durante 3 años, qué habilidades, formas de pensamiento y técnicas se deben utilizar para lograr pronósticos acertados.

Algunas de las ideas clave fueron: tener apertura ...more

flag 1 like · Like · see review

« previous 1 2 3 4 5 6 7 8 9 … next »

new topic
Discuss This Book

topics	posts	views	last activity
Superforecasting: White Rats and Foxes	1	12	Sep 06, 2015 06:15PM

Readers Also Enjoyed

Genres

Nonfiction

210 users

Business

149 users

Science

124 users

Psychology

117 users

Economics

109 users

See top shelves…

Books by Philip E. Tetlock

Expert Political Judgment: How Good Is It? How Can We Know?

Unmaking the West: "What-If?" Scenarios That Rewrite World History

Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives

Behavior, Society, and Nuclear War: Volume II

Behavior, Society, and Nuclear War: Volume I

More…

Share This Book

+ Share on your website

Trivia About Superforecasting:...

No trivia or quizzes yet. Add some now »

Quotes from Superforecasting:...

“For scientists, not knowing is exciting. It’s an opportunity to discover; the more that is unknown, the greater the opportunity.” — 10 likes

“If you don’t get this elementary, but mildly unnatural, mathematics of elementary probability into your repertoire, then you go through a long life like a one-legged man in an ass-kicking contest.” — 8 likes

More quotes…

title link	preview: Superforecasting: The Art and Science of Prediction
avg rating	preview: Superforecasting: The Art and Science of Prediction Goodreads rating: 4.14 (5611 ratings)
small image	preview: click here [close]
med image	preview: click here [close]
BBCode	[url=https://www.goodreads.com/book/show/23995360-superforecasting?utm_medium=api&utm_source=blog_book][img]https://images.gr-assets.com/books/1435684424l/23995360.jpg[/img][/url] [url=https://www.goodreads.com/book/show/23995360-superforecasting?utm_medium=api&utm_source=blog_book]Superforecasting: The Art and Science of Prediction by Philip E. Tetlock[/url]

See a Problem?

Preview — Superforecasting by Philip E. Tetlock

Superforecasting: The Art and Science of Prediction

Friend Reviews

Reader Q&A

Recent Questions

Lists with This Book

Community Reviews

new topic
Discuss This Book

Readers Also Enjoyed

Genres

Books by Philip E. Tetlock

Share This Book

Share on your website

Trivia About Superforecasting:...

Quotes from Superforecasting:...

See a Problem?

Preview — Superforecasting by Philip E. Tetlock

Superforecasting: The Art and Science of Prediction

Friend Reviews

Reader Q&A

Recent Questions

Lists with This Book

Community Reviews

new topicDiscuss This Book

Readers Also Enjoyed

Genres

Books by Philip E. Tetlock

Share This Book

Share on your website

Trivia About Superforecasting:...

Quotes from Superforecasting:...

new topic
Discuss This Book