Superforecasting: The Art and Science of Prediction
Rate it:
Open Preview
Read between November 28 - December 9, 2018
5%
Flag icon
The further out the forecaster tries to look, the more opportunity there is for chaos to flap its butterfly wings and blow away expectations.
6%
Flag icon
For scientists, not knowing is exciting. It’s an opportunity to discover; the more that is unknown, the greater the opportunity.
6%
Flag icon
Good Judgment Project
7%
Flag icon
When physicians finally accepted that their experience and perceptions were not reliable means of determining whether a treatment works, they turned to scientific testing—and medicine finally started to make rapid advances.
7%
Flag icon
It reviewed twenty studies showing that well-informed experts predicting outcomes—whether a student would succeed in college or a parolee would be sent back to prison—were not as accurate as simple algorithms that added up objective indicators like ability test scores and records of past conduct.
11%
Flag icon
the Cognitive Reflection Test, which has shown that most people—including very smart people—aren’t very reflective.
14%
Flag icon
be prepared to accept that what seems obviously true now may turn out to be false later.
15%
Flag icon
As another journalist wrote, when Ballmer announced his departure from Microsoft in 2013, “The iPhone alone now generates more revenue than all of Microsoft.”2
17%
Flag icon
intelligence analysis—the methodical examination of the information collected by spies and surveillance to figure out what it means, and what will happen next.
18%
Flag icon
CERTAINTY THE GENERAL AREA OF POSSIBILITY 100% Certain 93% (give or take about 6%) Almost certain 75% (give or take about 12%) Probable 50% (give or take about 10%) Chances about even 30% (give or take about 10%) Probably not 7% (give or take about 5%) Almost certainly not 0% Impossible
18%
Flag icon
when forecasters are forced to translate terms like “serious possibility” into numbers, they have to think carefully about how they are thinking, a process known as metacognition. Forecasters who practice get better at distinguishing finer degrees of uncertainty, just as artists get better at distinguishing subtler shades of gray.
18%
Flag icon
The only way to know for sure would be to rerun the day hundreds of times. If it rained in 70% of those reruns, and didn’t rain in 30%, she would be bang on.
21%
Flag icon
They talked about possibilities and probabilities, not certainties. And while no one likes to say “I was wrong,” these experts more readily admitted it and changed their minds.
22%
Flag icon
the more famous an expert was, the less accurate he was.
23%
Flag icon
With valid information piling up and errors nullifying themselves, the net result was an astonishingly accurate estimate.
23%
Flag icon
Now look at how foxes approach forecasting. They deploy not one analytical idea but many and seek out information not from one source but many. Then they synthesize it all into a single conclusion. In a word, they aggregate. They may be individuals working alone, but what they do is, in principle, no different from what Galton’s crowd did. They integrate perspectives and the information contained within them. The only real difference is that the process occurs within one skull.
27%
Flag icon
Quit pretending you know things you don’t and start running experiments.
38%
Flag icon
Active open-mindedness (AOM) is a concept coined by the psychologist Jonathan Baron,
38%
Flag icon
People should take into consideration evidence that goes against their beliefs.
38%
Flag icon
beliefs are hypotheses to be tested, not treasures to be guarded.
43%
Flag icon
distinction that philosophers have proposed between “epistemic” and “aleatory” uncertainty.
43%
Flag icon
bafflegab.
45%
Flag icon
So finding meaning in events is positively correlated with wellbeing but negatively correlated with foresight.
46%
Flag icon
Unpack the question into components. Distinguish as sharply as you can between the known and unknown and leave no assumptions unscrutinized. Adopt the outside view and put the problem into a comparative perspective that downplays its uniqueness and treats it as a special case of a wider class of phenomena. Then adopt the inside view that plays up the uniqueness of the problem. Also explore the similarities and differences between your views and those of others—and pay special attention to prediction markets and other methods of extracting wisdom from crowds. Synthesize all these different ...more
48%
Flag icon
Consider the 1942 argument of General John DeWitt, a strong supporter of the internment of Japanese Americans: “The very fact that no sabotage has taken place to date is a disturbing and confirming indication that such action will be taken.”6—or to put that more bluntly, “The fact that what I expected to happen didn’t happen proves that it will.”
53%
Flag icon
Try, fail, analyze, adjust, try again:
56%
Flag icon
Grit is passionate perseverance of long-term goals, even in the face of frustration and failure. Married with a growth mindset, it is a potent force for personal progress.
58%
Flag icon
Groups that get along too well don’t question assumptions or confront uncomfortable facts.
59%
Flag icon
On the one hand, we warned, groupthink is a danger. Be cooperative but not deferential. Consensus is not always good; disagreement not always bad. If you do happen to agree, don’t take that agreement—in itself—as proof that you are right.
61%
Flag icon
If many people agree with me that a stock is worth more than it’s selling for, they will try to buy it. Increasing demand pushes the price up. In that way, all the individual judgments of the buyers, and all the information guiding those judgments, becomes “priced in.”
61%
Flag icon
Prediction markets are simply markets that trade in predictions, meaning traders buy and sell contracts on specified outcomes—such as “Hillary Clinton will be elected president of the United States in 2016.” When the election of 2016 is held, that contract is settled. If Clinton loses, the contract pays out nothing. If she wins, it pays out $1. If the contract is currently selling for 40 cents and I think Clinton has a 60% or 70% chance of winning, I should buy.
62%
Flag icon
Wharton colleague Adam Grant categorizes people as “givers,” “matchers,” and “takers.”
62%
Flag icon
But Grant’s research shows that the pro-social example of the giver can improve the behavior of others, which helps everyone, including the giver—which explains why Grant has found that givers tend to come out on top.
62%
Flag icon
The more diverse his team, the greater the chance that some advisers will possess scraps of information that others don’t.
63%
Flag icon
Leaders must be reasonably confident, and instill confidence in those they lead, because nothing can be accomplished without the belief that it can be. Decisiveness is another essential attribute. Leaders can’t ruminate endlessly. They need to size up the situation, make a decision, and move on. And leaders must deliver a vision—the goal that everyone strives together to achieve.
63%
Flag icon
“No plan of operations extends with certainty beyond the first encounter with the enemy’s main strength,”
Mark Frauenfelder
Everyone has a plan until they get punched in the mouth
64%
Flag icon
The art of leadership consists of the timely recognition of circumstances and of the moment when a new decision is required.”
64%
Flag icon
commanders were to tell subordinates what their goal is but not how to achieve it.
71%
Flag icon
What exactly is a black swan? The stringent definition is something literally inconceivable before it happens.
72%
Flag icon
These limits on predictability are the predictable results of the butterfly dynamics of nonlinear systems.
72%
Flag icon
my EPJ research, the accuracy of expert predictions declined toward chance five years out.
72%
Flag icon
If you have to plan for a future beyond the forecasting horizon, plan for surprise. That means, as Danzig advises, planning for adaptability and resilience.
72%
Flag icon
Imagine a scenario in which reality gives you a smack in the ear and consider how you would respond. Then assume reality will give you a kick in the shin and think about dealing with that. “Plans are useless,” Eisenhower said about preparing for battle, “but planning is indispensable.”
72%
Flag icon
Knowing what we don’t know is better than thinking we know what we don’t.
77%
Flag icon
“Not everything that counts can be counted,” goes a famous saying, “and not everything that can be counted counts.”13 In this era of computers and algorithms, some social scientists have forgotten that.