Thinking in Bets: Making Smarter Decisions When You Don't Have All the Facts

by Annie Duke

Diversity is the foundation of productive group decision-making, but we can’t underestimate how hard it is to maintain. We all tend to gravitate toward people who are near clones of us. After all, it feels good to hear our ideas echoed back to us.

The study, encompassing over 6,000 federal appeals and nearly 20,000 individual votes, found, not surprisingly, that judicial voting generally followed political lines. Pure, unaided open-mindedness, even by life-tenured judges sworn to uphold the law, is hard.

At least one study has found that, yes, a betting market where scientists wager on the likelihood of experimental results replicating was more accurate than expert opinion alone.

Experts engaging in traditional peer review, providing their opinion on whether an experimental result would replicate, were right 58% of the time. A betting market in which the traders were the exact same experts and those experts had money on the line predicted correctly 71% of the time.

Companies implementing prediction markets to test decisions include Google, Microsoft, General Electric, Eli Lilly, Pfizer, and Siemens.

People are more willing to offer their opinion when the goal is to win a bet rather than get along with people in a room.

CUDOS: “An ideologically balanced science that routinely resorted to adversarial collaborations to resolve empirical disputes would bear a striking resemblance to Robert Merton’s ideal-type model of a self-correcting epistemic community, one organized around the norms of CUDOS.”

CUDOS stands for Communism (data belong to the group), Universalism (apply uniform standards to claims and evidence, regardless of where they came from), Disinterestedness (vigilance against potential conflicts that can influence the group’s evaluation), and Organized Skepticism (discussion among the group to encourage engagement and dissent).

His life spanned both world wars and the Cold War, in which he studied and witnessed nationalist movements in which people “arrayed their political selves in the garb of scientists,” explicitly evaluating scientific knowledge based on political and national affiliations.

If the group is discussing a decision and it doesn’t have all the details, it might be because the person providing them doesn’t realize the relevance of some of the data. Or it could mean the person telling the story has a bias toward encouraging a certain narrative that they likely aren’t even aware of.

When presenting a decision for discussion, we should be mindful of details we might be omitting and be extra-safe by adding anything that could possibly be relevant. On the evaluation side, we must query each other to extract those details when necessary.

I’ve encouraged companies to make sure they don’t define “winning” solely by results or providing a self-enhancing narrative. If part of corporate success consists of providing the most accurate, objective, and detailed evaluation of what’s going on, employees will compete to win on those terms. That will reward better habits of mind.

don’t disparage or ignore an idea just because you don’t like who or where it came from.

When we have a negative opinion about the person delivering the message, we close our minds to what they are saying and miss a lot of learning opportunities because of it. Likewise, when we have a positive opinion of the messenger, we tend to accept the message without much vetting. Both are bad.

I developed an exercise to practice and reinforce universalism. When I had the impulse to dismiss someone as a bad player, I made myself find something that they did well.

Another way to disentangle the message from the messenger is to imagine the message coming from a source we value much more or much less.

This can be incorporated into an exploratory group’s work, asking each other, “How would we feel about this if we heard it from a much different source?” We can take this process of vetting information in the group further, initially and intentionally omitting where or whom we heard the idea from. Leading off our story by identifying the messenger could interfere with the group’s commitment to universalism, biasing them to agree with or discredit the message depending on their opinion of the messenger.

Telling someone how a story ends encourages them to be resulters, to interpret the details to fit that outcome.

two people whose positions on an issue are far apart will move toward the middle after a debate or skilled explanation of the opposing position. Engaging in this type of exchange creates an understanding of and appreciation for other points of view much deeper and more powerful than just listening to the other perspective.

Skepticism gets a bum rap because it tends to be associated with negative character traits. Someone who disagrees could be considered “disagreeable.” Someone who dissents may be creating “dissention.” Maybe part of it is that “skeptical” sounds like “cynical.” Yet true skepticism is consistent with good manners, civil discourse, and friendly communications.

Skepticism is about approaching the world by asking why things might not be true rather than why they are true. It’s a recognition that, while there is an objective truth, everything we believe about the world is not true. Thinking in bets embodies skepticism by encouraging us to examine what we do and don’t know and what our level of confidence is in our beliefs and predictions. This moves us closer to what is objectively true.

The term “devil’s advocate” developed centuries ago from the Catholic Church’s practice, during the canonization process, of hiring someone to present arguments against sainthood.

There are several ways to communicate to maximize our ability to engage in a truthseeking way with anyone.

First, express uncertainty. Uncertainty not only improves truthseeking within groups but also invites everyone around us to share helpful information and dissenting opinions. Fear of being wrong (or of having to suggest someone else is wrong) countervails the social contract of confirmation, often causing people to withhold valuable insights and opinions from us. If we start by making clear our own uncertainty, our audience is more likely to understand that any discussion that follows will not involve right versus wrong, maximizing our truthseeking exchanges with those outside our chartered group.

Second, lead with assent. For example, listen for the things you agree with, state those and be specific, and then follow with “and” instead of “but.” If there is one thing we have learned thus far it is that we like having our ideas affirmed. If we want to engage someone with whom we have some disagreement (inside or outside our group), they will be more open and less defensive if we start with those areas of agreement, which there surely will be. It is rare that we disagree with everything that someone has to say. By putting into practice the strategies that promote universalism, actively looking for the ideas that we agree with, we will more naturally ...

When we lead with assent, our listeners will be more open to any dissent that might follow. In addition, when the new information is presented as supplementing rather than negating what has come before, our ...

“And” is an offer to contribute. “But” is a denial and repudiation of what came before.

Third, ask for a temporary agreement to engage in truthseeking. If someone is off-loading emotion to us, we can ask them if they are just looking to vent or if they are looking for advice. If they aren’t looking for advice, that’s fine. The rules of engagement have been made clear. Sometimes, people just want to vent. I certainly do. It’s in our nature. We want to be supportive of the people around us, and that includes comforting them when they just need some understanding and sympathy. But sometimes they’ll say they are looking for advice, and that is potentially an agreement to opt in to some truthseeking. (Even then, tread lightly because people may say they want advice when what they really want is to be affirmed.)

it doesn’t have to be offensive to ask, “Do you want to just let it all out, or are you thinking of what to do about it next?”

Finally, focus on the future. As I said at the beginning of this book, we are generally pretty good at identifying the positive goals we are striving for; our problem is in the execution of the decisions along the way to reaching those goals. People dislike engaging with their poor execution. That requires taking responsibility for what is often a bad outcome, which, as David Letterman found out, will shut down the conversation. Rather than rehashing what has already happened, try instead to engage about what the person might do so that things will turn out better going forward. Whether it’s our kids, other family members, friends, relationship partners, coworkers, or even ourselves, we share the common trait of generally being more rational about the future than the past. It’s harder to get defensive about something that hasn’t happened yet.

even with our own kids’ decisions, rehashing outcomes can create defensiveness. The future, on the other hand, can always be better if we can get them to focus on things in their control.

Accountability to a truthseeking group is also, in some ways, a time-travel portal. Because we know we will have to answer to the group, we start thinking in advance of how that will go. Anticipating and rehearsing those rational discussions can improve our initial decision-making and analysis, at a time when we might not otherwise be so rational.

the direction in which the chips flow in the short term only loosely correlates with decision quality. You can win a hand after making bad decisions and lose a hand after making good ones. But the mere fact that chips are changing hands is a reminder that every decision has consequences—that all those execution decisions you make along the way really matter.

This tendency we all have to favor our present-self at the expense of our future-self is called temporal discounting.* We are willing to take an irrationally large discount to get a reward now instead of waiting for a bigger reward later.

Thoreau, on the other hand, praised the power of regret: “Make the most of your regrets; never smother your sorrow, but tend and cherish it till it comes to have a separate and integral interest. To regret deeply is to live afresh.”

As Nietzsche points out, regret can do nothing to change what has already happened. We just wallow in remorse about something over which we no longer have any control.

Suzy Welch developed a popular tool known as 10-10-10 that has the effect of bringing future-us into more of our in-the-moment decisions. “Every 10-10-10 process starts with a question. . . . [W]hat are the consequences of each of my options in ten minutes? In ten months? In ten years?” This set of questions triggers mental time travel that cues that accountability conversation

Moving regret in front of a decision has numerous benefits. First, obviously, it can influence us to make a better decision. Second, it helps us treat ourselves (regardless of the actual decision) more compassionately after the fact. We can anticipate and prepare for negative outcomes. By planning ahead, we can devise a plan to respond to a negative outcome instead of just reacting to it.

Our problem is that we’re ticker watchers of our own lives. Happiness (however we individually define it) is not best measured by looking at the ticker, zooming in and magnifying moment-by-moment or day-by-day movements. We would be better off thinking about our happiness as a long-term stock holding. We would do well to view our happiness through a wide-angle lens, striving for a long, sustaining upward trend in our happiness stock, so it resembles the first Berkshire Hathaway chart.

The way we field outcomes is path dependent. It doesn’t so much matter where we end up as how we got there. What has happened in the recent past drives our emotional response much more than how we are doing overall.

Surfers have more than twenty terms to describe different kinds of waves. The reason is that the type of wave, the way it breaks, the direction it’s coming from, the bottom depth, etc., create differing challenges for surfers. There are closeouts (waves that break all at once) and double-ups (a type of wave created when two waves meet to form one wave) and reforms (a wave that will break, then die down, then break again). Non-surfers just call all of these “waves.” On rare occasions when we non-surfers need to be more specific, we just add a lot of extra words. Those extra words don’t cost us much because it doesn’t come up very often—maybe never. But for people involved in specialized activities, it’s worth it to be able to communicate a complex concept in a single word that laypeople would need lengthy phrases to convey. Having a nuanced, precise vocabulary is what jargon is all about.

We’ve all had this experience in our personal and professional lives: blowing out of proportion a momentary event because of an in-the-moment emotional reaction. By recognizing in advance these verbal and physiological signs that ticker watching is making us tilt, we can commit to develop certain habit routines at those moments.

We can take some space till we calm down and get some perspective, recognizing that when we are on tilt we aren’t decision fit.

This action—past-us preventing present-us from doing something stupid—has become known as a Ulysses contract.

Most illustrations of Ulysses contracts, like the original, involve raising a barrier against irrationality. But these kinds of precommitment contracts can also be designed to lower barriers that interfere with rational action.

Other violations of the Mertonian norm of universalism, shooting the message because we don’t think much of the messenger. Any sweeping term about someone, particularly when we equate our assessment of an idea with a sweeping personality or intellectual assessment of the person delivering the idea, such as “gun nut,” “bleeding heart,” “East Coast,” “Bible belter,” “California values”—political or social issues. Also be on guard for the reverse: accepting a message because of the messenger or praising a source immediately after finding out it confirms your thinking.

Once we recognize that we should watch out for particular words, phrases, and thoughts, when we find ourselves saying or thinking those things, we are breaking a contract, a commitment to truthseeking. These terms are signals that we’re succumbing to bias.

For us to make better decisions, we need to perform reconnaissance on the future.

Any decision can result in a set of possible outcomes.

“When faced with highly uncertain conditions, military units and major corporations sometimes use an exercise called scenario planning. The idea is to consider a broad range of possibilities for how the future might unfold to help guide long-term planning and preparation.”