More on this book
Community
Kindle Notes & Highlights
Figure 2 illustrates an important difference between bias and noise. It shows what you would see at the shooting range if you were shown only the backs of the targets at which the teams were shooting, without any indication of the bull’s-eye they were aiming at.
general property of noise is that you can recognize and measure it while knowing nothing about the target or bias.
To understand error in judgment, we must understand both bias and noise. Sometimes, as we will see, noise is the more important problem. But in public conversations about human error and in organizations all over the world, noise is rarely recognized. Bias is the star of the show. Noise is a bit player, usually offstage.
Years later, more of the unlucky children who have been assigned to foster care by these heavy-handed managers have poor life outcomes: higher delinquency rates, higher teen birth rates, and lower earnings.
noise audit,
decision hygiene.
mediating assessments protocol:
efforts to eliminate noise could undermine morale and give people a sense that they are being treated like cogs in a machine.
wherever there is judgment, there is noise—and more of it than you think.
arbitrary cruelties.
ukases
Mandatory guidelines reduce bias as well as noise.
leniency.
striking down the mandatory guidelines produced a return to something more like his nightmare: law without order.
System noise, that is, unwanted variability in judgments that should ideally be identical, can create rampant injustice, high economic costs, and errors of many kinds.
efforts at noise reduction often raise objections and run into serious difficulties. These issues must be addressed, too, or the fight against noise will fail.
“Criminal sentences should not depend on the judge’s mood during the hearing, or on the outside temperature.”
Notice the debate goes deeper than the causality vs. correlation discussion.
the potential value of an effort to increase consistency—to reduce noise—in the judgments of people who made significant financial decisions on the firm’s behalf. Everyone agreed that consistency is desirable.
Is consistency always desirable?
Goldilocks price that is just right—neither too high nor too low—and there is a good chance that the average judgment of a large group of professionals is not too far from this Goldilocks number. Prices that are higher or lower than this number are costly—this is how the variability of noisy judgments hurts the bottom line.
Unwanted Variability Versus Wanted Diversity
variability in judgments is not always unwanted.
Consider matters of preference or taste.
Variability in judgments is also expected and welcome in a competitive situation in which the best judgments will be rewarded.
the one who never strays from the consensus remains obscure.
In a market as in nature, selection cannot work without variation.
Matters of taste and competitive settings all pose interesting problems of judgment. But our focus is on judgments in which variability is undesirable.
Disagreements make markets.
Certainly, positive and negative errors in a judgment about the same case will tend to cancel one another out, and we will discuss in detail how this property can be used to reduce noise. But noisy systems do not make multiple judgments of the same case. They make noisy judgments of different cases. If one insurance policy is overpriced and another is underpriced, pricing may on average look right, but the insurance company has made two costly errors.
In noisy systems, errors do not cancel out. They add up.
Noise was like a leak in the basement. It was tolerated not because it was thought acceptable but because it had remained unnoticed. How could that be?
The noise audits suggested that respected professionals—and the organizations that employ them—maintained an illusion of agreement while in fact disagreeing in their daily professional judgments.
Most of us, most of the time, live with the unquestioned belief that the world looks as it does because that’s the way it is. There is one small step from this belief to another: “Other people view the world much the way I do.” These beliefs, which have been called naive realism, are essential to the sense of a reality we share with other people. We rarely question these beliefs. We hold a single interpretation of the world around us at any one time, and we normally invest little effort in generating plausible alternatives to it. One interpretation is enough, and we experience it as true. We
...more
Confidence is nurtured by the subjective experience of judgments that are made with increasing fluency and ease, in part because they resemble judgments made in similar cases in the past.
discomfort of disagreement.
Bad judgment is much easier to identify than good judgment.
Our conclusion is simple: wherever there is judgment, there is noise, and more of it than you think.
How do you know what i think?
The noise audit shattered the illusion of agreement.”
Noise in recurrent decisions is demonstrated by a noise audit,
counterfactual thinking,
If singular decisions are just as noisy as recurrent ones, then the strategies that reduce noise in recurrent decisions should also improve the quality of singular decisions.
From the perspective of noise reduction, a singular decision is a recurrent decision that happens only once.
Judgment can therefore be described as measurement in which the instrument is a human mind. Implicit in the notion of measurement is the goal of accuracy—to approach truth and minimize error.
the most common measure of variability is standard deviation, and we will use it to measure noise in judgments.
A matter of judgment is one with some uncertainty about the answer and where we allow for the possibility that reasonable and competent people might disagree. But there is a limit to how much disagreement is admissible.
Matters of judgment differ from matters of opinion or taste, in which unresolved differences are entirely acceptable.
Matters of judgment,
are defined by the expectation of bounded disagreement.
asked to explain your choice of a number, you would mention a few salient facts but not enough of them for a full accounting of your judgment. The thought process you went through illustrates several features of the mental operation we call judgment:
Selective attention and selective recall are a source of variability across people.
you informally integrated these cues into an overall impression of Gambardi’s prospects. The key word here is informally. You did not construct a plan for answering the question.
