More on this book
Community
Kindle Notes & Highlights
interjudge disparities increased significantly after 2005.
Mandatory guidelines reduce bias as well as noise.
law without order.
First, judgment is difficult because the world is a complicated, uncertain place.
Disagreement is unavoidable wherever judgment is involved.
Second, the extent of these disagreements is much greater than we expect.
System noise, that is, unwanted variability in judgments that should ideally be identical, can create rampant injustice, high economic costs, and errors of many kinds.
Third, noise can be reduced.
Fourth, efforts at noise reduction often raise objections and run into serious difficulties.
judgment lotteries we talk about allocate nothing. They just produce uncertainty.
the median difference in underwriting was 55%,
A defining feature of system noise is that it is unwanted, and we should stress right here that variability in judgments is not always unwanted.
Diversity of tastes is welcome and entirely expected.
Disagreements make markets.
System noise plagues many organizations: an assignment process that is effectively random often decides which doctor sees you in a hospital, which judge hears your case in a courtroom, which patent examiner reviews your application, which customer service representative hears your complaint, and so
In noisy systems, errors do not cancel out. They add up.
Noise was like a leak in the basement. It was tolerated not because it was thought acceptable but because it had remained unnoticed.
The noise audits suggested that respected professionals—and the organizations that employ them—maintained an illusion of agreement while in fact disagreeing in their daily professional judgments.
“Other people view the world much the way I do.” These beliefs, which have been called naive realism,
developed confidence in her judgment mainly by exercising it.
noise is a consequence of the informal nature of judgment. However, as we will see throughout this book, the amount of noise observed when an organization takes a serious look almost always comes as a shock. Our conclusion is simple: wherever there is judgment, there is noise, and more of it than you think.
we operate under the wrong assumption that another expert would produce a similar judgment.”
defined noise as undesirable variability in judgments of the same problem.
From the perspective of noise reduction, a singular decision is a recurrent decision that happens only once.
Judgment can therefore be described as measurement in which the instrument is a human mind.
Implicit in the notion of measurement is the goal of accuracy—to approach truth and minimize error.
Although accuracy is the goal, perfection in achieving this goal is never achieved even in scientific measurement, much less in judgment.
Variability across individuals is a biological given;
Within the same person, there is variability, too.
Noise constitutes the variability of your results, analogous to the scatter of shots we saw earlier.
In statistics, the most common measure of variability is standard deviation, and we will use it to measure noise in judgments.
When we make a prediction, we attempt to come close to a true value.
like a measuring instrument, the human mind is imperfect—it is both biased and noisy.
Matters of judgment differ from matters of opinion or taste, in which unresolved differences are entirely acceptable.
Selective attention and selective recall are a source of variability across people.
Without being fully aware of what you were doing, your mind worked to construct a coherent impression
Since each of these three steps in a complex judgment process entails some variability, we should not be surprised to find a lot of noise in answers
In measurement terms, the first problem illustrates within-person reliability, and the second illustrates between-person reliability.
The Gambardi exercise is an example of a nonverifiable predictive judgment, for two separate reasons: Gambardi is fictitious and the answer is probabilistic.
Verifiability does not change the experience of judgment.
We suggest this feeling is an internal signal of judgment completion, unrelated to any outside
the confidence it implies is inconsistent with the messy, ambiguous, conflicting evidence provided.
The aim of judgment, as you experienced it, was the achievement of a coherent solution.
One approach to the evaluation of the process of judgment is to observe how that process performs when it is applied to a large number of cases.
whether it conforms to the principles of logic or probability theory.
two ways of evaluating a judgment: by comparing it to an outcome and by assessing the quality of the process that led to it.
What they are effectively trying to achieve, regardless of verifiability, is the internal signal of completion provided by the coherence between the facts of the case and the judgment.
Sentencing a felon is
not a prediction. It is an evaluative judgment
Evaluative judgments partly depend on the values and preferences of those making them, but they are not mere matters of taste or opinion.
