More on this book
Community
Kindle Notes & Highlights
“Refugee Roulette.”
decision hygiene.
ukases
More generally, people who deal with organizations expect a system that reliably delivers consistent judgments.
They do not expect
system ...
This highlight has been truncated due to consecutive passage length restrictions.
naive realism,
true lesson, about the ubiquity of system noise, will never be learned.
noise is a consequence of the informal nature of judgment.
Our conclusion is simple: wherever there is judgment, there is noise, and more of it than you think.
“Wherever there is judgment, there is noise—and more of it than we think.”
We have defined noise as undesirable variability in judgments of the same problem.
There is no direct way to observe the presence of noise in singular decisions.
if we think counterfactually, we know for sure that noise is there.
our inability to observe variability would not make the decision less noisy.
From the perspective of noise reduction, a singular decision is a recurrent decision that happens only once.
very concept of judgment involves a reluctant acknowledgment that you can never be certain that a judgment is right.
allow for the possibility that reasonable and competent people might disagree.
Selective attention and selective recall are a source of variability across people.
illustrate two types of noise. The variability of judgments over successive trials with the stopwatch is noise within a single judge (yourself), whereas the variability of judgments of the Gambardi case is noise between different judges.
illustrates within-person reliability, and the second illustrates between-person reliability.
example of a nonverifiable predictive judgment, for two separate reasons: Gambardi is fictitious and the answer is probabilistic.
forecasts may be too long term for the professionals who make them to be brought to account—
fear of being exposed concentrates the mind.
This similarity is important to psychological research, much of which uses made-up problems.
We suggest this feeling is an internal signal of judgment completion, unrelated to any outside information.
The aim of judgment, as you experienced it, was the achievement of a coherent solution.
second way to evaluate judgments.
It consists in evaluating the process of judgment.
Another question that can be asked about the process of judgment is whether it conforms to the principles of logic or probability theory.
All the procedures we recommend in this book to reduce bias and noise aim to adopt the judgment process that would minimize error over an ensemble of similar cases.
Sentencing a felon is not a prediction. It is an evaluative judgment that seeks to match the sentence to the severity of the crime.
trade-offs are resolved by evaluative judgments.
agree that a level of disagreement that turns a judgment into a lottery is problematic.
People who are affected by evaluative judgments expect the values these judgments reflect to be those of the system, not of the individual judges.
System noise is inconsistency, and inconsistency damages the credibility of the system.
All we need to measure noise is multiple judgments of the same problem.
scatter in their forecasts is noise.
decision requires both predictive and evaluative judgments.”
The different errors add up; they do not cancel out.
in professional judgments of all kinds, whenever accuracy is the goal, bias and noise play the same role in the calculation of overall error.
the measurement and reduction of noise should have the same high priority as the measurement and reduction of bias.
Bias is simply the average of errors,
reduce noise, too? How would the value of such an improvement compare with the value of reducing bias?
mean squared error (MSE)—is the average of the squares of the individual errors of measurement.
The mean contains more information; it is affected by the size of the numbers, while the median is affected only by their order.
intuition about the mean being the best estimate is correct,
arithmetic mean as the value for which error is minimized.
MSE:
squaring gives large errors a far greater weight than it gives small ones.
