More on this book
Community
Kindle Notes & Highlights
In any system in which judges are assumed to be interchangeable and assigned quasi-randomly, large disagreements about the same case violate expectations of fairness and consistency.
System noise is inconsistency, and inconsistency damages the credibility of the system.
All we need to measure noise is multiple judgments of the same problem.
An important question, therefore, is how, and how much, bias and noise contribute to error.
in professional judgments of all kinds, whenever accuracy is the goal, bias and noise play the same role in the calculation of overall error.
Bias is simply the average of errors,
measure of overall error—called mean squared error (MSE)—is the average of the squares of the individual errors of measurement.
This is a key feature of MSE: squaring gives large errors a far greater weight than it gives small ones.
They are independent of each other and equally weighted in the determination of overall error.
In terms of overall error, noise and bias are independent: the benefit of reducing noise is the same, regardless of the amount of bias.
all of which aim to approach a true value with maximum accuracy (the least bias) and precision (the least noise).
The error equation does not apply to evaluative judgments, however, because the concept of error, which depends on the existence of a true value, is far more difficult to apply.
Predictive judgments will be improved by procedures that reduce noise, as long as they do not increase bias to a larger extent.
variability in judgments of the same case is still undesirable—it is system noise.
For the average case, the mean sentence was 7.0 years, and the standard deviation around that mean was 3.4 years.
New information, unless it is decisive, provides more opportunities for judges to differ from one another.
variation among judges
these deviations as level errors.
The general conclusion is that the average level of sentencing functions like a personality trait.
We use the term level noise for the variability of the judges’ average judgments, which is identical to the variability of level errors.
differences in average severity across individual judges.
they are harsher than their personal average in some and more lenient in others. We call these residual deviations pattern errors.
proper statistical term for pattern noise is judge × case interaction—pronounced “judge-by-case.”
random error.
Our name for the variability that is due to transient effects is occasion noise.
The variability in a shooter’s performance is a form of noise.
It is less easy to observe the variability of our minds.
we do not always produce identical judgments when faced with the same facts on two occasions.
direct measurements of occasion noise are hard to obtain whenever cases are easily memorable.
test-retest reliability, or reliability
mood has a measurable influence on what you think: what you notice in your environment, what you retrieve from your memory, how you make sense of these signals.
mood has another, more surprising effect: it also changes how you think.
People who are in a good mood are more likely to let their biases affect their thinking.
Inducing good moods makes people more receptive to bullshit and more gullible in general; they are less apt to detect deception or identify misleading information.
an important truth: you are not the same person at all times.
stress and fatigue.
cognitive bias known as the gambler’s fallacy: we tend to underestimate the likelihood that streaks will occur by chance.
But it pales in comparison with the variability between judges:
Even in this tightly controlled setting, exactly what factors drive occasion noise was a mystery.
They were testing for a particular driver of noise: social influence.
The key finding was that group rankings were wildly disparate: across different groups, there was a great deal of noise.
In politics, as in music, a great deal depends on social influences and, in particular, on whether people see that other people are attracted or repelled.
political positions can be just like songs, in the sense that their ultimate fate can depend on initial popularity.
independence is a prerequisite for the wisdom of crowds. If people are not making their own judgments and are relying instead on what other people think, crowds might not be so wise after all.
social influences are a problem because they reduce “group diversity without diminishing the collective error.”
two problems. First, people tend to neglect the possibility that most of the people in the crowd are in a cascade,
Second, informational cascades can lead groups of people in truly terrible directions.
Internal discussions often create greater confidence, greater unity, and greater extremism, frequently in the form of increased enthusiasm.
Deliberation had the effect of increasing noise.
noise is a major factor in the inferiority of human judgment.
