More on this book
Community
Kindle Notes & Highlights
Squared error is similarly irrelevant to the decision of when to leave home to catch a train. For that decision, the consequences of being either one minute late or five minutes late are the same.
Good decision making must be based on objective and accurate predictive judgments that are completely unaffected by hopes and fears, or by preferences and values.
For the elevator company, the first step would be a neutral calculation of the maximum technical load of the elevator under different engineering solutions. Safety becomes a dominant consideration only in the second step, when an evaluative judgment determines the choice of an acceptable safety margin to set the maximum capacity.
decisions depend on underlying predictions, which should be value-neutral. Their goal is accuracy—hitting as close as possible to the bull’s-eye—and MSE is the appropriate measure of error. Predictive judgments will be improved by procedures that reduce noise, as long as they do not increase bias to a larger extent.
The variance of biases across cases—some positive, some negative—is an important source of error and unfairness. Confusingly, this variance is what is often referred to as “bias.”
standard deviation,
normally distributed, it is 1.128 times the standard deviation,
“hanging judges,”
bleeding-heart judges,”
level e...
This highlight has been truncated due to consecutive passage length restrictions.
error is defined here as a deviation from the average; an error may in fact correct an injustice, if...
This highlight has been truncated due to consecutive passage length restrictions.
Variability in level errors will be found in an...
This highlight has been truncated due to consecutive passage length restrictions.
main goal of sentencing is incapacitation (removing the criminal from society), rehabilitation, or deterrence.
judges who think that the main goal is rehabilitation tend to assign shorter prison sentences and more supervised time than do judges who pointed to deterrence or incapacitation.
conservative ideology was also related to severity of sentences.
level noise
system noise
pattern noise.
judges are not equally severe in their sentencing of all cases: they are harsher than their personal average in some and more lenient in others. We call these residual deviations pattern errors.
If you wrote down these pattern errors in each cell of the table, you would find that they add up to zero for every judge (row) and that they also add up to zero for every case (column). However, the pattern errors do not cancel out in their contribution to noise, because the values in all cells are squared for the computation of noise.
pattern noise
reflects a complex pattern in the attitudes of judges to particular cases. One judge, for instance, may be harsher than average in general but relatively more lenient toward white-collar criminals. Another may be inclined to punish lightly but more severely when the offender is a recidivist.
(We use the term pattern noise in the interest of readability. The proper statistical term for pattern noise is judge × case interaction—pronounced “judge-by-case.”
system noise
level noise
pattern...
This highlight has been truncated due to consecutive passage length restrictions.
System Noise2 = Level Noise2 + Pattern Noise2
This within-person variability is conceptually distinct from the stable between-person differences that we have just discussed—but it is difficult to tell these sources of variability apart.
occasion noise.
System noise is undesirable variability in the judgments of the same case by multiple individuals.
Level noise is variability in the average level of judgments by different judges.
Pattern noise is variability in judges’ responses to particular cases.
pattern noise certainly contains some occasion noise,
“Level noise is when judges show different levels of severity. Pattern noise is when they disagree with one another on which defendants deserve more severe or more lenient treatment. And part of pattern noise is occasion noise—when judges disagree with themselves.”
countless factors can influence the player at the foul line: the fatigue of a long game, the mental pressure of a tight score, the cheers of the home court, or the boos of the opposing team’s fans. If someone like Curry or Nash misses, we will invoke one of these explanations. But in truth, we are unlikely to know the exact role these factors play. The variability in a shooter’s performance is a form of noise.
however hard we try to produce the same signature, it is still slightly different on every check.
indulgence.
In reality, our opinions do change without apparent reason.
When wine experts at a major US wine competition tasted the same wines twice, they scored only 18% of the wines identically (usually, the very worst ones).
just like a basketball player who never throws the ball twice in exactly the same way, we do not always produce identical judgments when faced with the same facts on two occasions.
We have described the process that picks an underwriter, a judge, or a doctor as a lottery that creates system noise.
Occasion noise is the product of a second lottery. This lottery picks the moment when the professional makes a judgment, the professional’s mood, the sequence of cases that are fresh in mind, and countless other features of the occasion. This ...
This highlight has been truncated due to consecutive passage length restrictions.
Occasion noise is the variability among these unseen possibilities.
test-retest reliability, or reliability for short)
If we are to control occasion noise, we must try to understand the mechanisms that produce it.
what percentage of the world’s airports are in the United States?
Two researchers, Edward Vul and Harold Pashler, had the idea of asking people to answer this question (and many similar ones) not once but twice. The subjects were not told the first time that they would have to guess again. Vul and Pashler’s hypothesis was that the average of the two answers would be more accurate than either of the answers on its own. The data proved them right. In general, the first guess was closer to the truth than the second, but the best estimate came from averaging the two guesses.
wisdom-of-crowds effect: averaging the independent judgments of different people generally improves accuracy.
Francis Galton, a cousin of Darwin and a famous polymath,
Of course, if questions are so difficult that only experts can come close to the answer, crowds will not necessarily be very accurate.
