Noise: A Flaw in Human Judgment

by Daniel Kahneman

audits reveal that noise is producing outrageous levels of unfairness, very high costs, or both. If so, the cost of noise reduction is hardly a good reason not to make the effort.

A noise-free scoring system that fails to take significant variables into account might be worse than reliance on (noisy) individual judgments.

try to devise a better noise-reduction strategy—for example, aggregating judgments rather than adopting silly rules or developing wise guidelines or rules rather than foolish ones.

algorithms could eliminate unwanted variability in judgment but also embed unacceptable bias.

algorithm might be biased for two main reasons. First, by design or not, it could use predictors that are highly correlated with race or gender.

Second, discrimination could also come from the source data.

people sometimes discriminate unconsciously and in ways that outside observers, including the legal system, cannot easily see.

whether we can design algorithms that do better than real-world human judges on a combination of criteria that matter: accuracy and noise reduction, and nondiscrimination and fairness.

although a predictive algorithm in an uncertain world is unlikely to be perfect, it can be far less imperfect than noisy and often-biased human judgment.

a noise-reduction strategy is crude, then, as we have urged, the best response is to try to come up with a better strategy—one attuned to a wide range of relevant variables.

A rule-bound system might eliminate noise, which is good, but it might also freeze existing norms and values, which is not so good.

If people use a shared scale grounded in an outside view, they can respond to changing values over time.

noise-reduction efforts need not and should not be permanent.

Some of the most important noise-reduction strategies, such as aggregating judgments, do allow for emerging values.

even when firm rules are in place, a process exists to challenge and rethink them—but not to break them by exercising case-by-case discretion.

If only general principles are in place, noise in their interpretation and enforcement will follow.

Rules are meant to eliminate discretion by those who apply them; standards are meant to grant such discretion.

Rules have an important feature: they reduce the role of judgment.

Those who devise standards effectively export decision-making authority to others. They delegate power.

Algorithms work as rules, not standards.

Setting standards without specifying details can lead to noise, which might be controlled through some of the strategies we have discussed, such as aggregating judgments and using the mediating assessments protocol.

those who decide between rules and standards must focus on the problem of noise, the problem of bias, or both.

Whenever a public or private institution tries to control noise through firm rules, it must always be alert to the possibility that the rules will simply drive discretion underground.

the amount of discretion they grant is closely connected with the level of trust they have in their agents.

but in some cases, noise can be counted as a rights violation, and in general, legal systems all over the world should be making much greater efforts to control noise.

Organizations all over the world see bias as a villain. They are right. They do not see noise that way. They should.

Noise is variability in judgments that should be identical.

When the bias is smaller than one standard deviation, noise is the bigger source of overall error.

wherever there is judgment, there is noise, and more of it than you think.

Level noise is the variability of the average judgments made by different individuals.

stable pattern noise reflects the uniqueness of judges: their response to cases is as individual as their personality.

differences produce is generally the single largest source of system noise.

People’s exaggerated confidence in their predictive judgment underestimates their objective ignorance as well as their biases.

The critical advantage of rules and models is that they are noise-free.

Psychological biases are, of course, a source of systematic error, or statistical bias. Less obviously, they are also a source of noise.

When the goal is accuracy and you expect others to agree with you, you should also consider what other competent judges would think if they were in your place.

excessive coherence, which causes people to distort or ignore information that does not fit a preexisting or emerging story. Overall accuracy suffers when impressions of distinct aspects of a case contaminate each other.

The principle of structuring inspires diagnostic guidelines, such as the Apgar score.

mediating assessments protocol. This protocol breaks down a complex judgment into multiple fact-based assessments and aims to ensure that each one is evaluated independently of the others.

An intuitive choice that is informed by a balanced and careful consideration of the evidence is far superior to a snap judgment.

sequence the information:

Because of cascade effects and group polarization, group discussions often increase noise.

Relative judgments are less noisy than absolute ones, because our ability to categorize objects on a scale is limited, while our ability to make pairwise comparisons is much better.

Your task here is to quantify the diagnostic value of the data you have, expressed as a correlation with the outcome you are predicting.