Noise

by Daniel Kahneman

A general property of noise is that you can recognize and measure it while knowing nothing about the target or bias.

in public conversations about human error and in organizations all over the world, noise is rarely recognized. Bias is the star of the show. Noise is a bit player, usually offstage.

Sure enough, some people are more receptive than others to bullshit. They can be impressed by “seemingly impressive assertions that are presented as true and meaningful but are actually vacuous.” But here again, this gullibility is not merely a function of permanent, unchanging dispositions. Inducing good moods makes people more receptive to bullshit and more gullible in general; they are less apt to detect deception or identify misleading information. Conversely, eyewitnesses who are exposed to misleading information are better able to disregard it—and to avoid false testimony—when they are in a bad mood.

Recall the basic finding of group polarization: after people talk with one another, they typically end up at a more extreme point in line with their original inclinations.

“People believe they capture complexity and add subtlety when they make judgments. But the complexity and the subtlety are mostly wasted—usually they do not add to the accuracy of simple models.”

“There is so much noise in judgment that a noise-free model of a judge achieves more accurate predictions than the actual judge does.”

When they listen to their gut, decision makers hear the internal signal and feel the emotional reward it brings. This internal signal that a good judgment has been reached is the voice of confidence, of “knowing without knowing why.” But an objective assessment of the evidence’s true predictive power will rarely justify that level of confidence.

Because the event explains itself as it occurs, we are under the illusion that it could have been anticipated.

Our sense of understanding life as it unfolds consists of the steady flow of hindsight in the valley of the normal.

a heuristic for answering a difficult question is to find the answer to an easier one. The substitution of one question for the other causes predictable errors, called psychological biases.

“We form impressions quickly and hold on to them even when contradictory information comes in. This tendency is called excessive coherence.”

“To reduce noise, maybe we should replace our judgments with a ranking?”

Multiple, conflicting cues create the ambiguity that defines difficult judgment problems. Ambiguity also explains why complex problems are noisier than simple ones. The rule is simple: if there is more than one way to see anything, people will vary in how they see it.

The main implication of this view of confidence is that subjective confidence in one’s judgment by no means guarantees accuracy.

Like any other unsurprising story, a success story explains itself once the outcome is known.

“No matter what has been tried over decades to improve [performance management] processes, they continue to generate inaccurate information and do virtually nothing to drive performance.”

1. At the beginning of the process, structure the decision into mediating assessments. (For recurring judgments, this is done only once.) 2. Ensure that whenever possible, mediating assessments use an outside view. (For recurring judgments: use relative judgments, with a case scale if possible.) 3. In the analytical phase, keep the assessments as independent of one another as possible. 4. In the decision meeting, review each assessment separately. 5. On each assessment, ensure that participants make their judgments individually; then use the estimate-talk-estimate method. 6. To make the final decision, delay intuition, but don’t ban it.

“People value and even need face-to-face interactions. They want a real human being to listen to their concerns and complaints and to have the power to make things better. Sure, those interactions will inevitably produce noise. But human dignity is priceless.”

short, we can be sure that there is error if judgments vary for no good reason. Noise is detrimental even when judgments are not verifiable and error cannot be measured. It is unfair for similarly situated people to be treated differently, and a system in which professional judgments are seen as inconsistent loses credibility.