Noise

by Daniel Kahneman

wherever there is judgment, there is noise—

Consider another mechanism that many companies resort to: postmortems of unfortunate judgments. As a learning mechanism, postmortems are useful. But if a mistake has truly been made—in the sense that a judgment strayed far from professional norms—discussing it will not be challenging.

The calling out of egregious mistakes and the marginalization of bad colleagues will not help professionals become aware of how much they disagree when making broadly acceptable judgments. On the contrary, the easy consensus about bad judgments may even reinforce the illusion of agreement. The true lesson, about the ubiquity of system noise, will never be learned.

Our conclusion is simple: wherever there is judgment, there is noise, and more of it than you think.

Judgment can therefore be described as measurement in which the instrument is a human mind.

Exactly how much disagreement is acceptable in a judgment is itself a judgment call and depends on the difficulty of the problem.

Selective attention and selective recall are a source of variability across people.

What made you feel you got the judgment right, or at least right enough to be your answer? We suggest this feeling is an internal signal of judgment completion, unrelated to any outside information.

The essential feature of this internal signal is that the sense of coherence is part of the experience of judgment. It is not contingent on a real outcome. As a result, the internal signal is just as available for nonverifiable judgments as it is for real, verifiable ones.

One approach to the evaluation of the process of judgment is to observe how that process performs when it is applied to a large number of cases.

Another question that can be asked about the process of judgment is whether it conforms to the principles of logic or probability theory.

what they should be trying to achieve, normatively speaking, is the judgment process that would produce the best judgment over an ensemble of similar cases.

Evaluative judgments partly depend on the values and preferences of those making them, but they are not mere matters of taste or opinion.

System noise is inconsistency, and inconsistency damages the credibility of the system.

decision requires both predictive and evaluative judgments.”

in professional judgments of all kinds, whenever accuracy is the goal, bias and noise play the same role in the calculation of overall error. In some cases, the larger contributor will be bias; in other cases it will be noise (and these cases are more common than one might expect). But in every case, a reduction of noise has the same impact on overall error as does a reduction of bias by the same amount.

Bias is simply the average of errors, which in this case is also 10%.

we need a “scoring rule” for errors, a way to weight and combine individual errors into a single measure of overall error. Fortunately, such a tool exists. It is the method of least squares,

Gauss proposed a rule for scoring the contribution of individual errors to overall error. His measure of overall error—called mean squared error (MSE)—is the average of the squares of the individual errors of measurement.

both suggest, bias and noise play identical roles in the error equation. They are independent of each other and equally weighted in the determination of overall error.

people are very keen to get perfect hits and highly sensitive to

small errors, but they hardly care at all about the difference between two large errors.

Admittedly, reducing noise would be less of a priority if bias were much larger than noise.

the use of MSE as the measure of overall error. The rule is appropriate for purely predictive judgments, including forecasts and estimates, all of which aim to approach a true value with maximum accuracy (the least bias) and precision (the least noise).

well-known phenomenon known as the wisdom-of-crowds effect: averaging the independent judgments of different people generally improves accuracy.

The reason is basic statistics: averaging several independent judgments (or measurements) yields a new judgment, which is less noisy, albeit not less biased, than the individual judgments.

And this result certainly provides a rationale for the age-old advice to decision makers: “Sleep on it, and think again in the morning.”

You can do this either after some time has passed—giving yourself distance from your first opinion—or by actively trying to argue against yourself to find another perspective on the problem. Finally, regardless of the type of crowd, unless you have very strong reasons to put more weight on one of the estimates, your best bet is to average them.

There is at least one source of occasion noise that we have all noticed: mood.

you are not the same person at all times. As your mood varies (something you are, of course, aware of), some features of your cognitive machinery vary with it (something you are not fully aware of).

This behavior reflects a cognitive bias known as the gambler’s fallacy: we tend to underestimate the likelihood that streaks will occur by chance.

you are more similar to yourself yesterday than you are to another person today.

if early speakers seem to like something or want to do something, others might assent. At least this is so if they do not have reason to distrust them and if they lack a good reason to think that they are wrong.

As the number of people who share the same view gets larger, relying on them becomes smarter still. Nonetheless, there are two problems. First, people tend to neglect the possibility that most of the people in the crowd are in a cascade, too—and are not making independent judgments of their own.

Second, informational cascades can lead groups of people in truly terrible directions. After all, Arthur might have been wrong about Thomas.

At a company or in government, people might silence themselves so as not to appear uncongenial, truculent, obtuse, or stupid. They want to be team players. That is why they follow the views and actions of others. People think that they know what is right or probably right, but they nonetheless go along with the apparent consensus of the group, or the views of early speakers, to stay in the group’s good graces.

Very similar groups can end up in divergent places because of social pressures.

Organizations and their leaders should take steps to control noise in the judgments of their individual members. They should also manage deliberating groups in a way that is likely to reduce noise, not amplify it. The