More on this book
Community
Kindle Notes & Highlights
Table 4: Main steps of the mediating assessments protocol 1. At the beginning of the process, structure the decision into mediating assessments. (For recurring judgments, this is done only once.) 2. Ensure that whenever possible, mediating assessments use an outside view. (For recurring judgments: use relative judgments, with a case scale if possible.) 3. In the analytical phase, keep the assessments as independent of one another as possible. 4. In the decision meeting, review each assessment separately. 5. On each assessment, ensure that participants make their judgments individually; then
...more
You may have recognized here an implementation of several of the decision hygiene techniques we presented in the preceding chapters: sequencing information, structuring the decision into independent assessments, using a common frame of reference grounded in the outside view, and aggregating the independent judgments of multiple individuals.
First, reducing noise can be expensive; it might not be worth the trouble. The steps that are necessary to reduce noise might be highly burdensome. In some cases, they might not even be feasible.
Second, some strategies introduced to reduce noise might introduce errors of their own.
Third, if we want people to feel that they have been treated with respect and dignity, we might have to tolerate some noise.
Fourth, noise might be essential to accommodate new values and hence to allow moral and political evolution.
Fifth, some strategies designed to reduce noise might encourage opportunistic behavior, allowing people to game the system or evade prohibitions.
Sixth, a noisy process might be a good deterrent. If people know that they could be subject to either a small penalty or a large one, they might steer clear of wrongdoing, at least if they are risk-averse.
Finally, people do not want to be treated as if they are mere things, or cogs in some kind of machine.
Shakespeare’s Merchant of Venice is easily read as an objection to noise-free rules and a plea for a role of mercy in law and in human judgment generally.
But if a noise-reduction strategy is crude, then, as we have urged, the best response is to try to come up with a better strategy—one attuned to a wide range of relevant variables. And if that better strategy eliminates noise and produces fewer errors, it would have obvious advantages over individualized treatment, even if it reduces or eliminates the opportunity to be heard.
A rule-bound system might eliminate noise, which is good, but it might also freeze existing norms and values, which is not so good.
Because rules have clear edges, people can evade them by engaging in conduct that is technically exempted but that creates the same or analogous harms. (Every parent of a teenager knows this!)
In short, a noisy system might be good for morale not because it is noisy but because it allows people to decide as they see fit.
Speaking of Dignity “People value and even need face-to-face interactions. They want a real human being to listen to their concerns and complaints and to have the power to make things better. Sure, those interactions will inevitably produce noise. But human dignity is priceless.” “Moral values are constantly evolving. If we lock everything down, we won’t make space for changing values. Some efforts to reduce noise are just too rigid; they would prevent moral change.” “If you want to deter misconduct, you should tolerate some noise. If students are left wondering about the penalty for
...more
When guidelines are tightened so as to eliminate that discretion, they turn into rules. Algorithms work as rules, not standards.
The great difficulty of getting diverse people to agree on noise-reducing rules is one reason why standards, and not rules, are put in place.
Speaking of Rules and Standards “Rules simplify life, and reduce noise. But standards allow people to adjust to the particulars of the situations.” “Rules or standards? First, ask which produces more mistakes. Then, ask which is easier or more burdensome to produce or work with.” “We often use standards when we should embrace rules—simply because we don’t pay attention to noise.”
Obviously, bias is always bad and reducing it always improves accuracy.
Less intuitive is the fact that noise is equally bad and that reducing noise is always an improvement.
wherever there is judgment, there is noise, and more of it than you think.
System noise can be broken down into level noise and pattern noise.
Level noise is the variability of the average judgments made by different individuals.
Their sentencing decisions will produce a different ranking of cases. We call this variability pattern noise (the technical term is statistical interaction).
The main source of pattern noise is stable: it is the difference in the personal, idiosyncratic responses of judges to the same case.
Pattern noise also has a transient component, called occasion noise.
People’s exaggerated confidence in their predictive judgment underestimates their objective ignorance as well as their biases.
What gives us this satisfying confidence is an internal signal, a self-generated reward for fitting the facts and the judgment into a coherent story.
Even simple linear models built on limited data, or simple rules that can be sketched on the back of an envelope, consistently outperform human judges.
Bias has a kind of explanatory charisma, which noise lacks. If we try to explain, in hindsight, why a particular decision was wrong, we will easily find bias and never find noise.
Task-specific skill, intelligence, and a certain cognitive style—best described as being actively open-minded—characterize the best judges.
We now recapitulate six principles that define decision hygiene, describe how they address the psychological mechanisms that cause noise, and show how they relate to the specific decision hygiene techniques we have discussed
The goal of judgment is accuracy, not individual expression.
Algorithmic evaluation is guaranteed to eliminate noise—indeed, it is the only approach that can eliminate noise completely.
Think statistically, and take the outside view of the case.
The outside-view principle favors the anchoring of predictions in the statistics of similar cases.
Structure judgments into several independent tasks.
This technique is analogous to the practice of structured interviews, in which interviewers evaluate one trait at a time and score it before moving to the next one.
mediating assessments protocol. This protocol breaks down a complex judgment into multiple fact-based assessments and aims to ensure that each one is evaluated independently of the others.
Resist premature intuitions.
The unwillingness of decision makers to give up this rewarding signal is a key reason for the resistance to the use of guidelines and algorithms and other rules that tie their hands.
sequence the information: professionals who make judgments should not be given information that they don’t need and that could bias them, even if that information is accurate.
Obtain independent judgments from multiple judges, then consider aggregating those judgments.
Because of cascade effects and group polarization, group discussions often increase noise.
Averaging independent judgments is guaranteed to reduce system noise (but not bias).
Favor relative judgments and relative scales.
Relative judgments are less noisy than absolute ones, because our ability to categorize objects on a scale is limited, while our ability to make pairwise comparisons is much better.
