More on this book
Community
Kindle Notes & Highlights
Instead of offering a gut feeling or some kind of global hunch, they ask and try to answer an assortment of subsidiary questions.
perpetual beta, the degree to which one is committed to belief updating and self-improvement.”
Training made a difference, teaming made a larger one, and selection had an even larger effect.
Their answer was simple: all three interventions worked primarily by reducing noise.
“Teaming—unlike training… allows forecasters to harness the information.”
Aggregating the estimates of higher-validity judges will further improve accuracy. Yet another gain in accuracy can be obtained by combining judgments that are both independent and complementary.
validity of pooled judgments increases faster when the judgments are uncorrelated with one another than when they are redundant.
Regardless of diversity, aggregation can only reduce noise if judgments are truly independent.
“variation in skill can explain 44% of the variation in diagnostic decisions,”
doctors are significantly more likely to order cancer screenings early in the morning than late in the afternoon.
The Apgar score exemplifies how guidelines work and why they reduce noise.
Yet guidelines succeed in reducing noise because they decompose a complex decision into a number of easier subjudgments on predefined dimensions.
“the reliance on the patient’s subjective symptoms, the clinician’s interpretation of the symptoms, and the absence of objective measure (such as a blood test) implant the seeds of diagnostic unreliability of psychiatric disorders.”
rankings are less noisy than ratings.
structuring a complex judgment into several dimensions. Structuring is an attempt to limit the halo effect, which usually keeps the ratings of one individual on different dimensions within a small range.
ranking reduces both pattern noise and level noise.
not impossible that they all “meet expectations,” if these expectations have been defined ex ante
it is also possible that most employees really do meet high expectations.
mandating that a set percentage of employees be rated as failing to meet (absolute) expectations is not just cruel; it is absurd.
Whenever judgments are forced onto an inappropriate scale, either because a relative scale is used to measure an absolute performance or because judges are forced to distinguish the indistinguishable, the choice of the scale mechanically adds noise.
At a minimum, performance rating scales must be anchored on descriptors that are sufficiently specific to be interpreted consistently.
raters are trained to recognize different dimensions of performance.
Frame-of-reference training has been known for decades and provides demonstrably less noisy and more accurate ratings.
this is a very good correlation by social science standards—but not a very good one on which to base your decisions.
variability is largely the product of pattern noise, the difference in interviewers’ idiosyncratic reactions to a given interviewee.
initial impressions have a deep effect on the way the interview proceeds.
our interpretation of facts is colored by prior attitudes.
impressions formed in an interview are vivid, and the interviewer is usually confident about them.
additional interviews added almost no predictive validity to what was achieved by the first four.
interviewers rate the candidate separately, before they communicate with one another.
aggregation works—but only if the judgments are independent.
structured complex judgment is defined by three principles: decomposition, independence, and delayed holistic judgment.
decomposition, breaks down the decision into components, or mediating assessments.
it focuses the judges on the imp...
This highlight has been truncated due to consecutive passage length restrictions.
independence, requires that information on each assessment be collected independently.
collect data about each assessment in the evaluation structure and to assign a score to the candidate on each assessment.
structured interviews are far more predictive of future performance than are traditional, unstructured ones.
Research has shown that work sample tests are among the best predictors of on-the-job performance.
The third principle of structured judgment, delayed holistic judgment, can be summarized in a simple prescription: do not exclude intuition, but delay it.
allows judgment and intuition in its decision-making process only after all the evidence has been collected and analyzed.
mediating assessments protocol. It incorporates most of the decision hygiene strategies that we have introduced in the preceding chapters.
similarity between the evaluation of candidates and the evaluation of options in big decisions: options are like candidates.
deciding on the major aspects of the acquisition that should be assessed
final list of seven assessments was superficially similar to the table of contents the board would expect in a regular report presenting an acquisition proposal.
provide an objective, independent evaluation on each of the mediating assessments.
base rate, the percentage of comparable transactions that are approved.
make evaluations as comparative as possible, because relative judgments are better than absolute ones.
When excessive coherence is kept in check, reality is not as coherent as most board presentations make it seem.
decision hygiene techniques we presented in the preceding chapters: sequencing information, structuring the decision into independent assessments, using a common frame of reference grounded in the outside view, and aggregating the independent judgments of multiple individuals.
whether an objection is convincing depends on the particular noise-reduction strategy to which it is meant to apply.
