More on this book
Community
Kindle Notes & Highlights
The low noise was a mechanical effect of statistical aggregation: the noise present in the independent, individual judgments is always reduced by averaging them.
Deliberation had the effect of increasing noise.
Not only were deliberating juries noisier than statistical juries, but they also accentuated the opinions of the individuals composing them.
And if people care about their reputation within the group, they will shift in the direction of the dominant tendency, which will also produce polarization.
found that deliberating juries are noisier than statistical juries.
Group dynamics can amplify this noise.
we explore the reasons for this outcome and show that noise is a major factor in the inferiority of human judgment.
How closely do the predictions co-vary with the outcomes?
A measure that captures this intuition is the percent concordant (PC), which answers a more specific question: Suppose you take a pair of employees at random. What is the probability that the one who scored higher on an evaluation of potential also performs better on the job?
standard measure is the correlation coefficient (r), which varies between 0 and 1 when two variables are positively related.
the correlation between two variables is their percentage of shared determinants.
We can also read the .60 correlation between height and foot size as suggesting that 60% of the causal factors that determine height also determine shoe size.
most judgments are made in a state of what we call objective ignorance, because many things on which the future depends can simply not be known.
we show that objective ignorance affects not just our ability to predict events but even our capacity to understand them—an important part of the answer to the puzzle of why noise tends to be invisible.
informal approach you took to this problem is known as clinical judgment.
multiple regression, produces a predictive score that is a weighted average of the predictors.
optimal weights minimize the MSE (mean squared error)
multiple regression is an example of mechanical prediction.
we will refer to linear models as simple models.
all share a simple structure:
reached the strong conclusion that simple mechanical rules were generally superior to human judgment. Meehl discovered that clinicians and other
reflects the common intuition that the same difference can be inconsequential in one context and critical in another.
judgment was an illusion: the illusion of validity.
Clinicians are not immune to the illusion of validity.
Meehl’s pattern contradicts the subjective experience of judgment, and most of us will trust our experience over a
scholar’s claim.
the evidence for the advantage of the mechanical approach to combining inputs was “massive and consistent.”
simple models beat humans.
people are inferior to statistical models in many ways.
Lewis Goldberg,
Big Five model of personality.
An as-if model that predicts what people will do with reasonable accuracy is useful, even when it is obviously wrong as a description of the process.
This is the case for simple models of judgment.
question that drove Goldberg’s research was how well a simple model of the judge would predict real outcomes.
How much accuracy is lost when the model replaces the judge?
In most cases, the model out-predicted the professional on which it was based.
review of fifty years of research concluded that models of judges consistently outperformed the judges they modeled.
The model-of-the-judge studies reinforce Meehl’s conclusion that the subtlety is largely wasted. Complexity and richness do not generally lead to more accurate predictions.
we need to understand what accounts for the differences between you and the model of you.
the model of you will not reproduce your complex rules—even if you apply them with flawless consistency.
subtle rules will result in a loss of accuracy
Some subtleties are valid, but many are not.
simple model of you will not represent the pattern noise in your judgments.
noisy errors of judgment are not systematically correlated with anything,
they can be considered random.
In short, replacing you with a model of you does two things: it eliminates your subtlety, and it eliminates your pattern noise.
the gains from
subtle rules in human judgment—when they exist—are generally not sufficient to compensate for the detrimental effects of noise.
when the complex rules are valid in principle, they inevitably apply under conditions that are rarely observed.
Errors of measurement at both ends inevitably attenuate the validity of predictions—and rare events are particularly likely to be missed. The advantages of true subtlety are quickly drowned in measurement error.
