More on this book
Community
Kindle Notes & Highlights
why his message has remained largely unheeded, and why decision makers continue to rely on their intuition. When they listen to their gut, decision makers hear the internal signal and feel the emotional reward it brings. This internal signal that a good judgment has been reached is the voice of confidence, of “knowing without knowing why.”
psychological biases create system noise when judges are biased in different ways, or to a different extent.
The uniqueness of personality is normally a cause for celebration, but this book is concerned with professional judgments, where variation is problematic and noise is error.
The fact that in these studies level noise is generally not the larger component of system noise is already an important message, because level noise is the only form of noise that organizations can (sometimes) monitor without conducting noise audits.
The most important component of system noise is the one we had initially neglected: stable pattern noise, the variability among judges in their judgments of particular cases.
our conclusions are tentative, but they do reflect a change in how we think about noise—and about how to tackle it.
Not very solid reason to change!!!!
focusing on level noise misses a large part of what individual differences are about. Noise is mostly a product not of level differences but of interactions:
Noise is mostly a by-product of our uniqueness, of our “judgment personality.”
We should be aware accepting real limitations, but not pretend to change nature
We do, however, feel a need to explain abnormal outcomes: the bad ones and, occasionally, the surprisingly good ones—such as the shocking business gamble that pays off.
Abnormal in the subjective expectations, but not statistical
A well-documented psychological bias called the fundamental attribution error is a strong tendency to assign blame or credit to agents for actions and outcomes that are better explained by luck or by objective circumstances.
our normal way of thinking is causal. We naturally attend to the particular, following and creating causally coherent stories about individual cases, in which failures are often attributed to errors, and errors to biases.
taking the statistical view is not easy. We effortlessly invoke causes for the events we observe, but thinking statistically about them must be learned and remains effortful. Causes are natural; statistics are difficult.
even the most enthusiastic proponents of AI agree that algorithms are not, and will not soon be, a universal substitute for human judgment.
We present case studies in five different domains.
Highly skilled people are less noisy, and they also show less bias.
What makes a respect-expert?
Instead, doctrine leaves room for interpretation. Experts still produce judgments, not computations.
noise inevitably occurs. Even identically trained professionals who agree on the doctrine they are applying will drift away from one another in their application of it.
ability to make and explain their judgments with confidence.
same as counterfeiters
confident people have more weight than others, even if they have no reason to be confident. Respect-experts excel at constructing coherent stories.
the CRT can be seen as a measure of people’s propensity to rely on slow, System 2 thinking rather than on fast, System 1 thinking.
To be sure, most of their decisions did not change, but for these kinds of decisions, a shift of one in six can be counted as large.
is “much higher than the general public (and, by extension, most jurors) would likely believe based on longstanding claims about the accuracy of fingerprint analysis.”
In other words, about half of these forensic professionals believe that their colleagues’ judgments are noisy but that their own are not.
mariguana questioner in college. How many do you know that smoke M? Above 50% Do you? Bellow 50%.
. In any judgment, some information is relevant, and some is not. More information is not always better, especially if it has the potential to bias judgments by leading the judge to form a premature intuition.
During the early stages of the COVID-19 pandemic, some doctors initially made diagnoses as a result of judgments reached after considering symptoms; as the pandemic progressed, testing became much more common, and the tests made judgment unnecessary.
no, only people who judgement said could be infected, were given a PCR test.
found to be inclined to assign patients to specific diagnostic categories. For
adhd
It would take a superhuman rater to recall and process accurate, relevant facts about numerous people being evaluated on so many dimensions.
Ray Dalio "Dot collector"
the expectations must be too low, perhaps because of a culture of complacency. Admittedly this interpretation may be valid, but it is also possible that most employees really do meet high expectations. Indeed, this is exactly what we would expect to find in a high-performance organization.
google used to expect people to have 5 goals. If they reach all, their goals were too easy.
To put it more starkly, they are often useless.
This means that you and another interviewer, after seeing the same two candidates in the same panel interview, will still disagree about which of two candidates is better about one-quarter of the time.
A difference makes a difference when it makes a difference. Disagreement between two good ones, or a good and an unqualified ?
Why do first impressions end up driving the outcome of a much longer interview?
Evolution, role of humor. Friend or foe
delayed holistic judgment, can be summarized in a simple prescription: do not exclude intuition, but delay it.
The challenge, the participants soon discovered, was to make the list short, comprehensive, and composed of nonoverlapping assessments. But the task was manageable.
What about interdependencies ?
Conventional wisdom holds that good decisions—especially the very best ones—emerge from the insight and creativity of great leaders. (We especially like to believe this when we are the leader in question.)
We have defined noise as unwanted variability, and if something is unwanted, it should probably be eliminated.
If there is unwanted, there is also wanted. What noise is wanted? The author says that there may be a reason for wanting some noise, yet his arguments are against any noise. Even in cases that are compromising, he chooses less noise.
A test might eliminate noise in diagnoses, but if the test is invasive, dangerous, and costly, and if variability in diagnoses is modest and has only mild consequences, then it might not be worthwhile for all doctors to require all patients to take the test.
“not as uniquely individual human beings, but as members of a faceless, undifferentiated mass to be subjected to the blind infliction”
The authors critique this idea, yet some hygiene suggestions make judges a collective of faceless individuals.
if that better strategy eliminates noise and produces fewer errors, it would have obvious advantages over individualized treatment, even if it reduces or eliminates the opportunity to be heard.
We are not saying that the interest in individualized treatment does not matter. But there is a high price to pay if such treatment leads to all sorts of terrible consequences, including palpable unfairness.
Is a specified speed limit unacceptable?
a speed limit is not a sanction on anyone. It is a mutual agreement that protects my life and property, by protecting others lives and properties.
a noisy system might be good for morale not because it is noisy but because it allows people to decide as they see fit. If employees are allowed to respond to customer complaints in their own way, evaluate their subordinates as they think best, or establish premiums as they deem appropriate, then they might enjoy their jobs more. If the company takes steps to eliminate noise, employees might think that their own agency has been compromised. Now they are following rules rather than exercising their own creativity. Their jobs look more mechanical, even robotic. Who wants to work in a place that
...more
we might be willing to tolerate noise if it makes for a happier and more inspired workforce.
“In the end, most of the efforts to defend noise aren’t convincing.
A strawman fallacy.
If the goal is to reduce noise or decide how and whether to do so (and to what degree),
rules and standards.
the central distinction between rules and standards. Rules are meant to eliminate discretion by those who apply them; standards are meant to grant such discretion.
Rules have an important feature: they reduce the role of judgment.
AI is based on codification of rules.
They might have to make numerous judgments to decide what counts as (for example) “reasonable” and “feasible.”
Justice Potter Stewart obscenity test: "I know it when I see it"
Algorithms work as rules, not standards.
