More on this book
Community
Kindle Notes & Highlights
no other formula would be compatible with your intuition that the mean is the best estimate.
Over two centuries later, it remains the standard way to evaluate errors wherever achieving accuracy is the goal.
The weighting of errors by their square is central to statistics.
two components with which you are now familiar: bias—the average error—and a residual “noisy error.”
noise is the standard deviation of measurements,
identical roles in the error equation. They are independent of each other and equally weighted in the determination of overall error.
straightforward: bias and noise are interchangeable in the error equation,
In terms of overall error, noise and bias are independent: the benefit of reducing noise is the same, regardless of the amount of bias.
Noise reduction seems to have made the forecasts more precisely wrong—
erroneous intuition about bias.
It is average error, which is the distance between the peak of the bell curve and the true value.
result is that MSE is the same in both panels:
same effect on MSE.
people’s intuitions in this regard are almost the mirror image of what they should be:
your emotional reaction to results may be incompatible with the achievement of accuracy as science defines it.
Achieving noise reduction will ensure that bias reduction is next on the company’s agenda.
The error equation is the intellectual foundation of this book.
Furthermore, even if errors could be specified, their costs would rarely be symmetrical and would be unlikely to be precisely proportional to their square.
A widely accepted maxim of good decision making is that you should not mix your values and your facts.
Predictive judgments will be improved by procedures that reduce noise, as long as they do not increase bias to a larger extent.
reducing bias and noise by the same amount has the same effect on accuracy.” “Reducing noise in predictive judgment is always useful,
split 84 to 16 between those that are above and below the true value, there is a large bias—that’s when bias and noise are equal.”
“Predictive judgments are involved in every decision, and accuracy should be their only goal.
We refer to these deviations as level errors.
other component of noise pattern noise.
call these residual deviations pattern errors.
The proper statistical term for pattern noise is judge × case interaction—
these patterns are not mere chance:
You may have noticed that the decomposition of system noise into level noise and pattern noise follows the same logic as the error equation
This time, the equation can be written as follows: System Noise2 = Level Noise2 + Pattern Noise2
all these cases, there will be pattern noise, with different judges producing different rankings of the cases.
Our name for the variability that is due to transient effects is occasion noise.
System noise is undesirable variability in the judgments of the same case by multiple individuals.
same analysis can be applied to any noise audit—
we do not always produce identical judgments when faced with the same facts on two occasions.
Occasion noise is the variability among these unseen possibilities.
research on variability in professional judgment (technically known as test-retest reliability, or reliability for short) included many studies in which the experts made the same judgment twice in the same session. Not surprisingly, they tended to agree with themselves.
well-known phenomenon known as the wisdom-of-crowds effect: averaging the independent judgments of different people generally improves accuracy.
The reason is basic statistics: averaging several independent judgments (or measurements) yields a new judgment, which is less noisy, albeit not less biased, than the individual judgments.
find out if the same effect extends to occasion noise:
combining two...
This highlight has been truncated due to consecutive passage length restrictions.
answer is yes.
the crowd within.
“You can gain about 1/10th as much from asking yourself the same question twice as you can from getting a second opinion from someone else.”
named dialectical bootstrapping,
you can get independent opinions from others, do it—this real wisdom of crowds is highly likely to improve your judgment.
regardless of the type of crowd, unless you have very strong reasons to put more weight on one of the estimates, your best bet is to average them.
“Responses made by a subject are sampled from an internal probability distribution, rather than deterministically selected on the basis of all the knowledge a subject has.”
occasion noise affects all our judgments, all the time.
mood has a measurable influence on what you think:
