Trusting Judgements: How to Get the Best out of Experts

by Mark Burgman

Policy- and decision-makers in government and industry constantly face important decisions without full knowledge of all the facts. They rely routinely on expert advice to fill critical scientific knowledge gaps. There are unprecedented opportunities for experts to influence decisions. Yet even the most experienced can be over-confident and error-prone, and the hidden risk is that scientists and other experts can over-reach, often with good intentions, placing more weight on the evidence they provide than is warranted. This book describes how to identify potentially risky advice, explains why group judgements outperform individual estimates, and provides an accessible and up-to-date guide to the science of expert judgement. Finally, and importantly, it outlines a simple, practical framework that will help policy- and decision-makers to ensure that the advice that they receive is relatively reliable and accurate, thus substantially improving the quality of information on which critical decisions are made.

214 pages, Paperback

First published October 21, 2015

Reviews

[Paul Harding · Rating 5 out of 5]

A very comprehensive and knowledgeable examination of the role and use of experts for decision making. It reads very smoothly, has plenty of relevant examples, and also offers a set of practical tools and tips both for experts themselves and for decision makers who rely on them! Highly recommended for anyone involved in policy, science, politics, or analysis / prediction.

[Mitch Flitcroft · Rating 4 out of 5]

This book argues that an expert's belief (in their field of expertise) is usually more accurate than layperson’s belief, but by a lesser margin than we might hope. It discusses epistemic factors that influence beliefs such as types of uncertainty, psychological biases, and the wisdom of crowds. It then discusses the types of epistemic norms that, given these factors, would be most likely to produce accurate beliefs. These include formulating questions that minimise potential bias, consulting several experts that have independent sources of evidence, and more.

Overall, it was a good book about an important topic. However, I was a little disappointed. It was ostensibly an academic book (high-priced, published by a university press, written by a leading expert), but it read much more like a popular-level book. It was short and contained pretty standard content, most of which would be familiar to anyone that’s read anything about critical thinking. To that end, I’d probably only recommend it to somebody unacquainted with the field.

[Jcphene · Rating 5 out of 5]

This is a short foundational book, which shows past use of solitary expert pronouncements have been ineffective. It shows systematic calibration and combination of 3 or more broadly based experts to evaluate complex decisions is far more effective. My favorite line was '...things that makes groups smart are not the same as things that make individuals smart.' The ability of groups to work effectively is driven by diversity. An argument from authority is improper when it is driven solely by the identity of the proponent [p.21].

[Robert Stewart · Rating 5 out of 5]

Very good, very accessible read for professional and layman alike.