The Tyranny of Metrics

by Jerry Z. Muller

Let’s begin with problems of the distortion of information. Measuring the most easily measurable. There is a natural human tendency to try to simplify problems by focusing on the most easily measureable elements.1 But what is most easily measured is rarely what is most important, indeed sometimes not important at all. That is the first source of metric dysfunction. Closely related is measuring the simple when the desired outcome is complex. Most jobs have multiple responsibilities and most organizations have multiple goals. Focusing measurement on just one responsibility or goal often leads to deceptive results. Measuring inputs rather than outcomes. It is often easier to measure the amount spent or the resources injected into a project than the results of the efforts. So organizations measure what they’ve spent, rather than what they produce, or they measure process rather than product. Degrading information quality through standardization. Quantification is seductive, because it organizes and simplifies knowledge. It offers numerical information that allows for easy comparison among people and institutions.2 But that simplification may lead to distortion, since making things comparable often means that they are stripped of their context, history, and meaning.3 The result is that the information appears more certain and authoritative than is actually the case: the caveats, the ambiguities, and uncertainties are peeled away, and nothing does more to create the appearance o...