The Tyranny of Metrics

by Jerry Z. Muller

Those who suffer from Obsessive Measurement Disorder, Natsios writes, ignore “a central principle of development theory—that those development programs that are most precisely and easily measured are the least transformational, and those programs that are the most transformational are the least measureable.”

Making internal deliberations open to public disclosure—that is, transparent—is counterproductive, Sunstein argues, since if government officials know that all of their ideas and positions may be made public, it inhibits openness, candor, and trust in communications.

When people are judged by performance metrics, they are incentivized to do what the metrics measure, and what the metrics measure will be some established goal. But that impedes innovation, which means doing something that is not yet established, indeed hasn’t been tried out. Innovation involves experimentation. Trying out something new entails risk, including the possibility, perhaps probability, of failure.6 When performance metrics discourage risk they inadvertently promote stagnation.

For the workers under scrutiny, mental stimulation is dulled, they decide neither the problems to be solved nor how to solve them, and there is no excitement of venturing into the unknown because the unknown is beyond the measureable. In short, the entrepreneurial element of human nature—which extends beyond the owners of enterprises—may be stifled by metric fixation.

Product just telling where to go and how

As we’ve seen time and again, measurement is not an alternative to judgment: measurement demands judgment: judgment about whether to measure, what to measure, how to evaluate the significance of what’s been measured, whether rewards and penalties will be attached to the results, and to whom to make the measurements available.

Measurement instruments, such as tests, are invaluable, but they are most useful for internal analysis by practitioners rather than for external evaluation by public audiences who may fail to understand their limits.

In the end, there is no silver bullet, no substitute for actually knowing one’s subject and one’s organization, which is partly a matter of experience and partly a matter of unquantifiable skill. Many matters of importance are too subject to judgment and interpretation to be solved by standardized metrics. Ultimately, the issue is not one of metrics versus judgment, but metrics as informing judgment, which includes knowing how much weight to give to metrics, recognizing their characteristic distortions, and appreciating what can’t be measured. In recent decades, too many politicians, business leaders, policymakers, and academic officials have lost sight of that.