The Tyranny of Metrics Quotes

“In situations where there are no real feasible solutions to a problem, the gathering and publication of performance data serves as a form of virtue signaling. There is no real progress to show, but the effort demonstrated in gathering and publicizing the data satisfies a sense of moral earnestness. In lieu of real progress, the progress of measurement becomes a simulacrum of success.”
― Jerry Z. Muller, The Tyranny of Metrics

“There are things that can be measured. There are things that are worth measuring. But what can be measured is not always what is worth measuring; what gets measured may have no relationship to what we really want to know. The costs of measuring may be greater than the benefits. The things that get measured may draw effort away from the things we really care about. And measurement may provide us with distorted knowledge—knowledge that seems solid but is actually deceptive.”
― Jerry Z. Muller, The Tyranny of Metrics

“just as Soviet managers responded by producing shoddy goods that met the numerical targets set by their overlords, so do schools, police forces, and businesses find ways of fulfilling quotas with shoddy goods of their own:”
― Jerry Z. Muller, The Tyranny of Metrics

“Metric fixation leads to a diversion of resources away from frontline producers toward managers, administrators, and those who gather and manipulate data.”
― Jerry Z. Muller, The Tyranny of Metrics

“The most characteristic feature of metric fixation is the aspiration to replace judgment based on experience with standardized measurement.”
― Jerry Z. Muller, The Tyranny of Metrics

“Accountability ought to mean being held responsible for one’s actions. But by a sort of linguistic sleight of hand, accountability has come to mean demonstrating success through standardized measurement, as if only that which can be counted really counts.”
― Jerry Z. Muller, The Tyranny of Metrics

“Discouraging cooperation and common purpose. Rewarding individuals for measured performance diminishes the sense of common purpose as well as the social relationships that provide the unmeasureable motivation for cooperation and institutional effectiveness.7 Reward based on measured performance tends to promote not cooperation but competition. If the individuals or units respond to the incentives created, rather than aiding, assisting, and advising one another, they strive to maximize their own metrics, ignoring, or even sabotaging, their fellows. As Donald Berwick, a leading medical reformer, has recounted, One hospital CEO described to me his system of profit-center management, in which middle management bonuses depended on local budget performance. I asked him if one of his managers would transfer resources from his department to another’s if it would help the organization as a whole. “Yes,” the CEO answered honestly, “if he were crazy.”
― Jerry Z. Muller, The Tyranny of Metrics

“Discouraging innovation. When people are judged by performance metrics, they are incentivized to do what the metrics measure, and what the metrics measure will be some established goal. But that impedes innovation, which means doing something that is not yet established, indeed hasn’t been tried out. Innovation involves experimentation. Trying out something new entails risk, including the possibility, perhaps probability, of failure.6 When performance metrics discourage risk they inadvertently promote stagnation.”
― Jerry Z. Muller, The Tyranny of Metrics

“Rewarding luck. Measuring outcomes when the people involved have little control over the results is tantamount to rewarding luck. It means that people are rewarded or penalized for outcomes that are actually independent of their efforts. Those penalized rightly feel that they’ve been treated unfairly.”
― Jerry Z. Muller, The Tyranny of Metrics

“Trying to force people to conform their work to preestablished numerical goals tends to stifle innovation and creativity—valuable qualities in most settings. And it almost inevitably leads to a valuation of short-term goals over long-term purposes.”
― Jerry Z. Muller, The Tyranny of Metrics

“Metric fixation is the persistence of these beliefs despite their unintended negative consequences when they are put into practice.6 It occurs because not everything that is important is measureable, and much that is measurable is unimportant.”
― Jerry Z. Muller, The Tyranny of Metrics

“If what is actually measured is a reasonable proxy for what is intended to be measured, and if it is combined with judgment, then measurement can help practitioners to assess their own performance, both for individuals and for organizations. But problems arise when such measures become the criteria used to reward and punish—when metrics become the basis of pay-for-performance or ratings.”
― Jerry Z. Muller, The Tyranny of Metrics

“What has come to be called “Campbell’s Law,” named for the American social psychologist Donald T. Campbell, holds that “[t]he more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
― Jerry Z. Muller, The Tyranny of Metrics

“3.  How useful are more metrics? Remember that measured performance, when useful, is more effective in identifying outliers, especially poor performers or true misconduct. It is likely to be less useful in distinguishing between those in the middle or near the top of the ladder of performance. Plus, the more you measure, the greater the likelihood that the marginal costs of measuring will exceed the benefits. So, the fact that metrics is helpful doesn’t mean that more metrics is more helpful.”
― Jerry Z. Muller, The Tyranny of Metrics

“2.  How useful is the information? Always begin by reminding yourself that the fact that some activity is measureable does not make it worth measuring, indeed, the ease of measuring may be inversely proportional to the significance of what is measured. To put it another way, ask yourself, is what you are measuring a proxy for what you really want to know? If the information is not very useful or not a good proxy for what you’re really aiming at, you’re probably better off not measuring it.”
― Jerry Z. Muller, The Tyranny of Metrics

“1.  What kind of information are you thinking of measuring? The more the object to be measured resembles inanimate matter, the more likely it is to be measureable: that is why measurement is indispensable in the natural sciences and in engineering. When the objects to be measured are influenced by the process of measurement, measurement becomes less reliable. Measurement becomes much less reliable the more its object is human activity, since the objects—people—are self-conscious, and are capable of reacting to the process of being measured. And if rewards and punishments are involved, they are more likely to react in a way that skews the measurement’s validity. By contrast, the more they agree with the goals of those rewards, the more likely they are to react in a way that enhances the measurement’s validity.”
― Jerry Z. Muller, The Tyranny of Metrics

“Costs to productivity. Economists who specialize in measuring economic productivity report that in recent years the only increase in total factor productivity in the American economy has been in the information-technology-producing industries.11 A question that ought to be asked is to what extent the culture of metrics—with its costs in employee time, morale, and initiative, and its promotion of short-termism—has itself contributed to economic stagnation?”
― Jerry Z. Muller, The Tyranny of Metrics

“Degradation of work. Compelling the people in an organization to focus their efforts on the narrow range of what gets measured leads to a degradation of the experience of work. Edmund Phelps, a Nobel Prize winning economist, claims in his book Mass Flourishing: How Grassroots Innovation Created Jobs, Challenge, and Change that one of the virtues of capitalism is its ability to provide “the experience of mental stimulation, the challenge of new problems to solve, the chance to try the new, and the excitement of venturing into the unknown.”9 That is indeed a possibility under capitalism. But those subject to performance metrics are forced to focus their efforts on limited goals, imposed by others, who may not understand the work that they do. For the workers under scrutiny, mental stimulation is dulled, they decide neither the problems to be solved nor how to solve them, and there is no excitement of venturing into the unknown because the unknown is beyond the measureable. In short, the entrepreneurial element of human nature—which extends beyond the owners of enterprises—may be stifled by metric fixation.10 One result is to motivate those with greater initiative and enterprise to move out of mainstream, large-scale organizations where the culture of accountable performance prevails. Teachers move out of public schools to private schools and charter schools. Engineers move out of large corporations to boutique firms. Enterprising government employees become consultants. There is a healthy element in this. But surely the large-scale organizations of our society are the poorer for driving out those most likely to innovate and initiate. The more that work becomes a matter of filling in the boxes by which performance is to be measured and rewarded, the more it will repel those who think outside the box.”
― Jerry Z. Muller, The Tyranny of Metrics

“Discouraging risk-taking. Attempts to measure productivity through performance metrics have other, more subtle effects: they not only promote short-termism, as noted earlier, but also discourage initiative and risk-taking. The intelligence analysts who ultimately located Bin Laden worked on the problem for years. If measured at any point, their productivity would have seemed to be zero. Month after month, their failure rate was 100 percent, until they achieved success. From the perspective of their superiors, allowing the analysts to work on the project for years involved a high degree of risk: the investment in time might not have panned out. Yet really great achievements often depend on such risks. This is typical of situations involving long-term investments of manpower.”
― Jerry Z. Muller, The Tyranny of Metrics

“Diminishing utility. Sometimes, newly introduced performance metrics will have immediate benefits in discovering poorly performing outliers.5 Having gleaned the low-hanging fruit, there is tendency to expect a continuingly bountiful harvest. The problem is that the metrics continue to get collected from everyone. And soon the marginal costs of assembling and analyzing the metrics exceed the marginal benefits. Rule cascades. In an attempt to staunch the flow of faulty metrics through gaming, cheating, and goal diversion, organizations institute a cascade of rules. Complying with them further slows down the institution’s functioning and diminishes its efficiency.”
― Jerry Z. Muller, The Tyranny of Metrics

“Diminishing utility. Sometimes, newly introduced performance metrics will have immediate benefits in discovering poorly performing outliers.5 Having gleaned the low-hanging fruit, there is tendency to expect a continuingly bountiful harvest. The problem is that the metrics continue to get collected from everyone. And soon the marginal costs of assembling and analyzing the metrics exceed the marginal benefits.”
― Jerry Z. Muller, The Tyranny of Metrics

“Costs in employee time. To the debit side of the ledger must also be added the transactional costs of metrics: the expenditure of employee time by those tasked with compiling and processing the metrics—not to speak of the time required to actually read them. That is exacerbated by the “reporting imperative”—the perceived need to constantly generate information, even when nothing significant is going on. Sometimes the metric of success is the number and size of the reports generated, as if nothing is accomplished unless it is extensively documented. Those within the organization end up spending more and more time compiling data, writing reports, and attending meetings at which the data and reports are coordinated. So, as the heterodox management consultants Yves Morieux and Peter Tollman note, employees work longer and harder at activities that add little to the real productiveness of their organization, while sapping their enthusiasm.”
― Jerry Z. Muller, The Tyranny of Metrics

“Promoting short-termism. Measured performance encourages what Robert K. Merton called “the imperious immediacy of interests … where the actor’s paramount concern with the foreseen immediate consequences excludes consideration of further or other consequences.”3 In short, advancing short-term goals at the expense of long-range considerations.”
― Jerry Z. Muller, The Tyranny of Metrics

“Goal displacement through diversion of effort to what gets measured. Goal displacement comes in many varieties. When performance is judged by a few measures, and the stakes are high (keeping one’s job, getting a raise, raising the stock price at the time that stock options are vested), people will focus on satisfying those measures—often at the expense of other, more important organizational goals that are not measured.1 Economists Bengt Holmström and Paul Milgrom have described it in more formal terms as a problem of misaligned incentives: workers who are rewarded for the accomplishment of measurable tasks reduce the effort devoted to other tasks.2 The result is that the metric means comes to replace the organizational ends that those means ought to serve.”
― Jerry Z. Muller, The Tyranny of Metrics

“For the workers under scrutiny, mental stimulation is dulled, they decide neither the problems to be solved nor how to solve them, and there is no excitement of venturing into the unknown because the unknown is beyond the measureable.”
― Jerry Z. Muller, The Tyranny of Metrics

“One result was the publication of the names of confidential informants, including political dissidents, who had spoken with American diplomats in Iran, China, Afghanistan, the Arab world, and elsewhere.6 As a consequence, some of these individuals had to be relocated to protect their lives. More importantly, the revelations made it more difficult for American diplomats to acquire human intelligence in the future, since the confidentiality of conversations could not be relied upon.”
― Jerry Z. Muller, The Tyranny of Metrics

“since if government officials know that all of their ideas and positions may be made public, it inhibits openness, candor, and trust in communications. The predictable result will be for government officials to commit ever less information to writing, either in print or in the form of emails. Instead, they will limit important matters to oral conversation. But that decreases the opportunity to carefully lay out positions.”
― Jerry Z. Muller, The Tyranny of Metrics

“Only when multiple compromises have been made and a deal has been reached can it be subjected to public scrutiny, that is, made transparent.”
― Jerry Z. Muller, The Tyranny of Metrics

“idea that Washington would work better if there were TV cameras monitoring every conversation gets it exactly wrong…. The lack of opportunities for honest dialogue and creative give-and-take lies at the root of today’s dysfunction.”
― Jerry Z. Muller, The Tyranny of Metrics

“Performance metrics as a measure of accountability help to allocate blame when things go badly, but do little to encourage success,28 especially when success requires imagination, innovation, and risk. Indeed, as the economist Frank Knight noted almost a century ago, entrepreneurship entails “immeasurable uncertainty,” which is not susceptible to metric calculation.”
― Jerry Z. Muller, The Tyranny of Metrics