How to Measure Anything: Finding the Value of Intangibles in Business

by Douglas W. Hubbard

Risk Paradox If an organization uses quantitative risk analysis at all, it is usually for routine operational decisions. The largest, most risky decisions get the least amount of proper risk analysis.

there are really only three basic reasons why information ever has value to a business: 1. Information reduces uncertainty about decisions that have economic consequences. 2. Information affects the behavior of others, which has economic consequences. 3. Information sometimes has its own market value.

The value of information regarding its effect on human behavior is exactly equal to the value of the difference in human behavior.

incentives from measurements may have unforeseen effects.

The cost of being wrong is the difference between the wrong choice you took and the best alternative available—that is, the one you would have chosen if you had perfect information.

a special case of measurement—the forecast—which is a measurement of likely future outcomes.

Expected Opportunity Loss (EOL)

When a risk-averse person takes a large number of very small bets, their choices will be close to risk neutral. With your own money, you may not consider a 20% chance to lose $100,000 to be exactly equal to a certain reward of $20,000, but you may consider a 20% chance of winning $10 to be very close to $2

All measurements that have value must reduce the uncertainty of some quantity that affects some decision with economic consequences.

The easiest measurement value to compute is the Expected Value of Perfect Information (EVPI).

we need to know the Expected Value of Information, not the Expected Value of Perfect Information.

the value of information tends to rise more quickly with small reductions in uncertainty but levels off as we approach perfect certainty.

It is often assumed that if you have a lot of uncertainty, you need a lot of data to reduce it. In fact, just the opposite is true.

The highest-value measurements almost always are a bit of a surprise to the client.

Measurement Inversion In a business case, the economic value of measuring a variable is usually inversely proportional to how much measurement attention it usually gets.

Because most organizations lack a method for measuring the value of conducting a measurement, they are almost guaranteed to measure all the wrong things.

the old joke about the drunk looking for his watch in the well-lit street, even though he knows he lost it in the dark alley.

to have a truly profound revelation, you almost always have to look at something other than what you have been looking at in the past.

quantifying uncertainty reduction

the early part of any measurement usually is the high-value part.

“Empirical methods” are formal, systematic approaches for making observations to avoid or at least reduce certain types of errors that observations (and observers) are like likely to have.

ask a few questions so that we might be able to determine the appropriate category of measurement methods. Those questions are: • What are the parts of the thing we’re uncertain about? Decompose the uncertain thing so that it is computed from other uncertain things. • How has this (or its decomposed parts) been measured by others? Chances are, you’re not the first to encounter a particular measurement problem, and there may even be extensive research on the topic already. Reviewing the work of others is called “secondary research.” • How do the “observables” identified lend themselves to measurement? You’ve already answered how you observe the thing. Follow through with that to identify how you observe the parts you identified in the first item above. And secondary research may already answer this for you. • How much do we really need to measure it? Take into account the previously computed current state of uncertainty, the threshold, and value of information. These are all clues that point toward the right measurement approach. • What are the sources of error? Think about how observations might be misleading. • What instrument do we select? Based on your answers to the previous questions, identify and design a measurement instrument. Once again, secondary research may provide guidance.

Daniel Fahrenheit’s mercury thermometer quantified what was previously considered the “quality” of temperature.

The measurement instrument, like any tool, gives an advantage to the user.

Instruments generally have six advantages.

1. Instruments detect what you can’t detect.

2. Instruments are more consistent.

3. Instruments can be calibrated to account for error.

4. Instruments deliberately don’t see some things.

5. Instruments record.

6. Instruments take a measurement faster and cheaper than a human.

simply decomposing a variable into the parts that make it up can be an enlightening first step.

Decomposition effect: The phenomenon that the decomposition itself often turns out to provide such a sufficient reduction in uncertainty that further observations are not required.

this assumes the sample was random

Subatomic particles and humans have something in common. The act of observing them causes them both to change behavior.

the Hawthorne Plant of the Western Electric Company

What’s the really simple question that makes the rest of the measurement moot?

everything we know from “experience” is just a sample.