How to Measure Anything: Finding the Value of Intangibles in Business

by Douglas W. Hubbard

A different and disturbing trend in management decision making is to develop a type of weighted score where the score and the weight are both subjective scales with arbitrary point values, not z-scores like those Dawes used. Like the simple linear models discussed previously, these methods might ask a project portfolio manager to rate a proposed project in categories such as “strategic alignment,” “organizational risk,” and so on. Most of these methods have between 4 and 12 categories of evaluation, but some have over 100. The proposed project is typically given a score of, say, 1 to 5 in each of these categories. The scores in each category are then multiplied by a weighting factor—perhaps also a scale of 1 to 5—which is meant to account for the relative importance of each of the scores categorized. The weighting factors are usually standardized for a given company so that all projects are evaluated by comparable criteria. The adjusted scores are then totaled to give an overall score for the proposed project. Scores are methods of attempting to express relative worth, preference, and so on, without employing a real unit of measure. Although scoring is fairly called one type of ordinal measurement system we discussed in Chapter 3, I’ve always considered an arbitrary score to be a sort of measurement wannabe. The problems with this method are one of the key criticisms I make about popular risk assessment methods in my second book The Failure of Risk Management: Why It’s Bro...