Messy: The Power of Disorder to Transform Our Lives

by Tim Harford

The problem here is known as “overfitting”—what happens when a detailed statistical analysis slavishly follows historical data. Imagine a scatterplot graph with a line or a smooth curve drawn through a cloud of dots to pick out the trend. An overfitted line looks more like a dot-to-dot puzzle, trying to pick out a pattern in the incidence of heart attacks or avalanches that isn’t really there. When new data arrive—new dots—they are unlikely to be anywhere near the wriggling curve. The complex rules are like the overfitted line: designed with too much hindsight, but poor foresight. A cruder rule—draw a straight line or a smooth curve—doesn’t fit the old data so well but will often do better when fresh data arrive. Making the targets more complex can’t be the right answer. A complex measure is just as likely to be gamed, and a simple rule of thumb is often an accurate guide to what is happening.