The Model Thinker: What You Need to Know to Make Data Work for You

by Scott E. Page

The models share three common characteristics: First, they simplify, stripping away unnecessary details, abstracting from reality, or creating anew from whole cloth. Second, they formalize, making precise definitions. Models use mathematics, not words. A model might represent beliefs as probability distributions over states of the world or preferences as rankings of alternatives. By simplifying and making precise, they create tractable spaces within which we can work through logic, generate hypotheses, design solutions, and fit data. Models create structures within which we can think logically. As Wittgenstein wrote in his Tractatus Logico-Philosophicus, “Logic takes care of itself; all we have to do is to look and see how it does it.” The logic will help to explain, predict, communicate, and design. But the logic comes at a cost, which leads to their third characteristic: all models are wrong, as George Box noted.6 That is true of all models; even the sublime creations of Newton that we refer to as laws hold only at certain scales. Models are wrong because they simplify. They omit details. By considering many models, we can overcome the narrowing of rigor by crisscrossing the landscape of the possible.

To sketch the argument for many-model thinking, we begin with a query from poet and dramatist T. S. Eliot: “Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?” To that we might add, where is the information we have lost in all this data?

If the world were turtles all the way—if the world were self-similar—then a model of the top level would apply at every level. But the economy, the political world, and society are not turtles all the way down, nor is the brain. At the sub-micron level, the brain is made up of molecules that form synapses, which in turn form neurons. The neurons combine in networks. The networks overlap in elaborate ways that can be studied with brain imaging. These neuronal networks exist on a scale below that of functional systems such as the cerebellum. Given that the brain differs at each level, we need multiple models, and those models differ. The models that characterize the robustness of neuronal networks bear little resemblance to the molecular biology models used to explain brain cell function, which in turn differ from the psychological models used to explain cognitive biases.

Emergent behaviors that lead to different models at different levels are still somehow related to the properties of models at the lower levels. You may not be able to explain human psychology with a model of the atom, but the limitations which exist within human behavior are still encoded in the properties of physical matter. Studying emergence of behaviors and properties is probably a good way forward in understanding complexity.

Critics of formalism claim that models repackage what we already know, that they pour old wine into shiny mathematical bottles, that we do not need a model to know that two heads are better than one or that he who hesitates is lost. We can learn the value of commitment from reading of Odysseus tying himself to the mast. That criticism fails to recognize that inferences drawn from models take conditional forms: if condition A holds, then result B follows (e.g., if you are packing bins and size is the only constraint, pack the biggest objects first). Lessons drawn from literature or proverbial advice from great thinkers often provide no conditions. If we try to lead our lives or manage others by unconditional rules, we find ourselves lost in a sea of opposite proverbs. Are two heads better than one? Or, do too many cooks spoil the broth? Proverb: Two heads are better than one Opposite: Too many cooks spoil the broth Proverb: He who hesitates is lost Opposite: A stitch in time saves nine Proverb: Tie yourself to the mast Opposite: Keep your options open Proverb: The perfect is the enemy of the good Opposite: Do it well or not at all Proverb: Actions speak louder than words Opposite: The pen is mightier than the sword While opposite proverbs abound, opposite theorems cannot. Within models, we make assumptions and prove theorems. Two theorems that disagree on the optimal action, make different predictions, or offer distinct explanations must make different assumptions.

The actionable information in stories and proverbs is very low resolution. It treats vague generalizations as absolutes, so as not to overburden the reader.

Empirical studies of prediction align with that inference. While adding models improves accuracy (they have to, given the theorems), the marginal contribution of each model falls off after a handful of models. Google found that using one interviewer to evaluate job candidates (instead of picking at random) increases the probability of an above-average hire from 50% to 74%, adding a second interviewer increases the probability to 81%, adding a third raises it to 84%, and using a fourth lifts it to 86%. Using twenty interviewers only increases the probability to a little over 90%. That evidence suggests a limit to the number of relevant ways of looking at a potential hire.