The Book of Why: The New Science of Cause and Effect (Penguin Science)

by Judea Pearl

Why can’t we answer our floss question just by observation? Why not just go into our vast database of previous purchases and see what happened previously when toothpaste cost twice as much? The reason is that on the previous occasions, the price may have been higher for different reasons. For example, the product may have been in short supply, and every other store also had to raise its price. But now you are considering a deliberate intervention that will set a new price regardless of market conditions. The result might be quite different from when the customer couldn’t find a better deal elsewhere. If you had data on the market conditions that existed on the previous occasions, perhaps you could make a better prediction … but what data do you need? And then, how would you figure it out? Those are exactly the questions the science of causal inference allows us to answer. A very direct way to predict the result of an intervention is to experiment with it under carefully controlled conditions. Big-data companies like Facebook know this and constantly perform experiments to see what happens if items on the screen are arranged differently or the customer gets a different prompt (or even a different price).