The Book of Why: The New Science of Cause and Effect

by Judea Pearl

None of the letters k, B, or P is in any mathematical way privileged over any of the others. How then can we express our strong conviction that it is the pressure that causes the barometer to change and not the other way around? And if we cannot express even this, how can we

hope to express the many other causal convictions that do not have mathematical formulas, such as that the rooster’s crow does not cause the sun to rise? My college professors could not do it and never complained. I would be willing to bet that none of yours ever did either. We now understand why: never were they shown a mathematical language of causes; nor were they shown its benefits.

never were they shown a mathematical language of causes; nor were they shown its benefits. It is in fact an indictment of science that it has neglected to develop such a language for so many generations.

No representation of “common sense”

scientific tools are developed to meet scientific needs. Precisely because we are so good at handling questions about switches, ice cream, and barometers, our need for special mathematical machinery to handle them was not obvious.

Precisely because we are so good at handling questions about switches, ice cream, and barometers, our need for special mathematical machinery to handle them was not obvious. But as scientific curiosity increased and we began posing causal questions in complex legal, business, medical, and policy-making situations, we found ourselves lacking the tools and principles that mature science should provide.

Despite heroic efforts by the geneticist Sewall Wright (1889–1988), causal vocabulary was virtually prohibited for more than half a century. And when you prohibit speech, you prohibit thought and stifle principles, methods, and tools.

The rooster’s crow is highly correlated with the sunrise; yet it does not cause the sunrise.

participate in the “data economy.” But I hope with this book to convince you that data are profoundly dumb.

But I hope with this book to convince you that data are profoundly dumb. Data

can tell you that the people who took a medicine recovered faster than those who did not take it,...

calculus of causation consists of two languages: causal diagrams, to express what we know, and a symbolic language, resembling algebra, to express what we want to know. The causal diagrams are simply dot-and-arrow pictures that summarize our existing scientific knowledge. The dots represent quantities of interest, called “variables,” and the arrows represent known or suspected causal relationships between those variables—namely, which variable “

to

which o...

If you can navigate using a map of one-way streets, then you can understand causal diagrams, and you can solve the type of questions posed at the beginning of this introduction.

Side by side with this diagrammatic “language of knowledge,” we also have a symbolic “language of queries” to express the questions we want answers to. For

counterfactuals are not products of whimsy but reflect the very structure

world model.

My emphasis on language also comes from a deep conviction that language shapes our thoughts. You cannot answer a question that you cannot ask, and you cannot ask a question that you have no words for.

For this reason, some statisticians to this day find it extremely hard to understand why some knowledge lies outside the province of statistics and why data alone cannot make up for lack of scientific knowledge.

Another advantage causal models have that data mining and deep learning lack is adaptability.

That’s all that a deep-learning program can do: fit a function to data. On the other hand, if she possessed a model of how the drug operated and its causal structure remained intact in the new location, then the estimand she obtained in training would remain valid. It could be applied to the new data to generate a new population-specific prediction function.

think of a cause as something that makes a difference, and the difference it makes must be a difference from what would have happened without

it.”

you are smarter than your data.

Data do not understand causes and effects; humans do.

I hope that the new science of causal inference will enable us to better understand how we do it, because there is no better way to understand ourselves than by emulating ourselves. In the age of computers, this new understanding also brings with it the prospect of amplifying our inn...

causal explanations, not dry facts, make up the bulk of our knowledge, and should be the cornerstone of machine intelligence.

Humans acquired the ability to modify their environment and their own abilities at a dramatically faster rate.

In his book Sapiens, historian Yuval Harari posits that our ancestors’ capacity to imagine nonexistent things was the key to everything, for it allowed them to communicate better.

FIGURE 1.1. Perceived

when we seek to emulate human thought on a computer, or indeed when we try to solve unfamiliar scientific problems, drawing an explicit dots-and-arrows picture is extremely useful.

my research on machine learning has taught me that a causal learner must master at least three distinct levels of cognitive ability: seeing, doing, and imagining.

The second, doing, entails predicting the effect(s) of deliberate alterations of the environment and choosing among these alterations to produce a desired outcome.

Counterfactual learners, on the top rung, can imagine worlds that do not exist and infer reasons for observed phenomena.

Good predictions need not have good explanations.

Nevertheless, deep learning has succeeded primarily by showing that certain questions or tasks we thought were difficult are in fact not. It has not addressed the truly difficult questions that continue to prevent us from achieving humanlike AI.

Deep learning has instead given us machines with truly impressive abilities but no intelligence. The difference is profound and lies in the absence of a model of reality.

This lack of flexibility and adaptability is inevitable in any system that works at the first level of the Ladder of Causation.

This already calls for a new kind of knowledge, absent from the data, which

Intervention ranks higher than association because it involves not just seeing but changing what is.

A sufficiently strong and accurate causal model can allow us to use rung-one (observational) data to answer rung-two (interventional) queries. Without the causal model, we could not go from rung one to rung two.

They cannot tell us what will happen in a counterfactual or imaginary world where some observed facts are bluntly negated.

creature of pure imagination.

flexibility, the ability to reflect and improve on past actions, and, perhaps even more significant, our willingness to take responsibility for past and current actions.

think that Turing was on to something. We probably will not succeed in creating humanlike intelligence until we can create childlike intelligence, and a key component of this intelligence is the mastery of causation.

we want our computer to understand causation, we have to teach it how to break the rules.

(Research has shown that three-year-olds already

understand the entire Ladder of Causation.)

It is because of this robustness, I conjecture, that human intuition is organized around causal, not statistical, relations.

The main point is this: while probabilities encode our beliefs about a static world, causality tells us whether and how probabilities change when the world changes, be it by intervention or by act of imagination.