The Fabric of Reality: Towards a Theory of Everything (Penguin Science)

by David Deutsch

This is possible because understanding does not depend on knowing a lot of facts as such, but on having the right concepts, explanations and theories. One comparatively simple and comprehensible theory can cover an infinity of indigestible facts.

Being able to predict things or to describe them, however accurately, is not at all the same thing as understanding them.

Facts cannot be understood just by being summarized in a formula, any more than by being listed on paper or committed to memory. They can be understood only by being explained. Fortunately, our best theories embody deep explanations as well as accurate predictions.

For example, the general theory of relativity explains gravity in terms of a new, four-dimensional geometry of curved space and time. It explains precisely how this geometry affects and is affected by matter. That explanation is the entire content of the theory; predictions about planetary motions are merely some of the consequences that we can deduce from the explanation.

What makes the general theory of relativity so important is not that it can predict planetary motions a shade more accurately than Newton’s theory can, but that it reveals and explains previously unsuspected aspects of reality, such as the curvature of space and time. This is typical of scientific explanation. Scientific theories explain the objects and phen...

Yet some philosophers – and even some scientists – disparage the role of explanation in science. To them, the basic purpose of a scientific theory is not to explain anything, but to predict the outcomes of experiments: its entire content lies in its predictive formulae. They consider that any consistent explanation that a theory may give for its predictions is as good as any other – or as good as no explanation at all – so long as the predictions are true. This view is called instrumentalism (because it says that a theory is no more than an ‘instrument’ for making predictions). To instrumentalists, the idea that science can enable us to understand the underlying reality that accounts for our observations is a fallacy and a conceit.

An extreme form of instrumentalism, called positivism (or logical positivism), holds that all statements other than those describing or predicting observations are not only superfluous but meaningless. Although this doctrine is itself meaningless, according to its own criterion, it was nevertheless the prevailing theory of scientific knowledge during the first half of the twentieth century!

The scientific method involves postulating a new theory to explain some class of phenomena and then performing a crucial experimental test, an experiment for which the old theory predicts one observable outcome and the new theory another. One then rejects the theory whose predictions turn out to be false. Thus the outcome of a crucial experimental test to decide between two theories does depend on the theories’ predictions, and not directly on their explanations.

But experimental testing is by no means the only process involved in the growth of scientific knowledge. The overwhelming majority of theories are rejected because they contain bad explanations, not because they fail experimental tests. We reject them without ever bothering to test them.

To say that prediction is the purpose of a scientific theory is to confuse means with ends. It is like saying that the purpose of a spaceship is to burn fuel. In fact, burning fuel is only one of many things a spaceship has to do to accomplish its real purpose, which is to transport its payload from one point in space to another. Passing experimental tests is only one of many things a theory has to do to achieve the real purpose of science, which is to explain the world.

So what distinguishes understanding from mere knowing? What is an explanation, as opposed to a mere statement of fact such as a correct description or prediction? In practice, we usually recognize the difference easily enough. We know when we do not understand something, even if we can accurately describe and predict it (for instance, the course of a known disease of unknown origin), and we know when an explanation helps us to understand it better. But it is hard to give a precise definition of ‘explanation’ or ‘understanding’. Roughly speaking, they are about ‘why’ rather than ‘what’; about the inner workings of things; about how things really are, not just how they appear to be; about what must be so, rather than what merely happens to be so; about laws of nature rather than rules of thumb. They are also about coherence, elegance and simplicity, as opposed to arbitrariness and complexity, though none of those things is easy to define either.

understanding is one of the higher functions of the human mind and brain, and a unique one. Many other physical systems, such as animals’ brains, computers and other machines, can assimilate facts and act upon them. But at present we know of nothing that is capable of understanding an explanation – or of wanting one in the first place – other than a human mind. Every discovery of a new explanation, and every act of grasping an existing explanation, depends on the uniquely human faculty of creative thought.

So, even though our stock of known theories is indeed snowballing, just as our stock of recorded facts is, that still does not necessarily make the whole structure harder to understand than it used to be. For while our specific theories are becoming more numerous and more detailed, they are continually being ‘demoted’ as the understanding they contain is taken over by deep, general theories. And those theories are becoming fewer, deeper and more general. By ‘more general’ I mean that each of them says more, about a wider range of situations, than several distinct theories did previously. By ‘deeper’ I mean that each of them explains more – embodies more understanding – than its predecessors did, combined.

When admiring centuries-old structures, people often forget that we see only the surviving ones. The overwhelming majority of structures built in medieval and earlier times have collapsed long ago, often soon after they were built. That was especially so for innovative structures. It was taken for granted that innovation risked catastrophe, and builders seldom deviated much from designs and techniques that had been validated by long tradition.

Progress to our current state of knowledge was not achieved by accumulating more theories of the same kind as the master builder knew. Our knowledge, both explicit and inexplicit, is not only much greater than his but structurally different too. As I have said, the modern theories are fewer, more general and deeper. For each situation that the master builder faced while building something in his repertoire – say, when deciding how thick to make a load-bearing wall – he had a fairly specific intuition or rule of thumb, which, however, could give hopelessly wrong answers if applied to novel situations. Today one deduces such things from a theory that is general enough for it to be applied to walls made of any material, in all situations: on the Moon, underwater, or wherever. The reason why it is so general is that it is based on quite deep explanations of how materials and structures work.

A typical theory in a modern student’s syllabus may be harder to understand than any of the master builder’s rules of thumb; but the modern theories are far fewer, and their explanatory power gives them other properties such as beauty, inner logic and connections with other subjects which make them easier to learn.

I am not, of course, denying that specialization is occurring in many subjects in which knowledge is growing, including architecture. This is not a one-way process, for specializations often disappear too: wheels are no longer designed or made by wheelwrights, nor ploughs by ploughwrights, nor are letters written by scribes.

the deepening, unifying tendency I have been describing is not the only one at work: a continual broadening is going on at the same time. That is, new ideas often do more than just supersede, simplify or unify existing ones. They also extend human understanding into areas that were previously not understood at all – or whose very existence was not guessed at. They may open up new opportunities, new problems, new specializations and even new subjects. And when that happens it may give us, at least temporarily, more to learn in order to understand it all.

Thus the issue of whether it is becoming harder or easier to understand everything that is understood depends on the overall balance between these two opposing effects of the growth of knowledge: the increasing breadth of our theories, and their increasing depth. Breadth makes it harder; depth makes it easier. One thesis of this book is that, slowly but surely, depth is winning.

The reductionist conception leads naturally to a classification of subjects and theories in a hierarchy, according to how close they are to the ‘lowest-level’ predictive theories that are known. In this hierarchy, logic and mathematics form the immovable bedrock on which the edifice of science is built. The foundation stone would be a reductive ‘theory of everything’, a universal theory of particles, forces, space and time, together with some theory of what the initial state of the universe was. The rest of physics forms the first few storeys. Astrophysics and chemistry are at a higher level, geology even higher, and so on. The edifice branches into many towers of increasingly high-level subjects like biochemistry, biology and genetics. Perched at the tottering, stratospheric tops are subjects like the theory of evolution, economics, psychology and computer science, which in this picture are almost inconceivably derivative.

The reason why higher-level subjects can be studied at all is that under special circumstances the stupendously complex behaviour of vast numbers of particles resolves itself into a measure of simplicity and comprehensibility. This is called emergence: high-level simplicity ‘emerges’ from low-level complexity.

The laws of motion for any physical system make only conditional predictions, and are therefore compatible with many possible histories of that system.

most of our knowledge of supplementary data – of what specifically happens – is in the form of high-level theories about emergent phenomena, and is therefore by definition not practically expressible in the form of statements about the initial state. For example, in most solutions of the equations of motion the initial state of the universe does not have the right properties for life to evolve from it. Therefore our knowledge that life has evolved is a significant piece of the supplementary data.

Thus the character of many of our descriptions, predictions and explanations of reality bear no resemblance to the ‘initial state plus laws of motion’ picture that reductionism leads to. There is no reason to regard high-level theories as in any way ‘second-class citizens’.

What makes a theory more fundamental, and less derivative, is not its closeness to the supposed predictive base of physics, but its closeness to our deepest explanatory theories.

Up to now, all our understanding has been about some aspect of reality, untypical of the whole. In the future it will be about a unified conception of reality: all explanations will be understood against the backdrop of universality, and every new idea will automatically tend to illuminate not just a particular subject, but, to varying degrees, all subjects.

Scientific knowledge, like all human knowledge, consists primarily of explanations. Mere facts can be looked up, and predictions are important only for conducting crucial experimental tests to discriminate between competing scientific theories that have already passed the test of being good explanations.

As new theories supersede old ones, our knowledge is becoming both broader (as new subjects are created) and deeper (as our fundamental theories explain more, and become more general). Depth is winning. Thus we are not heading away from a state in which one person could understand everything that was understood, but towards it.

Our deepest theories are becoming so integrated with one another that they can be understood only jointly, as a single theory of a unified fabric of reality. This Theory of Everything has a far wider scope than the ‘theory of everything’ that elementary particle physicists are seeking, because the fabric of reality does not consist only of reductionist ingredients such...

particles are grouped into parallel universes. They are ‘parallel’ in the sense that within each universe particles interact with each other just as they do in the tangible universe, but each universe affects the others only weakly, through interference phenomena.

Thus we have reached the conclusion of the chain of reasoning that begins with strangely shaped shadows and ends with parallel universes. Each step takes the form of noting that the behaviour of objects that we observe can be explained only if there are unobserved objects present, and if those unobserved objects have certain properties. The heart of the argument is that single-particle interference phenomena unequivocally rule out the possibility that the tangible universe around us is all that exists. There is no disputing the fact that such interference phenomena occur.

every subatomic particle has counterparts in other universes, and is interfered with only by those counterparts. It is not directly affected by any other particles in those universes. Therefore interference is observed only in special situations where the paths of a particle and its shadow counterparts separate and then reconverge (as when a photon and shadow photon are heading towards the same point on the screen).

Second, the detection of interference between any two universes requires an interaction to take place between all the particles whose positions and other attributes are not identical in the two universes. In practice this means that interference is strong enough to be detected only between universes that are very alike.

if one does start from theory, there are two things that everyone agrees on. The first is that quantum theory is unrivalled in its ability to predict the outcomes of experiments, even if one blindly uses its equations without worrying much about what they mean. The second is that quantum theory tells us something new and bizarre about the nature of reality. The dispute is only about what exactly this is.

Without knowing anything of quantum theory, one can see that those shadows could not be the result of any single history of the photon as it travels from the torch to the observer’s eye. They are incompatible with any explanation in terms of only the photons that we see. Or in terms of only the barrier that we see. Or in terms of only the universe that we see. Therefore, if the best theory available to physics did not refer to parallel universes, it would merely mean that we needed a better theory, one that did refer to parallel universes, in order to explain what we see.

If ‘shadow’ observers, be they frogs or people, are real, then their sensations must be real too. When they observe what we might call a shadow object, they observe that it is tangible. They observe this by the same means, and according to the same definition, as we apply when we say that the universe we observe is ‘tangible’. Tangibility is relative to a given observer.

parallel universes They are ‘parallel’ in the sense that within each universe particles interact with each other just as they do in the tangible universe, but each universe affects the others only weakly, through interference phenomena.

quantization The property of having a discrete (rather than continuous) set of possible values. Quantum theory gets its name from its assertion that all measurable quantities are quantized. However, the most significant quantum effect is not quantization but interference.

interference The effect of a particle in one universe on its counterpart in another. Photon interference can cause shadows to be much more complicated than mere silhouettes of the obstacles causing them.

There is no getting away from the fact that we human beings are small creatures with only a few inaccurate, incomplete channels through which we receive all information from outside ourselves. We interpret this information as evidence of a large and complex external universe (or multiverse). But when we are weighing up this evidence, we are literally contemplating nothing more than patterns of weak electric current trickling through our own brains.

In fact, it is impossible to extrapolate observations unless one has already placed them within an explanatory framework.

Problem-solving does begin with an inadequate theory – but not with the notional ‘theory’ consisting of past observations. It begins with our best existing theories. When some of those theories seem inadequate to us, and we want new ones, that is what constitutes a problem. Thus, contrary to the inductivist scheme shown in Figure 3.1, scientific discovery need not begin with observational evidence. But it does always begin with a problem.

By a ‘problem’ I do not necessarily mean a practical emergency, or a source of anxiety. I just mean a set of ideas that seems inadequate and worth trying to improve. The existing explanation may seem too glib, or too laboured; it may seem unnecessarily narrow, or unrealistically ambitious. One may glimpse a possible unification with other ideas. Or a satisfactory explanation in one field may appear to be irreconcilable with an equally satisfactory explanation in another. Or it may be that there have been some surprising observations – such as the wandering of planets – which existing theories did not predict and cannot explain.

One solves a problem by finding new or amended theories, containing explanations which do not have the deficiencies, but do retain the merits, of existing explanations (Figure 3.2). Thus, after a problem presents itself (stage 1), the next stage always involves conjecture: proposing new theories, or modifying or reinterpreting old ones, in the hope of solving the problem (stage 2). The conjectures are then criticized which, if the criticism is rational, entails examining and comparing them to see which offers the best explanations, according to the criteria inherent in the problem (stage 3). When a conjectured theory fails to survive criticism – that is, when it appears to offer worse explanations than other theories do – it is abandoned. If we find ourselves abandoning one of our originally held theories in favour of one of the newly proposed ones (stage 4), we tentatively deem our problem-solving enterprise to have made progress. I say ‘tentatively’, because subsequent problem-solving will probably involve altering or replacing even these new, apparently satisfactory theories, and sometimes even resurrecting some of the apparently unsatisfactory ones. Thus the solution, however good, is not the end of the story: it is a starting-point for the next problem-solving process (stage 5).

Scientific problem-solving always includes a particular method of rational criticism, namely experimental testing. Where two or more rival theories make conflicting predictions about the outcome of an experiment, the experiment is performed and the theory or theories that made false predictions are abandoned. The very construction of scientific conjectures is focused on finding explanations that have experimentally testable predictions.

But if we take the view that science is about explanations, we see that this rule is really a special case of something that applies naturally to all problem-solving: theories that are capable of giving more detailed explanations are automatically preferred. They are preferred for two reasons. One is that a theory that ‘sticks its neck out’ by being more specific about more phenomena opens up itself and its rivals to more forms of criticism, and therefore has more chance of taking the problem-solving process forward. The second is simply that, if such a theory survives the criticism, it leaves less unexplained – which is the object of the exercise.

One difference is that in biology variations (mutations) are random, blind and purposeless, while in human problem-solving the creation of new conjectures is itself a complex, knowledge-laden process driven by the intentions of the people concerned.

Perhaps an even more important difference is that there is no biological equivalent of argument. All conjectures have to be tested experimentally, which is one reason why biological evolution is slower and less efficient by an astronomically large factor.

Inductivism is observation- and prediction-based, whereas in reality science is problem- and explanation-based.

Inductivism supposes that theories are somehow extracted or distilled from observations, or are justified by them, whereas in fact theories begin as unjustified conjectures in someone’s mind, which typically precede the observations that rule out rival theories.