Dreams of a Final Theory: The Scientist's Search for the Ultimate Laws of Nature

by Steven Weinberg

This point tends to get obscured because, in the actual work of thermodynamics or fluid dynamics or population biology, scientists use languages that are special to their own fields, speaking of entropy or eddies or reproductive strategies and not the language of elementary particles. This is not only because we are unable to use our first principles actually to calculate complicated phenomena; it is also a reflection of the sort of question we want to ask about these phenomena. Even if we had an enormous computer that could follow the history of every elementary particle in a tidal flow or a fruit fly, this mountain of computer printout would not be of much use to someone who wanted to know whether the water was turbulent or the fly was alive.

There is no reason to suppose that the convergence of scientific explanations must lead to a convergence of scientific methods.

As Linus Pauling puts it, “There is no part of chemistry that does not depend, in its fundamental theory, upon quantum principles.”

Of all the areas of experience that we try to link to the principles of physics by arrows of explanation, it is consciousness that presents us with the greatest difficulty.

It is clear that there is what a literary critic might call an objective correlative to consciousness; there are physical and chemical changes in my brain and body that I observe to be correlated (either as cause or effect) with changes in my conscious thoughts. I tend to smile when pleased; my brain shows different electrical activity when I am awake or asleep; powerful emotions are triggered by hormones in my blood; and I sometimes speak my thoughts. These are not consciousness itself; I can never express in terms of smiles or brain waves or hormones or words what it feels like to be happy or sad. But setting consciousness to one side for a moment, it seems reasonable to suppose that these objective correlatives to consciousness can be studied by the methods of science and will eventually be explained in terms of the physics and chemistry of the brain and body. (By “explained” I do not necessarily mean that we will be able to predict everything or even very much, but that we will understand why smiles and brain waves and hormones work the way they do, in the same sense that, although we cannot predict next month’s weather, still we understand why the weather works the way it does.)

Suppose then that we will come to understand the objective correlatives to consciousness in terms of physics (including chemistry) and that we will also understand how they evolved to be what they are. It is not unreasonable to hope that when the objective correlatives to consciousness have been explained, somewhere in our explanations we shall be able to recognize something, some physical system for processing information, that corresponds to our experience of consciousness itself, to what Gilbert Ryle has called “the ghost in the machine.” That may not be an explanation of consciousness, but it will be pretty close.

As a physicist I perceive scientific explanations and laws as things that are what they are and cannot be made up as I go along, so my relation to these laws is not so different from my relation to my chair, and I therefore accord the laws of nature (to which our present laws are an approximation) the honor of being real.

In the same way, our discovery of the connected and convergent pattern of scientific explanations has done the very great service of teaching us that there is no room in nature for astrology or telekinesis or creationism or other superstitions.

CHAPTER III

TWO CHEERS FOR REDUCTIONISM

For me, reductionism is not a guideline for research programs, but an attitude toward nature itself. It is nothing more or less than the perception that scientific principles are the way they are because of deeper scientific principles (and, in some cases, historical accidents) and that all these principles can be traced to one simple connected set of laws.

The reason we give the impression that we think that elementary particle physics is more fundamental than other branches of physics is because it is. I do not know how to defend the amounts being spent on particle physics without being frank about this. But by elementary particle physics being more fundamental I do not mean that it is more mathematically profound or that it is more needed for progress in other fields or anything else but only that it is closer to the point of convergence of all our arrows of explanation.

It is not that the discovery of DNA was fundamental to all of the science of life, but rather that DNA itself is fundamental to all life itself.

Living things are the way they are because through natural selection they have evolved to be that way, and evolution is possible because the properties of DNA and related molecules allow organisms to pass on their genetic blueprint to their offspring. In precisely the same sense, whether or not the discoveries of elementary particle physics are useful to all other scientists, the principles of elementary particle physics are fundamental to all nature.

Indeed, elementary particles are not in themselves very interesting, not at any rate in the way that people are interesting. Aside from their momentum and spin, every electron in the universe is just like every other electron—if you have seen one electron, you have seen them all. But this very simplicity suggests that electrons, unlike people, are not made up of numbers of more fundamental constituents, but are themselves something close to the fundamental constituents of everything else. It is because elementary particles are so boring that they are interesting; their simplicity suggests that the study of elementary particles will bring us closer to a comprehensive understanding of nature.

Condensed matter physicists will doubtless eventually solve the problem of high-temperature superconductivity without any direct help from elementary particle physicists, and, when elementary particle physicists understand the origin of mass, it will very likely be without direct inputs from condensed matter physics. The difference between these two problems is that, when condensed matter physicists finally explain high-temperature superconductivity—whatever brilliant new ideas have to be invented along the way—in the end the explanation will take the form of a mathematical demonstration that deduces the existence of this phenomenon from known properties of electrons, photons, and atomic nuclei; in contrast, when particle physicists finally understand the origin of mass in the standard model the explanation will be based on aspects of the standard model about which we are today quite uncertain, and which we cannot learn (though we may guess) without new data from facilities like the Super Collider. Elementary particle physics thus represents a frontier of our knowledge in a way that condensed matter physics does not.

Some of the issues in the debate over reductionism within physics have been usefully raised by the author James Gleick. (It was Gleick who introduced the physics of chaos to a general readership.) In a recent talk he argued: Chaos is anti-reductionist. This new science makes a strong claim about the world: namely, that when it comes to the most interesting questions, questions about order and disorder, decay and creativity, pattern formation and life itself, the whole cannot be explained in terms of the parts. There are fundamental laws about complex systems, but they are new kinds of laws. They are laws of structure and organization and scale, and they simply vanish when you focus on the individual constituents of a complex system—just as the psychology of a lynch mob vanishes when you interview individual participants.

I would reply first that different questions are interesting in different ways. Certainly questions about creativity and life are interesting because we are alive and would like to be creative. But there are other questions that are interesting because they carry us closer to the point of convergence of our explanations. Discovering the source of the Nile did nothing to illuminate the problems of Egyptian agriculture, but who can say that it was not interesting? Also, it misses the point of this sort of question to speak of explaining the whole “in terms of the parts”; the study of quarks and electrons is fundamental not because all ordinary matter is composed of quarks and electrons but because we think that by studying quarks and electrons we will learn something about the principles that govern everything. (It was an experiment using electrons fired at the quarks inside atomic nuclei that clinched the case for the modern unified theory of two of the four fundamental forces of nature, the weak and electromagnetic forces.) In fact the elementary particle physicist today gives more attention to exotic particles that are not present in ordinary matter than to the quarks and electrons that are, because we think that right now the questions that need to be answered will be better illuminated by studying these exotic particles.

The reductionist attitude provides a useful filter that saves scientists in all fields from wasting their time on ideas that are not worth pursuing. In this sense, we are all reductionists now.

CHAPTER IV

QUANTUM MECHANICS AND ITS DISCONTENTS

At the heart of Schrödinger’s approach was a dynamical equation (known ever since as the Schrödinger equation) that dictated the way that any given particle wave would change with time. Some of the solutions of the Schrödinger equation for electrons in atoms simply oscillate at a single pure frequency, like the sound wave produced by a perfect tuning fork. Such special solutions correspond to the possible stable quantum states of the atom or molecule (something like the stable waves of vibration within a tuning fork), with the energy of the atomic state given by the frequency of the wave times Planck’s constant. These are the energies that are revealed to us through the colors of the light that the atom can emit or absorb. The Schrödinger equation is mathematically the same sort of equation (known as a partial differential equation) that had been used since the nineteenth century to study waves of sound or light. Physicists in the 1920s were already so comfortable with this sort of wave equation that they were able immediately to set about calculating the energies and other properties of all sorts of atoms and molecules. It was a golden time for physics. Other successes followed rapidly, and one by one the mysteries that had surrounded atoms and molecules seemed to melt away.

Any sort of wave is described at any one moment by a list of numbers, one number for each point of the space through which the wave passes. For instance, in a sound wave the numbers give the air pressure at each point of the air. In a light wave, the numbers give the strengths and directions of the electric and magnetic fields at each point of the space through which the light travels. The electron wave could also be described at any moment as a list of numbers, one number for each point of the space in and around the atom. It is this list that is known as the wave function, and the individual numbers are called the values of the wave function. But at first all one could say about the wave function is that it was a solution of the Schrödinger equation; no one yet knew what physical quantity was being described by these numbers.

The observation of phenomena like diffraction (the failure of light rays to follow straight lines when passing very close to objects or through very small holes) had suggested to Thomas Young and Augustin Fresnel that light was some sort of wave and that it did not travel in straight lines when forced to squeeze through small holes because the holes were smaller than its wavelength. But no one in the early nineteenth century knew what light was a wave of; only with the work of James Clerk Maxwell in the 1860s did it become clear that light was a wave of varying electric and magnetic fields. But what is it that is varying in an electron wave?

It is natural to describe an electron traveling through empty space as a wave packet, a little bundle of electron waves that travel along together, like the pulse of light waves produced by a searchlight that is turned on only for an instant. The Schrödinger equation shows that, when such a wave packet strikes an atom, it breaks up; wavelets go traveling off in all directions like sprays of water when the stream from a garden hose hits a rock. This was puzzling; electrons striking atoms fly off in one direction or another but they do not break up—they remain electrons. In 1926 Max Born in Göttingen proposed to interpret this peculiar behavior of the wave function in terms of probabilities. The electron does not break up, but it can be scattered in any direction, and the probability that an electron is scattered in some particular direction is greatest in those directions where the values of the wave function are largest. In other words, electron waves are not waves of anything; their significance is simply that the value of the wave function at any point tells us the probability that the electron is at or near that point.

Physicists continued to wrangle over the interpretation of quantum mechanics for years after they had become used to solving the Schrödinger equation. Einstein was unusual in rejecting quantum mechanics in his work; most physicists were simply trying to understand it. Much of this debate went on at the University Institute for Theoretical Physics in Copenhagen, under the guidance of Niels Bohr.* Bohr focused particularly on a peculiar feature of quantum mechanics that he called complementarity: knowledge of one aspect of a system precludes knowledge of certain other aspects of the system. Heisenberg’s uncertainty principle provides one example of complementarity: knowledge of a particle’s position (or momentum) precludes knowledge of the particle’s momentum (or position).†

A DIALOGUE ON THE MEANING OF QUANTUM MECHANICS

The orthodox Copenhagen interpretation that I have been describing up to now is based on a sharp separation between the physical system, governed by the rules of quantum mechanics, and the apparatus used to study it, which is described classically, that is, according to the prequantum rules of physics. Our mythical particle may have a wave function with both here and there values, but, when it is observed, it somehow becomes definitely here or there, in a manner that is essentially unpredictable, except with regard to probabilities. But this difference of treatment between the system being observed and the measuring apparatus is surely a fiction. We believe that quantum mechanics governs everything in the universe, not only individual electrons and atoms and molecules but also the experimental apparatus and the physicists who use it. If the wave function describes the measuring apparatus as well as the system being observed and evolves deterministically according to the rules of quantum mechanics even during a measurement, then, as Tiny Tim asks, where do the probabilities come from?

Indeed, quantum mechanics may survive not merely as an approximation to a deeper truth, in the way that Newton’s theory of gravitation survives as an approximation to Einstein’s general theory of relativity, but as a precisely valid feature of the final theory.

CHAPTER V

TALES OF THEORY AND EXPERIMENT

My first tale has to do with the general theory of relativity, Einstein’s theory of gravitation.

It is widely supposed that the true test of a theory is in the comparison of its predictions with the results of experiment. Yet, with the benefit of hindsight, one can say today that Einstein’s successful explanation in 1915 of the previously measured anomaly in Mercury’s orbit was a far more solid test of general relativity than the verification of his calculation of the deflection of light by the sun in observations of the eclipse of 1919 or in later eclipses. That is, in the case of general relativity a retrodiction, the calculation of the already-known anomalous motion of Mercury, in fact provided a more reliable test of the theory than a true prediction of a new effect, the deflection of light by gravitational fields.

It was not until after World War II that new techniques of radar and radio astronomy led to a significant improvement in the accuracy of these experimental tests of general relativity. We can now say that the predictions of general relativity for the deflection (and also delay) of light passing the sun and for the orbital motion not only of Mercury but also of the asteroid Icarus and other natural and man-made bodies have been confirmed with experimental uncertainties less than 1%. But this was a long time in coming.

I believe that the general acceptance of general relativity was due in large part to the attractions of the theory itself—in short, to its beauty.

I have no way of knowing to what extent the successful calculation of the precession of Mercury’s orbit contributed to Einstein’s confidence in general relativity in 1916, but well before then, before he did this calculation, something must have given him enough confidence in the ideas that underlie general relativity to keep him working on it, and this could only have been the attractiveness of the ideas themselves.

To understand this, you must look more closely at both Newton’s and Einstein’s theories. Newtonian physics did explain virtually all the observed motions of the solar system, but at the cost of introducing a set of somewhat arbitrary assumptions. For example, consider the law that says that the gravitational force produced by any body decreases like the inverse square of the distance from the body. In Newton’s theory there is nothing about an inverse-square law that is particularly compelling. Newton developed the idea of an inverse-square law in order to explain known facts about the solar system, like Kepler’s relation between the size of planetary orbits and the time it takes planets to go around the sun. Apart from these observational facts, in Newton’s theory one could have replaced the inverse-square law with an inverse-cube law or an inverse 2.01-power law without the slightest change in the conceptual framework of the theory. It would be changing a minor detail in the theory. Einstein’s theory was far less arbitrary, far more rigid. For slowly moving bodies in weak gravitational fields, for which one can legitimately speak of an ordinary gravitational force, general relativity requires that the force must fall off according to an inverse-square law. It is not possible in general relativity to adjust the theory to get anything but an inverse-square law without doing violence to the underlying assumptions of the theory.

My next tale has to do with quantum electrodynamics—the quantum-mechanical theory of electrons and light.

Oppenheimer found in his calculation that, because the sum includes contributions from photons of unlimitedly high energy, it turns out to be infinite, leading to an infinite shift in the energy of the atom.* High energy corresponds to short wavelengths; because ultraviolet light has a wavelength shorter than that of visible light, this infinity became known as the ultraviolet catastrophe.

The problem of infinities could have been solved by brute force, by simply decreeing that electrons can emit and absorb photons only with energies below some limiting value. All the successes scored in the 1930s by quantum electrodynamics in explaining the interactions of electrons and photons involved low-energy photons, so these successes could be preserved by supposing that this limiting value of photon energies is sufficiently high, for example, 10 million volts. With this sort of limit on virtual photon energies, quantum electrodynamics would predict a very small energy shift of atoms. No one at this time had measured the energies of atoms with enough precision to tell whether or not this tiny energy shift was actually present, so there was no question of any disagreement with experiment. (In fact, quantum electrodynamics was regarded with such pessimism that no one even tried to calculate just what the energy shift would be.) The trouble with this solution to the problem of infinities was not that it was in conflict with experiment but that it was too arbitrary and ugly to contemplate.

In the physics literature of the 1930s and 1940s one can find a host of other possible unpalatable solutions of the problem of infinities, including even theories in which the infinity caused by emitting and reabsorbing high-energy photons is canceled by other processes of negative probability. The concept of negative probability is of course meaningless; its introduction into physics is a measure of the desperation that was felt over the problem of infinities.

Among the physicists at Shelter Island was Willis Lamb, a young physicist at Columbia University. Using some of the microwave radar technology developed during the war, Lamb had just succeeded in measuring precisely the sort of effect that Oppenheimer had tried to calculate in 1930, a shift in the energy of the hydrogen atom owing to photo emissions and reabsorptions. This shift has since become known as the Lamb shift. This measurement in itself did nothing to solve the problem of infinities but forced physicists to come to grips with this problem again in order to account for the measured value of the Lamb shift.

They reasoned that the shift in the energy of an atom owing to emission and reabsorption of photons is not really an observable; the only observable is the total energy of the atom, which is calculated by adding this energy shift to the energy calculated in 1928 by Dirac. This total energy depends on the bare mass and bare charge of the electron, the mass and charge that appear in the equations of the theory before we start worrying about photon emissions and reabsorptions. But free electrons as well as electrons in atoms are always emitting and reabsorbing photons that affect the electron’s mass and electric charge, and so the bare mass and charge are not the same as the measured electron mass and charge that are listed in tables of elementary particles. In fact, in order to account for the observed values (which of course are finite) of the mass and charge of the electron, the bare mass and charge must themselves be infinite. The total energy of the atom is thus the sum of two terms, both infinite: the bare energy that is infinite because it depends on the infinite bare mass and charge, and the energy shift calculated by Oppenheimer that is infinite because it receives contributions from virtual photons of unlimited energy. This raised the question: Is it possible that these two infinities cancel and leave a finite total energy?

It was not long before physicists did more accurate calculations of the Lamb shift that included positrons and other relativistic effects. The importance of these calculations was not so much that they obtained a more accurate result but that the problem of infinities had been tamed; the infinities turned out to cancel without having arbitrarily to throw away the contributions of high-energy virtual photons.

Quantum electrodynamics had almost been killed off by the problem of infinities but had been saved by the idea of canceling the infinities in a redefinition or renormalization of the electron mass and charge. But in order for the problem of infinities to be solved in this way, it is necessary that the infinities occur in calculations in only certain very limited ways, which is the case only for a limited class of specially simple quantum field theories. Such theories are called renormalizable. The simplest version of quantum electrodynamics is renormalizable in this sense, but any sort of small change in this theory would spoil this property and lead to a theory with infinities that could not be canceled by a redefinition of the constants of the theory. Thus this theory was not only mathematically satisfactory and in agreement with experiment, but it seemed to contain within itself an explanation of why it was the way it was; any small change in the theory would lead not only to a disagreement with experiment but to results that were totally absurd—infinite answers to perfectly sensible questions.

After such successes, it is not surprising that quantum electrodynamics in its simple renormalizable version has become generally accepted as the correct theory of photons and electrons. Nevertheless, despite the experimental success of the theory, and even though the infinities in this theory all cancel when one handles them correctly, the fact that the infinities occur at all continues to produce grumbling about quantum electrodynamics and similar theories.

the demand for a completely finite theory is similar to a host of other aesthetic judgments that theoretical physicists always need to make.

My third tale has to do with the development and final acceptance of the modern theory of the weak nuclear force.

One may ask why the acceptance of the validity of the electroweak theory was so rapid and widespread. Well, of course, the neutral currents had been predicted, and then they were found. Isn’t that the way that any theory becomes established?

The aesthetic criterion of naturalness was being used to help physicists weigh conflicting experimental data.