Dreams of a Final Theory: The Scientist's Search for the Ultimate Laws of Nature

by Steven Weinberg

There now is general agreement among experimentalists as well as among theorists that the predicted left-right asymmetry effect really is there with about the expected magnitude in atoms as well as in the high-energy electron scattering studied in the Stanford accelerator experiment.

It appears that anything you say about the way that theory and experiment may interact is likely to be correct, and anything you say about the way that theory and experiment must interact is likely to be wrong.

There are also experiments that present us with complete surprises that no theorist had anticipated. In this category are the experiments that discovered X rays or the so-called strange particles or, for that matter, the anomalous precession of the orbit of the planet Mercury. These I think are the experiments that bring the most joy to the hearts of experimentalists and journalists.

There are also experiments that present us with almost complete surprises—that is, that find effects that had been discussed as a possibility, but only as a logical possibility that there was no compelling reason to expect. These include the experiments that discovered the violation of the so-called time-reversal symmetry and the experiments that found certain new particles, such as the “bottom” quark and a sort of very heavy electron known as the tau lepton.

There is also an interesting class of experiments that have found effects that had been predicted by theorists but that were nevertheless discovered accidentally because the experimentalists did not know of the prediction, either because the theorists did not have enough faith in their theory to advertise it to experimentalists or because the channels of scientific communication were just too noisy. Among these experiments are the d...

Then there are experiments that are done even though one knows the answer, even though the theoretical prediction is so firm that the theory is beyond serious doubt, because the phenomena themselves are so entrancing and offer so many possibilities of further experiments that one simply has to go ahead and find these things. I would include in this category the discovery of the antiproton and the neutrino and the more recent discovery of the W and Z particles. ...

Finally one can imagine a category of experiments that refute well-accepted theories, theories that have become part of the standard consensus of physics. Under this category I can find no examples whatever in the past one hundred years. There are of course many cases where theories have been found to have a narrower realm of application than had been thought. Newton’s theory of motion does not apply at high speeds. Parity, the symmetry between right and left, does not work in the weak forces. And so on. But in this century no theory that has been generally accepted as valid by the world of physics has t...

I think that one should not hope for a science of science, the formulation of any definite rules about how scientists do or ought to behave, but only aim at a description of the sort of behavior that historically has led to scientific progress—an art of science.

CHAPTER VI

BEAUTIFUL THEORIES

Some of the talk about the importance of beauty in science has been little more than gushing. I do not propose to use this chapter just to say more nice things about beauty. Rather, I want to focus more closely on the nature of beauty in physical theories, on why our sense of beauty is sometimes a useful guide and sometimes not, and on how the usefulness of our sense of beauty is a sign of our progress toward a final theory.

An elegant proof or calculation is one that achieves a powerful result with a minimum of irrelevant complication. It is not important for the beauty of a theory that its equations should have elegant solutions.

Simplicity is part of what I mean by beauty, but it is a simplicity of ideas, not simplicity of a mechanical sort that can be measured by counting equations or symbols.

As Einstein said of general relativity, “The chief attraction of the theory lies in its logical completeness. If a single one of the conclusions drawn from it proves wrong, it must be given up; to modify it without destroying the whole structure seems to be impossible.”

Any symmetry principle is at the same time a principle of simplicity. If the laws of nature did distinguish among directions like up or down or north, then we would have to put something into our equations to keep track of the orientation of our laboratories, and they would be correspondingly less simple. Indeed, the very notation that is used by mathematicians and physicists to make our equations look as simple and compact as possible has built into it an assumption that all directions in space are equivalent.

In 1929 Werner Heisenberg and Wolfgang Pauli (building on earlier work of Max Born, Heisenberg, Pascual Jordan, and Eugene Wigner) explained in a pair of papers how massive particles like the electron could also be understood as bundles of energy and momenta in different sorts of fields, such as the electron field. Just as the electromagnetic force between two electrons is due in quantum mechanics to the exchange of photons, the force between photons and electrons is due to the exchange of electrons. The distinction between matter and force largely disappears; any particle can play the role of a test body on which forces act and by its exchange can produce other forces.

This is precisely the sort of logical rigidity that gives a really fundamental theory its beauty: quantum mechanics and special relativity are nearly incompatible, and their reconciliation in quantum field theory imposes powerful restrictions on the ways that particles can interact with one another.

The fact that the laws of nature seem to distinguish between stationary and rotating frames of reference bothered Isaac Newton and continued to trouble physicists in the following centuries. In the 1880s the Viennese physicist and philosopher Ernst Mach pointed the way toward a possible reinterpretation. Mach emphasized that there was something else besides centrifugal force that distinguishes the rotating merry-go-round and more conventional laboratories. From the point of view of an astronomer on the merry-go-round, the sun, stars, galaxies—indeed, the bulk of the matter of the universe—seems to be revolving around the zenith. You or I would say that this is because the merry-go-round is rotating, but an astronomer who grew up on the merry-go-round and naturally uses it as his frame of reference would insist that it is the rest of the universe that is spinning around him. Mach asked whether there was any way that this great apparent circulation of matter could be held responsible for centrifugal force. If so, then the laws of nature discovered on the merry-go-round might actually be the same as those found in more conventional laboratories; the apparent difference would simply arise from the different environment seen by observers in their different laboratories.

Mach’s hint was picked up by Einstein and made concrete in his general theory of relativity. In general relativity there is indeed an influence exerted by the distant stars that creates the phenomenon of centrifugal force in a spinning merry-go-round: it is the force of gravity. Of course nothing like this happens in Newton’s theory of gravitation, which deals only with a simple attraction between all masses. General relativity is more complicated; the circulation of the matter of the universe around the zenith seen by observers on the merry-go-round produces a field somewhat like the magnetic field produced by the circulation of electricity in the coils of an electromagnet. It is this “gravitomagnetic” field that in the merry-go-round frame of reference produces the effects that in more conventional frames of reference are attributed to centrifugal force. The equations of general relativity, unlike those of Newtonian mechanics, are precisely the same in the merry-go-round laboratory and conventional laboratories; the difference between what is observed in these laboratories is entirely due to their different environment—a universe that revolves around the zenith or one that does not. But, if gravitation did not exist, this reinterpretation of centrifugal force would be impossible, and the centrifugal force felt on a merry-go-round would allow us to distinguish between the merry-go-round and more conventional laboratories and would thus rule out any possible equivalence be...

Again in analogy with general relativity, the fact that the laws of nature are unaffected even if the mixtures vary from place to place and time to time makes it necessary to include a family of fields in the theory that interact with quarks, analogous to the gravitational field. There are eight of these fields; they are known as gluon fields because the strong forces they produce glue the quarks together inside the proton and neutron. Our modern theory of these forces, quantum chromo dynamics, is nothing but the theory of quarks and gluons that respects this local color symmetry.

There is no logical formula that establishes a sharp dividing line between a beautiful explanatory theory and a mere list of data, but we know the difference when we see it—we demand a simplicity and rigidity in our principles before we are willing to take them seriously. Thus not only is our aesthetic judgment a means to the end of finding scientific explanations and judging their validity—it is part of what we mean by an explanation.

As Abdus Salam has said, it is not particles or forces with which nature is sparing, but principles. The important thing is to have a set of simple and economical principles that explain why the particles are what they are.

The kind of beauty that we find in physical theories is of a very limited sort. It is, as far as I have been able to capture it in words, the beauty of simplicity and inevitability—the beauty of perfect structure, the beauty of everything fitting together, of nothing being changeable, of logical rigidity. It is a beauty that is spare and classic, the sort we find in the Greek tragedies. But this is not the only kind of beauty that we find in the arts. A play of Shakespeare does not have this beauty, at any rate not to the extent that some of his sonnets have. Often the director of a Shakespeare play chooses to leave out whole speeches. In the Olivier film version of Hamlet, Hamlet never says, “Oh what a rogue and peasant slave am I!…” And yet the performance works, because Shakespeare’s plays are not spare, perfect structures like general relativity or Oedipus Rex; they are big messy compositions whose messiness mirrors the complexity of life.

There is another respect in which it seems to me that theoretical physics is a bad model for the arts. Our theories are very esoteric—necessarily so, because we are forced to develop these theories using a language, the language of mathematics, that has not become part of the general equipment of the educated public. Physicists generally do not like the fact that our theories are so esoteric. On the other hand, I have occasionally heard artists talk proudly about their work being accessible only to a band of cognoscenti and justify this attitude by quoting the example of physical theories like general relativity that also can be understood only by initiates. Artists like physicists may not always be able to make themselves understood by the general public, but esotericism for its own sake is just silly.

It is very strange that mathematicians are led by their sense of mathematical beauty to develop formal structures that physicists only later find useful, even where the mathematician had no such goal in mind. A well-known essay by the physicist Eugene Wigner refers to this phenomenon as “The Unreasonable Effectiveness of Mathematics.” Physicists generally find the ability of mathematicians to anticipate the mathematics needed in the theories of physicists quite uncanny. It is as if Neil Armstrong in 1969 when he first set foot on the surface of the moon had found in the lunar dust the footsteps of Jules Verne.

Where then does a physicist get a sense of beauty that helps not only in discovering theories of the real world, but even in judging the validity of physical theories, sometimes in the teeth of contrary experimental evidence? And how does a mathematician’s sense of beauty lead to structures that are valuable decades or centuries later to physicists, even though the mathematician may have no interest in physical applications?

There seem to me to be three plausible explanations, two of them applicable throughout much of science and the third limited to t...

The first explanation is that the universe itself acts on us as a random, inefficient, and yet in the long run effective, teaching machine. Just as through an infinite series of accidental events, atoms of carbon and nitrogen and oxygen and hydrogen joined together to form primitive forms of life that later evolved into protozoa and fishes and people, in the same mann...

I suppose this would be everyone’s explanation of why the horse trainer’s sense of beauty helps when it does help in judging which horse can win races. The racehorse trainer has been at the track for many years—has experienced many horses winning or losing—and has come to associate, without being able to express it explicitly, certain visual cues with the expectation of a winning horse.

The second of the reasons why we expect successful scientific theories to be beautiful is simply that scientists tend to choose problems that are likely to have beautiful solutions. The same may even apply to our friend the racehorse trainer. He trains horses to win races; he has learned to recognize which horses are likely to win and he calls these horses beautiful; but, if you take him aside and promise not to repeat what he says, he may confess to you that the reason he went into the business of training horses to win races in the first place was because the horses that he trains are such beautiful animals.

It is when we study truly fundamental problems that we expect to find beautiful answers. We believe that, if we ask why the world is the way it is and then ask why that answer is the way it is, at the end of this chain of explanations we shall find a few simple principles of compelling beauty. We think this in part because our historical experience teaches us that as we look beneath the surface of things, we find more and more beauty.

Although we do not yet have a sure sense of where in our work we should rely on our sense of beauty, still in elementary particle physics aesthetic judgments seem to be working increasingly well. I take this as evidence that we are moving in the right direction, and perhaps not so far from our goal.

E.g., the frequency with which the wave function of any system in a state of definite energy oscillates is given by the energy divided by a constant of nature known as Planck’s constant. This system appears much the same to two observerswho have set their watches differently by one second, but, if they both observe the system when the hands on their watches both point precisely to twelve noon, they observe that the oscillation is at a different phase; because their watches are set differently they are really observing the system at different times, so that one observer may, e.g., see a crest in the wave, while the other sees a trough. Specifically, the phase is different by the number of cycles (or parts of cycles) that occur in one second; in other words, by the frequency of the oscillation in cycles per second, and hence by the energy divided by Planck’s constant. In today’s quantum mechanics, we define the energy of any system as the change in phase (in cycles or parts of cycles) of the wave function of the system at a given clock time when we shift the way our clocks are set by one second. Planck’s constant gets into the act only because energy is historically measured in units like calories or kilowatt hours or electron volts that were adopted before the advent of quantum mechanics; Planck’s constant simply provides the conversion factor between these older systems of units and the natural quantum-mechanical unit of energy, which is cycles per second. It can be shown ...

CHAPTER VII

AGAINST PHILOSOPHY

Can philosophy give us any guidance toward a final theory?

Physicists do of course carry around with them a working philosophy. For most of us, it is a rough-and-ready realism, a belief in the objective reality of the ingredients of our scientific theories. But this has been learned through the experience of scientific research and rarely from the teachings of philosophers.

It is only fair to admit my limitations and biases in making this judgment. After a few years’ infatuation with philosophy as an undergraduate I became disenchanted. The insights of the philosophers I studied seemed murky and inconsequential compared with the dazzling successes of physics and mathematics. From time to time since then I have tried to read current work on the philosophy of science. Some of it I found to be written in a jargon so impenetrable that I can only think that it aimed at impressing those who confound obscurity with profundity. Some of it was good reading and even witty, like the writings of Wittgenstein and Paul Feyerabend. But only rarely did it seem to me to have anything to do with the work of science as I knew it. According to Feyerabend, the notion of scientific explanation developed by some philosophers of science is so narrow that it is impossible to speak of one theory being explained by another, a view that would leave my generation of particle physicists with nothing to do.

Science is of course a social phenomenon, with its own reward system, its revealing snobberies, its interesting patterns of alliance and authority. For instance, Sharon Traweek has spent years with elementary particle experimentalists at both the Stanford Linear Accelerator Center and the KEK Laboratory in Japan and has described what she had seen from the perspective of an anthropologist. This kind of big science is a natural topic for anthropologists and sociologists, because scientists belong to an anarchic tradition that prizes individual initiative, and yet they find in today’s experiments that they have to work together in teams of hundreds. As a theorist I have not worked in such a team, but many of her observations seem to me to have the ring of truth, as for instance: The physicists see themselves as an elite whose membership is determined solely by scientific merit. The assumption is that everyone has a fair start. This is underscored by the rigorously informal dress code, the similarity of their offices, and the “first naming” practices in the community. Competitive individualism is considered both just and effective: the hierarchy is seen as a meritocracy which produces fine physics. American physicists, however, emphasize that science is not democratic: decisions about scientific purposes should not be made by majority rule within the community, nor should there be equal access to a lab’s resources. On both these issues, most Japanese physicists assume the opp...

It seems to have been an easy step from these useful historical and sociological observations to the radical position that the content of the scientific theories that become accepted is what it is because of the social and historical setting in which the theories are negotiated. (The elaboration of this position is sometimes known as the strong program in the sociology of science.) This attack on the objectivity of scientific knowledge is made explicit and even brought into the title of a book by Andrew Pickering: Constructing Quarks. In his final chapter, he comes to the conclusion: “And, given their extensive training in sophisticated mathematical techniques, the preponderance of mathematics in particle physicists’ accounts of reality is no more hard to explain than the fondness of ethnic groups for their native language. On the view advocated in this chapter, there is no obligation upon anyone framing a view of the world to take account of what twentieth-century science has to say.” Pickering describes in detail a great change of emphasis in high-energy experimental physics that took place in the late 1960s and early 1970s. Instead of a commonsense (Pickering’s term) approach of concentrating on the most conspicuous phenomena in collisions of high-energy particles (i.e., the fragmentation of the particles into great numbers of other particles going mostly in the original direction of the particle beams), experimentalists instead began to do experiments suggested by theo...

It is simply a logical fallacy to go from the observation that science is a social process to the conclusion that the final product, our scientific theories, is what it is because of the social and historical forces acting in this process. A party of mountain climbers may argue over the best path to the peak, and these arguments may be conditioned by the history and social structure of the expedition, but in the end either they find a good path to the peak or they do not, and when they get there they know it.

The “negotiations” over changes in scientific theory go on and on, with scientists changing their minds again and again in response to calculations and experiments, until finally one view or another bears an unmistakable mark of objective success. It certainly feels to me that we are discovering something real in physics, something that is what it is without any regard to the social or historical conditions that allowed us to discover it.

I suspect that Gerald Holton is close to the truth in seeing the radical attack on science as one symptom of a broader hostility to Western civilization that has bedeviled Western intellectuals from Oswald Spengler on. Modern science is an obvious target for this hostility; great art and literature have sprung from many of the world’s civilizations, but ever since Galileo scientific research has been overwhelmingly dominated by the West.

Balancing this against the benign applications of science and its role in liberating the human spirit, I think that modern science, along with democracy and contrapuntal music, is something that the West has given the world in which we should take special pride.

We can look forward to the day when science can no longer be identified with the West but is seen as the shared possession of humankind.

CHAPTER VIII

TWENTIETH CENTURY BLUES

The electroweak theory is the part of the standard model that deals with weak and electromagnetic forces. It is based on an exact symmetry principle, which says that the laws of nature take the same form if everywhere in the equations of the theory we replace the fields of the electron and neutrino with mixed fields—for instance, one field that is 30% electron and 70% neutrino and another field that is 70% electron and 30% neutrino—and at the same time similarly mix up the fields of other families of particles, such as the up quark and the down quark. This symmetry principle is called local, meaning that the laws of nature are supposed to be unchanged even if these mixtures vary from moment to moment or from one location to another. There is one other family of fields, whose existence is dictated by this symmetry principle in something like the way that the existence of the gravitational field is dictated by the symmetry among different coordinate systems. This family consists of the fields of the photon and the W and Z particles, and these fields too must be mixed up with each other when we mix up the electron and neutrino fields and the quark fields. The exchange of photons is responsible for the electromagnetic force, while the exchange of W and Z particles produces the weak nuclear force, so this symmetry between electrons and neutrinos is also a symmetry between the electromagnetic force and the weak nuclear force.

In such cases we say that the symmetry is broken, although a better term would be “hidden,” because the symmetry is still there in the equations, and these equations govern the properties of the particles. We call this phenomenon a spontaneous symmetry breaking because nothing breaks the symmetry in the equations of the theory; the symmetry breaking appears spontaneously in the various solutions of these equations.

The value of the field that breaks the symmetry is commonly called its vacuum value, because the field takes this value in the vacuum, far from the influence of any particles.