The 4% Universe: Dark Matter, Dark Energy, and the Race to Discover the Rest of Reality

by Richard Panek

Maybe how gradually or abruptly a light curve rose and fell could serve as a reliable indicator of its brightness relative to other Type Ia supernovae. And if you knew the relative luminosities among supernovae, then, through the inverse-square law, you would also be able to figure out relative distances. You would be able to use supernovae to do cosmology.

three years after proving to themselves that they could find a distant supernova, they had gone ahead and figured out how to find supernovae on a regular basis. After discovering the three in early 1994 on the Isaac Newton Telescope, they found three more with the Kitt Peak 4-meter telescope, in the mountains southwest of Tucson, Arizona. By June 1995 they had accumulated eleven distant Type Ia in total,

“Why don’t you subtract the galaxy?” “Subtracting the galaxy” is just about the first thing you do if you’re trying to get the spectrum of a supernova. If you want to isolate the supernova light, you take a spectrum from the part of the galaxy containing the supernova, which is flooded with light from the galaxy, and then you take a spectrum from a different part of the galaxy, away from the supernova, and then you subtract the second reading from the first. Ideally, the spectrum of the supernova itself pops out.

The spectrum from the galaxy would still be the same; he wasn’t changing the quality of the data. He was just changing its intensity. He subtracted this spectrum from the supernova spectrum (which also contained the galaxy spectrum), and out popped a beautiful supernova spectrum.

astronomy itself was changing. The traditional go-it-alone aesthetic was disappearing. The diversity of the science and the complications of technology were forcing the field into greater and greater specialization. You couldn’t just study the heavens anymore; you studied planets, or stars, or galaxies, or the Sun. But you didn’t study just stars anymore, either; you studied only the stars that explode. And you didn’t study just supernovae; you studied only one type. And you didn’t study just Type Ia; you specialized in the mechanism leading to the thermonuclear explosion, or you specialized in what metals the explosion creates, or you specialized in how to use the light from the explosion to measure the deceleration of the expansion of the universe—how to perform the photometry or do the spectroscopy or write the code. A collaboration could easily become unwieldy.

Hamuy examined it and told Riess he thought it was, as scientists say by way of praise, “robust.” Riess, however, said he had a problem. So far he hadn’t been able to test the LCS method on real data. Could he see Hamuy’s? Hamuy hesitated. Your data was your data. Until you published it, it was yours and yours alone. But Riess was persistent, and Hamuy was a guest (at Harvard, of Bob Kirshner), and he relented. Hamuy agreed to show Riess his first thirteen light curves, though not before exacting a promise: Riess could use them only to test his technique, not as part of a paper about the technique. A few weeks later, Hamuy got an e-mail from Riess. The technique was working. Riess was excited. Could he publish the results after all? That, Hamuy reminded Riess, wasn’t part of the deal. But again he relented, though not before exacting another promise: that Riess wouldn’t publish his paper using Hamuy’s data before Hamuy published his own paper on the thirteen supernovae. Riess would have to wait until Hamuy’s paper had cleared the referee stage at the Astronomical Journal. When it did, in early September 1994, Hamuy let Harvard—meaning Riess and Kirshner, as well as William Press, who provided mathematical guidance—know that they were free to submit their own paper. They did. But they submitted it to Astrophysical Journal Letters, a publication that, as its name suggests, traffics in shorter papers—and, therefore, briefer lead times. Hamuy had to work hard to convince the A...

The Hubble Space Telescope didn’t see a lot; its field of view was minuscule compared with the old 200-inch or new 10-meter behemoths on terra firma. But what it saw, it saw with a clarity that no other telescope could approach. Through a CCD camera on an earthbound telescope, a very distant galaxy appeared as a smudge of pixels. Subtracting the light of the galaxy to isolate the light of the supernova was difficult work; witness the four months Leibundgut needed to figure out that the “very faint” 1995K was a Type Ia. The high resolution of HST, however, would make a supernova pop out of its host galaxy. Subtracting the light of the galaxy would be not only easier but far more precise. That increase in photometric precision was crucial.

HST was an important resource, and the search for high-redshift supernovae was a new field, and HST would surely get better results if both groups used it for their nearly identical experiments.

universes would match his data. And one did. It was a universe that not only didn’t have enough matter to slow the expansion but had a mass density of negative 36 percent. It was a universe without matter. It was a universe that didn’t exist. “Lo and behold,” Riess told himself. Both teams had been operating under the assumption that the universe was full of matter and only matter. They knew some of it was dark, of course, but what was missing was still fundamentally matter. They had therefore assumed that only matter would be influencing the expansion of the universe.

They looked at the error bars and figured that the matter, dark or otherwise, was maybe 20 or 30 or 40 percent. Which left 60 or 70 or 80 percent . . . something else.

What they didn’t have—between the dark matter they couldn’t see and this new force they couldn’t imagine—was any idea what the universe was.

On monday evenings throughout the mid-1980s, the DuPage County Center for Scientific Culture held what would have been the only course in its catalogue, if it had had a catalogue. The classroom was the basement of a split-level suburban home. The student body was sparse: a handful of researchers, postdocs, and graduate students from the University of Chicago or the nearby Fermi National Accelerator Laboratory, as well as, often, a distinguished visitor. The students served as the instructors, too. Tuition was five bucks a week, which bought you pizza (or sometimes barbecued “backup” hamburgers, resurrected from the bowels of the freezer), beer, and a turn at the blackboard. The topics varied from week to week, and from moment to moment. The first topic of the evening might be a recently published paper that had gotten it all wrong, whatever “it” was, or a wildly speculative hypothesis that someone wanted to test. From there the evening would follow its own path. The chalk would pass from hand to hand, feverishly, amid shouts of criticism or approval and screams of sudden insight or instant regret, and by the end of the class the participants were vowing to write a response eviscerating the recent paper that had gotten it all wrong, whatever “it” was, or a new paper championing an original theory that, whether one of the participants had arrived at the meeting espousing it or it had arisen over the course of the evening, had already gone through its own peer evisceration. (...

That universe was now nearly twenty years old. While observers were trying to measure the two numbers in cosmology—the universe’s current rate of expansion, and how much the expansion was slowing down—theorists were trying to figure out how the expansion itself worked. Like Jim Peebles in his instant classic Physical Cosmology, they wanted to make explicit the connection between the physics of the early universe and the universe we see today. That connection had been implicit from the start, in Lemaître’s invocation of a primeval atom.

The discovery of the cosmic microwave background, however, made a dialogue between particle physicists and astronomers necessary.

Their colleague Jim Peebles had already performed the calculation for the relic temperature of the primeval fireball, and Bob Dicke himself had invented some of the equipment in the Bell Labs experiment.

Don’t try to solve a problem until you think you have the answer. That approach was the opposite of how particle physics usually worked. In particle physics, the math came first. The math told you that a particle should exist, and that you could create that hypothetical particle from existing particles. Then you (and a thousand colleagues) commandeered an accelerator and smashed those existing particles together at velocities approaching the speed of light and waited for the hypothetical particle to pop into existence. Nothing wrong with that approach. It worked. But Feynman had taught Turner that sometimes you didn’t need to do the math first. Instead, you needed to trust your intuition. To leap to a conclusion first. To imagine what the universe might be, and then go back and do the math until, with luck, it matched.

Hawking and Gary W. Gibbons, also at Cambridge, decided to consolidate the remaining funds and go all out: an assault on the farthest frontier of cosmology, the “very early Universe,” which the invitation defined as “< 1 sec.”

And the universe did seem to be homogeneous and isotropic. The discovery of the cosmic microwave background seventeen years earlier had satisfied most cosmologists that they now had the answer to the question of whether the universe was simple: Yes. On the largest scale it would look the same no matter where you were in it. And they had answered the question of how simple the universe was: Very. The cosmic microwave background was extremely smooth, just as theory had predicted. But assuming that something is the way it is—even if those assumptions turn out to be correct, as the Big Bang theory’s apparently were—is no substitute for understanding how it got that way. Why would a universe be, of all things a universe could be, simple—and not just simple, but so simple? On reflection, maybe the answer to the question of how simple the universe was shouldn’t have been the satisfying “Very” but a suspect “Too.” Now, however, cosmology had a possible answer to the question of how the universe became so simple.

According to his calculations, the universe had gone through a monumental expansion in its first moment of existence. At the age of a trillionth of a trillionth of a trillionth of one second—or 1/1,000,000,000,000,000,000,000,000,000,000,000,000th of a second—the universe had expanded ten septillion-fold—or to 10,000,000,000,000,000,000,000,000 times its previous size.

“Is the Universe a Quantum Fluctuation?” According to the laws of quantum mechanics, virtual particles can arise out of the emptiness of space—and actually do, as experiments since the middle of the century had shown again and again. Tryon wondered if the universe might be the result of one such quantum pop. The argument became less sensational if you kept in mind that in quantum theory everything was a matter of probabilities. Therefore, anything was possible. Perhaps specific events were vanishingly unlikely—the creation of a universe from the nothingness of the vacuum, for instance. But they weren’t impossible. And over the course of eternity, why shouldn’t one or another of those vanishingly unlikely events come to pass? The universe, Tryon wrote, “is simply one of those things which happen from time to time.”

The problem with Tryon’s idea was that it couldn’t account for the size of our universe. Inflation, however, could. Guth realized that the infant universe could have gone through a process that physicists call a “phase transition” and everybody else calls “the thing that happens when water turns into ice or vice versa.” When the temperature of water changes, the transformation doesn’t happen all at once. It’s not as if the word goes out and suddenly every molecule of H2O in the lake has melted into liquid or hardened into ice. Instead, the transformation happens piecemeal. Even within small sections of the pond the ice isn’t freezing or melting uniformly. Cracks and fissures appear faintly, then harden, leaving a veined appearance. Guth found that if you apply that transformation mathematically to the conditions of the early universe, the phase transition would have produced a temporary vacuum. That vacuum, in turn, would have produced a negative pressure—a strong gravitational repulsion—that would have expanded space exponentially. The universe would have doubled in size, then doubled in size again, then doubled in size yet again. It would have done this at least a hundred times, and it would have done so over the course of 10−35 seconds (or 1/1035). After that, the vacuum would have decayed, the exponential expansion would have stopped, and the standard expansion of the universe—the one in the Big Bang theory that we can see for ourselves in the redshifting of the light ...

Guth recalled a lecture by Bob Dicke that he had attended the previous year, one of a series that Dicke and Peebles had been delivering on a topic they called the “flatness problem.” They would explain to their audiences that the fate of the universe depended on how much matter was in the universe: enough to reverse the expansion, not enough, or just right. The designation that scientists had given to the measure determining the fate of the universe was, aptly, the final letter in the Greek alphabet, omega. If the universe contained half the mass necessary to halt the expansion, then you would say omega equaled 0.5, or if it contained three-quarters of the necessary mass, you would say omega equaled 0.75. If the universe contained more than enough mass to halt the expansion, then omega equaled more than 1—1.5 times, maybe, or 2 times, or 100 times. And if the universe contained just the right amount—precisely the critical density to stop the universe from expanding but keep it from collapsing back on itself—then omega equaled 1. Astronomers would even be able to measure omega, if they had a standard candle that they could trace far enough across the universe. But you might not need observations to know omega, Dicke argued. Theory alone might be enough.

the value of omega depended on the measure of matter, and whatever matter the universe had then, it would have now and forever. But for Big Bang theorists like Dicke and Peebles, a flat universe posed a problem similar to the one Newton and Einstein faced: Why would a universe that was full of matter not be collapsing through the effects of gravity? Newton had to invoke a universe of evenly spaced stars—plus God. Einstein had to invoke a universe of randomly spaced stars—plus lambda. Evidence for an expanding universe had allowed Einstein to abandon lambda and prompted future generations to try to figure out how to measure the rate at which the expansion was slowing. But now Dicke and Peebles were arguing that in a Big Bang universe, omega had to equal 1. The expansion had to slow to a virtual stop and stay there forever. All the matter in the universe had to reach a state of gravitational equilibrium—an eventuality with the same likelihood as a pencil standing on its point forever. Not impossible, according to the laws of classical physics, but not likely either.

If inflation did occur, then two distant parts of the universe would have been in contact with each other when the universe was less than 10 seconds old.

Other theorists—Andrei Linde, at the Lebedev Physical Institute in Moscow, and, independently, Paul Steinhardt and Andreas Albrecht, at the University of Pennsylvania—identified the problem and found the solution. They reconceived the inflationary period to be, as Guth came to think of it, less like the bubbling of boiling water than the congealing of a single Jell-O bubble. The problem with the one-bubble inflationary model, however, was that it still had to account for the visible universe—homogeneous and isotropic, but not too homogeneous and isotropic, or else we wouldn’t be here. They were all borrowing from Hawking. In 1973 Hawking had redefined the study of the early universe with his work on black holes; he found that, owing to a combination of quantum and gravitational effects, they weren’t one-way tickets to a singularity. At the edge of the event horizon—the black hole’s ring of no return—quantum effects dictated that particles and antiparticles would be popping into existence, while gravitational effects dictated that one partner would disappear into the black hole but not the other. Rather than annihilating each other “immediately,” one would slip over the edge, into the black hole, but the other would escape into space and the universe as we know it. Black holes, Hawking contended, aren’t black after all. They leak radiation—Hawking radiation, as it came to be called. In effect, Hawking had begun to bridge the two seemingly irreconcilable theories of the twen...

Even being able to analyze the problem was progress of a sort. Had they made new inflation work? No. But they had agreed on a way that they might make it work. Now they knew they had the right equations, even if they hadn’t yet figured out how to solve them.

Cosmologists in the early 1980s had leaped to a conclusion, embracing inflation simply because it explained and solved so much, and then they had gone back and labored to make the math work. And they’d succeeded.

Just as Darwin explained how single-cell creatures could evolve into species upon species, B2FH explained how single-proton atoms could eventually form the elements in the periodic table.

echoing Darwin’s last line in On the Origin of Species, “The elements have evolved, and are evolving.”

The “Inner Space” had changed over the years. In Schramm’s original vision, inner space referred to particle physics, and he and his colleagues had succeeded in beating down the processes of element formation to what they called the “era of nucleosynthesis”—the period when the universe was between 1 second and 100 seconds old and the cosmic fog had cooled enough to allow the formation of elements. They knew what should have been happening in the previous fraction of a second, when protons and neutrons and electrons were ricocheting. But Hawking and Guth had changed the game; they came at the universe from the other end—not from the present backward but the beginning forward. They took into account not only particle physics but quantum physics. If inflation was right, then the quantum jiggling during the inflationary period—all 10 seconds of it—had frozen into the fissures in the cosmic pond, the veins in the ice, creating the structure around which matter (dark or not) had clustered, leading to the universe we see today.

In 1976, the same year that Rubin and colleagues published the paper on the Rubin-Ford effect, a team led by Richard Muller and George Smoot at LBNL had taken a suggestion by Peebles in Physical Cosmology and, planting a Dicke radiometer aboard a U-2 plane, tried to measure the motion of our galaxy against the cosmic microwave background to determine whether the universe as a whole rotates. What they discovered instead was that our galaxy seemed to be racing through space at nearly 400 miles per second. Smoot made the announcement in April 1977 at an American Physical Society meeting during time that Peebles had yielded to him from his own talk. The phenomenon “is a real dilemma for theorists,” Peebles said, and Smoot suspected that the two of them were the only physicists in the room who understood the implications: For the universe to contain such local volatility yet still appear homogeneous and isotropic on a large scale, that scale was going to have to be much larger than anyone had ever imagined.

That same year, Jim Peebles compiled a map of the millions of galaxies that the Lick Observatory had observed and found that not only did galaxies seem to be doing what galaxies interacting gravitationally with one another would be doing—clustering—but the clusters seemed to be doing what clusters interacting gravitationally with one another would be doing—superclustering.

The universe was flat. The universe was open. And that’s where cosmology rested as the decade stretched on: a neither-nor state of suspension that would have to await further observations,

In 1917, in considering the implications of general relativity, Einstein saw that the universe was inherently unstable. Just as Newton had invoked God to keep his version of the universe from collapsing, so Einstein added a symbol to his equations—arbitrarily, the Greek letter lambda, A. Whatever lambda was, it was counteracting gravity, because, in Einstein’s idea of a stable universe, something had to be. It was the reason that a universe full of matter attracting other matter through gravity wasn’t collapsing. After Hubble’s discovery of evidence for the expansion, the universe didn’t need lambda, and Einstein abandoned it. Unlike Newton’s God, however, you couldn’t altogether ignore it. Lambda was, after all, in the equation. What you could do instead was set lambda to zero. That’s what generations of observers and theorists had done. Sometimes they left the assumption implicit, simply failing to mention lambda. Often they stated the assumption explicitly: “Assume Λ = 0.” For most observers and theorists, lambda was there and it wasn’t there. It occupied a parallel existence, like a ghost in the attic.

the validation of the Big Bang theory through the discovery of the cosmic background radiation eliminated the need for what had come to be called “the cosmological constant.”

It next took up residence in quasars, those mysterious sources of tremendous energy at mystifying distances. In 1967, a trio of Cornell theorists published a paper in the Astrophysical Journal examining, as the title said, “Quasi-Stellar Objects in Universes with Non-Zero Cosmological Constant.” They were trying to resolve some possible inconsistencies in the behavior of quasars. But as the understanding of the evolution of quasars became clearer, the need for lambda again receded. Then in 1975 two prominent astronomers argued in Nature that studies of elliptical galaxies as standard candles indicated that “the most plausible cosmological models have a positive cosmological constant.”

Then came inflation. It solved problems, the flatness and horizon problems. It explained improbabilities, the homogeneity and isotropy of the universe on the largest scales.

inflation came with a prediction: that the universe was flat. That the amount of matter in the universe was equal to the critical amount that would keep it from collapsing. That omega equaled 1.

The problem for the inflation theorists, however, was that the observers were consistently finding evidence that the amount of matter in the universe was only 20 percent of the critical amount—that omega equaled 0.2.

At the final session of the Nuffield workshop, the theoretical physicist Frank Wilczek summarized the conference proceedings, concluding with “A Shopping List of Questions.” Among them was whether omega was equal to 1. “If not,” he said, “we must give up on inflation.” Simple subtraction led you to conclude that for omega to equal 1 while observers were finding ...

Maybe the rest of the matter was in a form that astronomers hadn’t yet detected.

“Flatness of the Universe: Reconciling Theoretical Prejudices with Observational Data.” Those “theoretical prejudices” referred to inflation’s prediction of a flat universe, and the paper explored two ways of reconciling those prejudices with the data. One was a particle of some sort from the era of Big Bang nucleosynthesis—the field that Schramm had pioneered. The other possibility was “a relic cosmological constant.”

“The cosmological constant,” Turner liked to say, “is the last refuge of scoundrel cosmologists, beginning with Einstein.” He himself, in his “heart of hearts,” thought the cosmological constant must be zero. But he also knew that the cosmological constant had “every right to be there.” And as he and Rocky Kolb often insisted, their generation wasn’t going to make the mistake that Einstein and other twentieth-century cosmologists had made by not taking every remotely serious option seriously.

“High mass density is dead in the water.” Their conclusion: an omega of 0.2.

The following year, Peebles wrote a paper, “Tests of Cosmological Models Constrained by Inflation,” that offered his theoretical interpretation of that data. Maybe omega was indeed 0.2 and lambda equaled 0, he wrote, but in that case “we lose the attractive inflationary explanation for the observed large-scale homogeneity of the universe.” He didn’t want a cosmological constant. “It’s ugly,” he often said. “It’s an addition.” If he were building a universe, he thought, he wouldn’t put in a cosmological constant: “No bells and whistles.” But perhaps because inflation solved the flatness problem he’d articulated with Dicke, or perhaps because he constitutionally distrusted a simple universe, he accepted the possibility with equanimity. “Considering the observations,” he said, “I think the universe might have put in bells and whistles—a cosmological constant.”

Theorists are always saying something. That’s their job. They don’t need to believe what they’re saying. The theorist’s goal isn’t to be right but to be reasonable—to make an internally consistent argument that observers can then go out and reinforce or disprove.

In 1992 observers threw cosmological theorists the biggest bone since the discovery of the cosmic microwave background more than a quarter of a century earlier: the Cosmic Background Explorer results—the ones that said that the universe was flat. The following year, Turner and Kolb added a preface to the paperback edition of The Early Universe reviewing the COBE results and declaring them “a shot in the arm” for a flat universe.

“The Observational Case for a Low-Density Universe with a Non-Zero Cosmological Constant” was the title of another paper. And then there was Turner again, again with Lawrence Krauss: “The Cosmological Constant Is Back.” The cosmological constant was still the last refuge, but it was a refuge nonetheless.

“Critical Dialogues in Cosmology” conference at Princeton, part of the university’s celebration of its 250th anniversary, in the summer of 1996. The purpose of the conference was to bring together the world’s leading cosmologists to address the field’s greatest challenges. One such event, inevitably, involved the value of omega, and it took the form of a debate. On one side was Avishai Dekel, who had recently measured galaxy motions that were consistent with an omega equal to 1. On the other side was Turner, arguing that the amount of matter in the universe was not enough to nudge omega to 1. But he didn’t stop there. Instead he used the forum to argue that omega was indeed equal to 1, because the cosmological constant would close the gap.