Turing's Cathedral: The Origins of the Digital Universe

by George Dyson

Von Neumann’s project was the physical realization of Alan Turing’s Universal Machine, a theoretical construct invented in 1936. It was not the first computer. It was not even the second or third computer. It was, however, among the first computers to make full use of a high-speed random-access storage matrix, and became the machine whose coding was most widely replicated and whose logical architecture was most widely reproduced. The stored-program computer, as conceived by Alan Turing and delivered by John von Neumann, broke the distinction between numbers that mean things and numbers that do things. Our universe would never be the same.

As World War II drew to a close, the scientists who had built the atomic bomb at Los Alamos wondered, “What’s next?” Some, including Richard Feynman, vowed never to have anything to do with nuclear weapons or military secrecy again. Others, including Edward Teller and John von Neumann, were eager to develop more advanced nuclear weapons, especially the “Super,” or hydrogen bomb.

Numerical simulation of chain reactions within computers initiated a chain reaction among computers, with machines and codes proliferating as explosively as the phenomena they were designed to help us understand.

A digital universe—whether 5 kilobytes or the entire Internet—consists of two species of bits: differences in space, and differences in time. Digital computers translate between these two forms of information—structure and sequence—according to definite rules. Bits that are embodied as structure (varying in space, invariant across time) we perceive as memory, and bits that are embodied as sequence (varying in time, invariant across space) we perceive as code. Gates are the intersections where bits span both worlds at the moments of transition from one instant to the next.

That two symbols were sufficient for encoding all communication had been established by Francis Bacon in 1623. “The transposition of two Letters by five placeings will be sufficient for 32 Differences [and] by this Art a way is opened, whereby a man may expresse and signifie the intentions of his minde, at any distance of place, by objects … capable of a twofold difference onely,” he wrote, before giving examples of how such binary coding could be conveyed at the speed of paper, the speed of sound, or the speed of light.3 That zero and one were sufficient for logic as well as arithmetic was established by Gottfried Wilhelm Leibniz in 1679, following the lead given by Thomas Hobbes in his Computation, or Logique of 1656. “By Ratiocination, I mean computation,” Hobbes had announced. “Now to compute, is either to collect the sum of many things that are added together, or to know what remains when one thing is taken out of another. Ratiocination, therefore is the same with Addition or Substraction; and if any man adde Multiplication and Division, I will not be against it, seeing … that all Ratiocination is comprehended in these two operations of the minde.”4 The new computer, for all its powers, was nothing more than a very fast adding machine, with a memory of 40,960 bits.

Von Neumann set out to build a Universal Turing Machine that would operate at electronic speeds. At its core was a 32-by-32-by-40-bit matrix of high-speed random-access memory—the nucleus of all things digital ever since. “Random access” meant that all individual memory locations—collectively constituting the machine’s internal “state of mind”—were equally accessible at any time. “High speed” meant that the memory was accessible at the speed of light, not the speed of sound. It was the removal of this constraint that unleashed the powers of Turing’s otherwise impractical Universal Machine.

Three technological revolutions dawned in 1953: thermonuclear weapons, stored-program computers, and the elucidation of how life stores its own instructions as strings of DNA.

The mechanism of translation between sequence and structure in biology and the mechanism of translation between sequence and structure in technology were set on a collision course. Biological organisms had learned to survive in a noisy, analog environment by repeating themselves, once a generation, through a digital, error-correcting phase, the same way repeater stations are used to convey intelligible messages over submarine cables where noise is being introduced. The transition from digital once a generation to digital all the time began in 1953.

What began as an isolated 5-kilobyte matrix is now expanding by over two trillion transistors per second (a measure of the growth in processing and memory) and five trillion bits of storage capacity per second (a measure of the growth in code).16 Yet we still face the same questions that were asked in 1953. Turing’s question was what it would take for machines to begin to think. Von Neumann’s question was what it would take for machines to begin to reproduce.

When the Institute for Advanced Study agreed, against all objections, to allow von Neumann and his group to build a computer, the concern was that the refuge of the mathematicians would be disturbed by the presence of engineers. No one imagined the extent to which, on the contrary, the symbolic logic that had been the preserve of the mathematicians would unleash the powers of coded sequences upon the world. “In those days we were all so busy doing what we were doing we didn’t think very much about this enormous explosion that might happen,” says Willis Ware.

Von Neumann’s talents stood out, even in Budapest. “Johnny’s most characteristic trait was his boundless curiosity about everything and anything, his compulsive ambition to know, to understand any problem, no matter on what level,” Klári recalls. “Anything that would tickle his curiosity with a question mark, he could not leave alone; he would sulk, pout and be generally impossible until, at least to his own satisfaction, he had found the right answer.” He was able to disassemble any problem and then reassemble it in a way that rendered the answer obvious as a result. He had an ability, “perhaps somewhat rare among mathematicians,” explains Stan Ulam, “to commune with the physicists, understand their language, and to transform it almost instantly into a mathematician’s schemes and expressions. Then, after following the problems as such, he could translate them back into expressions in common use among physicists.”14

Axiomatization is the reduction of a subject to a minimal set of initial assumptions, sufficient to develop the subject fully without new assumptions having to be introduced along the way. The axiomatization of set theory formed the foundations, mathematically, of everything else. An ambitious previous attempt, by Bertrand Russell and Alfred North Whitehead, despite 1,984 pages extending across three volumes, still left fundamental questions unresolved. Von Neumann started fresh. “The conciseness of the system of axioms is surprising,” comments Stan Ulam. “The axioms take only a little more than one page of print. This is sufficient to build up practically all of the naive set theory and therewith all of modern mathematics … and the formal character of the reasoning employed seems to realize Hilbert’s goal of treating mathematics as a finite game.”32 The mathematical landscape of the early twentieth century was dominated by Göttingen’s David Hilbert, who believed that from a strictly limited set of axioms, all mathematical truths could be reached by a sequence of well-defined logical steps. Hilbert’s challenge, taken up by von Neumann, led directly both to Kurt Gödel’s results on the incompleteness of formal systems of 1931 and Alan Turing’s results on the existence of noncomputable functions (and universal computation) of 1936. Von Neumann set the stage for these two revolutions, but missed taking the decisive steps himself. Gödel proved that within any formal system suff...

In his axiomatization of set theory, “one can divine the germ of von Neumann’s future interest in computing machines,” says Ulam, speaking with hindsight from 1958. “The economy of the treatment seems to indicate a more fundamental interest in brevity than in virtuosity for its own sake. It thereby helped prepare the grounds for an investigation of the limits of finite formalism by means of the concept of ‘machine.’ ”33 Von Neumann’s style was now set. He would approach a subject, identify the axioms that made it tick, and then, using those axioms, extend the subject beyond where it was when he showed up. “What made it possible for him to make so many contributions in so many different parts of mathematics?” asks Paul Halmos. “It was his genius at synthesizing and analyzing things. He could take large units, rings of operators, measures, continuous geometry, direct integrals, and express the unit in terms of infinitesimal little bits. And he could take infinitesimal little bits and put together large units with arbitrarily prescribed properties. That’s what Johnny could do, and what no one else could do as well.”34

“Formal logic has to be taken over by mathematicians,” Veblen had announced on New Year’s Eve 1924, when the plans for what would become the Institute for Advanced Study were first taking form in his mind. “There does not exist an adequate logic at the present time, and unless the mathematicians create one, no one else is likely to do so.”10 It was Gödel, above anyone else—and now directly above von Neumann—who proved Veblen’s instincts correct. In 1924 both von Neumann and Gödel were working on the logical foundations of mathematics, before Gödel’s incompleteness theorems brought the Hilbert program to a close. Von Neumann “believed in Hilbert’s goal of a final and conclusive axiomatization of mathematics,” according to Stan Ulam, “and yet, in a 1925 paper, in a mysterious flash of intuition, he pointed out the limits of any axiomatic formulation of set theory. That was perhaps a sort of vague forecast of Gödel’s result.”11 The seeds of doubt were sown. In September of 1930, at the Königsberg conference on the epistemology of the exact sciences, Gödel made the first, tentative announcement of his incompleteness results. Von Neumann immediately saw the implications, and, as he wrote to Gödel on November 30, 1930, “using the methods you employed so successfully … I achieved a result that seems to me to be remarkable, namely, I was able to show that the consistency of mathematics is unprovable,” only to find out, by return mail, that Gödel had got there first.12 “He was disa...

Shift registers, as Leibniz had demonstrated 260 years earlier, could perform binary arithmetic simply by shifting an entire row of binary digits one position to the right or left. Data were never transferred directly between adjacent toggles; instead, the state of each individual toggle was replicated upward into a temporary register, the lower register was cleared, and then and only then were the data shifted, diagonally, back down into the original register. There was no lower bound to how slowly the computer could be stepped through a sequence of instructions. Unlike the well-behaved physical marbles that Leibniz had imagined shifting from column to column in 1679, electrons were always looking for an escape.

Von Neumann’s approach was to bring a handful of engineers into a den of mathematicians, rather than a handful of mathematicians into a den of engineers. This freed the project from any constraints that might have been imposed by an established group of engineers with preexisting opinions as to how a computer should be built.

In our universe, we measure time with clocks, and computers have a “clock speed,” but the clocks that govern the digital universe are very different from the clocks that govern ours. In the digital universe, clocks exist to synchronize the translation between bits that are stored in memory (as structures in space) and bits that are communicated by code (as sequences in time). They are clocks more in the sense of regulating escapement than in the sense of measuring time. “The I.A.S. computing machine is non-synchronous; that is, decisions between elementary alternatives, and enforcement of these decisions are initiated not with reference to time as an independent variable but rather according to sequence,” Bigelow explained to Maurice Wilkes—who had just succeeded in coaxing Cambridge’s delay-line storage EDSAC computer into operation before the Institute’s—in 1949. “Time, therefore, does not serve as an index for the location of information, but instead counter readings are used, the counters themselves being actuated by the elementary events.”14 “It was all of it a large system of on and off, binary gates,” Bigelow reiterated fifty years later. “No clocks. You don’t need clocks. You only need counters. There’s a difference between a counter and a clock. A clock keeps track of time. A modern general purpose computer keeps track of events.”15 This distinction separates the digital universe from our universe, and is one of the few distinctions left. The acceleration from kil...

What about von Neumann’s question—whether machines would begin to reproduce? We gave digital computers the ability to modify their own coded instructions—and now they are beginning to exercise the ability to modify our own. Are we using digital computers to sequence, store, and better replicate our own genetic code, thereby optimizing human beings, or are digital computers optimizing our genetic code—and our way of thinking—so that we can better assist in replicating them? In the beginning was the command line: a human programmer supplied an instruction and a numerical address. There is no proscription against computers supplying their own instructions, and an ever-diminishing fraction of commands have ever been touched by human hands or human minds. Now the commands and addresses are as likely to be delivered the other way: the global computer supplies an instruction, and an address that maps to a human being via a personal device. That the resulting human behavior can only be counted on statistically, not deterministically, is, as von Neumann demonstrated in 1952 with his Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components, no obstacle to the synthesis of those unreliable human beings into a reliable organism. We are returning to the landscape as envisioned by von Neumann in 1948: with a few large computers handling much of the computation in the world. The big computers, however, are not physically centralized; they are distributed ac...