A Mind at Play: How Claude Shannon Invented the Information Age

by Jimmy Soni

digital watch is nothing like the sun; an analog watch is the memory of a shadow’s circuit around a dial.

Bush said in a speech at MIT. “Men of our profession—we teachers—are bound to be impressed with the tendency of youths of strikingly capable minds to become interested in one small corner of science and uninterested in the rest of the world. . . . It is unfortunate when a brilliant and creative mind insists upon living in a modern monastic cell.”

For instance, in our 26-letter alphabet there are 676 possible two-letter strings (or 262), but 17,576 three-letter strings (or 263). Hartley, like Nyquist before him, found this inconvenient. A measure of information would be more workable if it increased linearly with each additional symbol, rather than exploding exponentially. In this way, a 20-letter telegram could be said to hold twice as much information as a 10-letter telegram, provided that both messages used the same alphabet. That explains what the logarithm is doing in Hartley’s formula (and Nyquist’s): it’s converting an exponential change into a linear one. For Hartley, this was a matter of “practical engineering value.”

How much information, for instance, is in a picture? We can think of a picture just as we think of a telegraph. In the same way we can break a telegraph into a discrete string of dots and dashes, we can break a picture into a discrete number of squares that Hartley called “elementary areas”: what were later termed picture elements, or pixels.

even to the messages the world had not yet conceived of—messages for which Shannon was, here, preparing the way. They encompass human voices as electromagnetic waves that bounce off satellites and the ceaseless digital churn of the Internet. They pertain just as well to the codes written into DNA. Although the molecule’s discovery was still five years in the future, Shannon was arguably the first to conceive of our genes as information bearers, an imaginative leap that erased the border between mechanical, electronic, and biological messages.

Biological messages

This was the hunch that Shannon had suggested to Hermann Weyl in Princeton in 1939, and which he had spent almost a decade building into theory: Information is stochastic. It is neither fully unpredictable nor fully determined. It unspools in roughly guessable ways. That’s why the classic model of a stochastic process is a drunk man stumbling down the street. He doesn’t walk in the respectably straight line that would allow us to predict his course perfectly.

Appearances to the contrary, Theseus the mouse was mainly the passive part of the endeavor: the underlying maze itself held the information and propelled Theseus with its magnet. Technically, as Shannon would point out, the mouse wasn’t solving the maze; the maze was solving the mouse. Yet, one way or another, the system was able to learn.

The mouse wasn't soving the maze; the maze solved the mouse

Yet there were moods in which Shannon’s cheeriness over the future of machines curdled into misanthropy. “We artificial intelligence people are insatiable,” he once wrote. Once machines were beating our grandmasters, writing our poetry, completing our mathematical proofs, and managing our money, we would, Shannon observed only half-jokingly, be primed for extinction. “These goals could mark the beginning of a phase-out of the stupid, entropy-increasing, and militant human race in favor of a more logical, energy conserving, and friendly species—the computer.”

a genius is simply someone who is usefully irritated.

In 1973, for instance, Shannon was invited to give the first Claude Shannon lecture in Ashkelon, Israel, for the Institute of Electrical and Electronics Engineers Information Theory Society. “I have never seen such stage fright,” the mathematician Elwyn Berlekamp recalled. “It never would have occurred to me that anyone in front of friends could be so scared.” Shannon required extensive nerve calming in the wings and would only take the stage accompanied by a friend. Another attendee remembered, “He just felt that people were going to expect so much of him in this talk, and he was afraid that he didn’t have anything significant to say. Needless to say, he gave a fantastic talk, but in my mind . . . it showed me what a modest man he was.”

I don’t know how history is taught here in Japan, but in the United States in my college days, most of the time was spent on the study of political leaders and wars—Caesars, Napoleons and Hitlers. I think this is totally wrong. The important people and events of history are the thinkers and innovators, the Darwins, Newtons and Beethovens whose work continues to grow in influence in a positive fashion.

Shannon also remained a tinkerer through even his last days; he set to taking apart his walker, imagining a better design all the way.

Shannon’s work left its lasting mark on generations of American engineers and mathematicians, in part, because it resonated with their fundamental values. What were those values? Simplicity matters. Elegant math was forceful math. Inessential items, superfluous writing, extra work—all of them should be discarded.

Shannon’s puzzle-solving research style was in full swing when I was an MIT graduate student. Intellectualism was in the air. Everyone wanted to understand mathematics and physics as well as communication. Starting companies, making millions, developing real applications was secondary. There was interest in bringing the theory closer to reality, but it was theory-based. Our role models were relaxed, curious, and had time to reflect.