The Information: A History, a Theory, a Flood

by James Gleick

Most speakers’ instincts about phonemes are biased, too, by their knowledge of the written alphabet, which codifies language in its own sometimes arbitrary ways.

Having reduced spoken words, in all their sonic richness, to such a minimal code, how could the drums distinguish them? The answer lay partly in stress and timing, but these could not compensate for the lack of consonants and vowels. Thus, Carrington discovered, a drummer would invariably add “a little phrase” to each short word. Songe, the moon, is rendered as songe li tange la manga—“the moon looks down at the earth.” Koko, the fowl, is rendered koko olongo la bokiokio—“the fowl, the little one that says kiokio.” The extra drumbeats, far from being extraneous, provide context.

Redundancy—inefficient by definition—serves as the antidote to confusion. It provides second chances.

“Language in fact bears the same relationship to the concept of mind that legislation bears to the concept of parliament,” says Jonathan Miller: “it is a competence forever bodying itself in a series of concrete performances.”

For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom.

There is a progression from pictographic, writing the picture; to ideographic, writing the idea; and then logographic, writing the word.

Because the basic unit was the word, thousands of distinct symbols were required. This is efficient in one way, inefficient in another. Chinese unifies an array of distinct spoken languages: people who cannot speak to one another can write to one another.

All known alphabets, used today or found buried on tablets and stone, descend from the same original ancestor, which arose near the eastern littoral of the Mediterranean Sea, sometime not much before 1500 BCE, in a region that became a politically unstable crossroads of culture, covering Palestine, Phoenicia, and Assyria.

The persistence of writing made it possible to impose structure on what was known about the world and, then, on what was known about knowing. As soon as one could set words down, examine them, look at them anew the next day, and consider their meaning, one became a philosopher, and the philosopher began with a clean slate and a vast project of definition to undertake.

Speech is too fleeting to allow for analysis. Logic descended from the written word, in Greece as well as India and China, where it developed independently. Logic turns the act of abstraction into a tool for determining what is true and what is false: truth can be discovered in words alone, apart from concrete experience.

Logic implicates symbolism directly: things are members of classes; they possess qualities, which are abstracted and generalized.

Literate people take for granted their own awareness of words, along with the array of word-related machinery: classification, reference, definition. Before literacy, there is nothing obvious about such techniques.

Names are not the things they name. Classes are not coextensive with subclasses.

“The larger the number of senses involved, the better the chance of transmitting a reliable copy of the sender’s mental state.”

The lexis is a measure of shared experience, which comes from interconnectedness.

Like the printing press, the telegraph, and the telephone before it, the Internet is transforming the language simply by transmitting information differently.

I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.

With tongue in cheek he wrote Alfred, Lord Tennyson, to suggest a correction for the poet’s couplet: “Every minute dies a man, / Every minute one is born.” I need hardly point out to you that this calculation would tend to keep the sum total of the world’s population in a state of perpetual equipoise, whereas it is a well-known fact that the said sum total is constantly on the increase. I would therefore take the liberty of suggesting that in the next edition of your excellent poem the erroneous calculation to which I refer should be corrected as follows: “Every moment dies a man / And one and a sixteenth is born.” I may add that the exact figures are 1.167, but something must, of course, be conceded to the laws of metre.

The difficulty of forming a clear conception of the subject is increased by the fact that while we have to deal with novel and strange facts, we have also to use old words in novel and inconsistent senses.

“A mathematician is not a man who can readily manipulate figures; often he cannot,” Bush wrote. “He is primarily an individual who is skilled in the use of symbolic logic on a high plane, and especially he is a man of intuitive judgment.”

Even though a particular closed system of formal logic must contain statements that could neither be proved nor disproved from within the system, it might conceivably be decided, as it were, by an outside referee—by external logic or rules.

Now, instead of boosting the power, a sender can overcome noise by using extra symbols for error correction—just as an African drummer makes himself understood across long distances, not by banging the drums harder, but by expanding the verbosity of his discourse.

A stochastic process is neither deterministic (the next event can be calculated with certainty) nor random (the next event is totally free). It is governed by a set of probabilities. Each event has a probability that depends on the state of the system and perhaps also on its previous history. If for event we substitute symbol, then a natural written language like English or Chinese is a stochastic process. So is digitized speech; so is a television signal.

Shannon wanted to define the measure of information (represented as H) as the measure of uncertainty: “of how much ‘choice’ is involved in the selection of the event or of how uncertain we are of the outcome.”

If a letter can be guessed from what comes before, it is redundant; to the extent that it is redundant, it provides no new information. If English is 75 percent redundant, then a thousand-letter

Paradoxical though it sounded, random messages carry more information. The implication was that natural-language text could be encoded more efficiently for transmission or storage.

Whether removing redundancy to increase efficiency or adding redundancy to enable error correction, the encoding depends on knowledge of the language’s statistical structure to do the encoding.

For Wiener, entropy was a measure of disorder; for Shannon, of uncertainty. Fundamentally, as they were realizing, these were the same. The more inherent order exists in a sample of English text—order in the form of statistical patterns, known consciously or unconsciously to speakers of the language—the more predictability there is, and in Shannon’s terms, the less information is conveyed by each subsequent letter.

Babbage’s engine allows Turing to rebut a superstition he senses forming in the zeitgeist of 1950. People seem to feel that the magic of digital computers is essentially electrical; meanwhile, the nervous system is also electrical. But Turing is at pains to think of computation in a universal way, which means in an abstract way. He knows it is not about electricity at all: Since Babbage’s machine was not electrical, and since all digital computers are in a sense equivalent, we see that this use of electricity cannot be of theoretical importance.… The feature of using electricity is thus seen to be only a very superficial similarity.