A Mind at Play: How Claude Shannon Invented the Information Age

by Jimmy Soni

For instance, in our 26-letter alphabet there are 676 possible two-letter strings (or 262), but 17,576 three-letter strings (or 263). Hartley, like Nyquist before him, found this inconvenient. A measure of information would be more workable if it increased linearly with each additional symbol, rather than exploding exponentially. In this way, a 20-letter telegram could be said to hold twice as much information as a 10-letter telegram, provided that both messages used the same alphabet. That explains what the logarithm is doing in Hartley’s formula (and Nyquist’s): it’s converting an exponential change into a linear one. For Hartley, this was a matter of “practical engineering value.”