An engaging multi-biography/history of the repeatedly-reinvented Kelly criterion, mixed in with overviews of Claude Shannon, John Larry Kelly, Jr., Ed Thorp, and their famous gambling adventures in beating blackjack and roulette and, as some of the first ‘quants’, the stock market. (Like Thompson sampling, the Kelly criterion has been reinvented many times; Poundstone lists at least 4 inventors: Kelly, Leo Breiman, Bernoulli, and Henry Latané.)

Poundstone starts with the early mob and the ‘numbers racket’ and sports gambling, where Kelly’s metaphor of ‘the wire’ giving an edge on betting was quite literal: spotters at the race-tracks would race to communicate the results to bettors and bookies across the country, so they could take bets on already-won races, leading to mob wars over the lucrative monopoly over using telephones/telegraph services to communicate said results, which constituted a remarkable fraction of telecom profits. (Shades of HFT.) Thus, notorious characters like Bugsy Siegel enter into a book about statistics as gambling becomes a major revenue source replacing the loss of alcohol. (Poundstone speculates that Edgar Hoover’s famous denial of the existence of the Mafias was due to being paid off by betting on fixed horse races.) The mob part may seem like a colorful and interesting yet irrelevant diversion, but it sets the context for inveterate mob gambler Manny Kimmel (famous for betting on anything, and knowing clever tricks like betting people about whether anyone in the room shared birthdays - in other words, one of the only practical applications of the birthday paradox I’ve seen outside of cryptography), who, aside from being the founder of Time Warner (!) would eventually pop up as Thorp & Shannon’s bankroller. Thorp then enters the picture as a grad student deeply interested in making money using physics, starting with roulette wheels, which didn’t work out initially, and then publishing an instantly famous paper on beating blackjack with card counting, which brought him to Shannon (for mechanical & mathematical assistance) and Kelly (for deciding how much to bet) and Manny Kimmel (for the money to bet with). An interlude brings in Kelly and his Kelly criterion itself, and makes clear the connection to information theory and efficient markets: a few bits of information about outcomes (ie having probabilities which do not match the implicit probabilities in the prices of bets/investments) equates to excess returns, and the more information, the larger the returns with aggressive betting. The Kelly criterion optimizes the extraction of money, compared to other betting strategies like the martingale which don’t take into account the extra information. While excellent in theory, Thorp/Shannon/Kimmel’s (Kelly was uninvolved and busy chasing the still-elusive dream of voice synthesis) blackjack did not go well: the casinos shamelessly cheated any customer doing well, Thorp claims one even drugged him twice (although he was never beaten by casino thugs like other card counters), and new unpopular rules were announced to negate card counting. So Thorp moved onto roulette and the stock market. Thorp’s first big edge was in warrants: since warrants expire quickly, they need to go to 0 or 1 over a short time period, and if the market is efficient, they should follow a random walk of the sort familiar in physics from molecules, and their expected value easily calculated… and mispriced warrants spotted and purchased. Which sounds a bit paradoxical. And the risk of buying warrants can be offset just buy buying or selling short just some of the underlying stock. Thorp made money off warrants, and then published the strategy for increasing the credibility of his new hedge fund, and moved onto convertible bonds by applying similar reasoning: the bond should have a certain value which reflects the probability that the stock will spike high enough to make the built-in option worth exercising, and since stocks should follow a random walk, all you need to know is the variance… inventing Black-Scholes. With Kelly, he could bet heavily on the safest profitable investments, up to 150% of the fund, without blowing up. (In one amusing anecdote, Black-Scholes used their pricing model to spot a particularly mispriced warrant; then the company changed the terms of the warrants, wiping out the warrant holders and Black-Scholes, in a way that insiders had known was coming and sold all their warrants.) Thorp had a genius for regularly spotting these sorts of opportunities, and Poundstone says ‘“I’ve estimated for myself that if I had to pay no taxes, state or federal, I’d have about thirty-two times as much wealth as I actually do,” Thorp told me recently’ (Thorp’s net worth is estimated somewhere in the hundreds of millions) because his fund would have grown much faster if it could’ve reinvested all its earnings & profitability didn’t have to take into account taxation. This is plausible considering compound growth, the fund’s final 15.1% average annual return, and what ultimately killed Thorp’s fund: involvement in Michael Milken’s financial empire as their stock broker, which, as part of Rudy Giuliani’s crusade in applying RICO to anything possible to get himself elected, turned up some tax fraud on Thorp’s fund’s part (he blames his partner who was in charge of the implementation end of things). The timing was particularly bad for Thorp because investors would flock to hedge funds during that time period, as exemplified by LTCM, which Poundstone devotes a section to, arguing that LTCM also exemplified the perils of non-Kelly investment by putting too much at risk (which seems a little tendentious, since my understanding was that the real problem was they underestimated the correlations of many assets in an economic crisis; the underestimation led them to overbet and thus exposed them to huge losses, and some formalized Kelly-like proportional investment wouldn’t’ve saved them from the fundamental mistakes, any more than the KC saves you from an incorrect estimate of your edge or assuming that correlated bets are independent). Thorp returned to trading eventually, and in terms of his lifetime performance:

In May 1998 Thorp reported that his investments had grown at an average 20 percent annual return (with 6 percent standard deviation) over 28.5 years. “To help persuade you that this may not be luck,” Thorp wrote, “I estimate that…I have made $80 billion worth of purchases and sales (‘action,’ in casino language) for my investors. This breaks down into something like one and a quarter million individual ‘bets’ averaging about $65,000 each, with on average hundreds of ‘positions’ in place at any one time. Over all, it would seem to be a moderately ‘long run’ with a high probability that the excess performance is more than chance.”

Thorp’s money may continue on:

The Thorps recently endowed a chair at the University of California at Irvine mathematics department. The gift consists of one million dollars to be invested entirely in stocks, with the university limited to withdrawing only 2 percent a year. The fund is expected to compound exponentially in inflation-adjusted dollars. Ultimately, Thorp hopes, it will fund the most richly endowed university chair in the world, and will help draw exceptional mathematical talent to UC Irvine.

Poundstone goes in more depth into the statistics than I expected, and although there’s not that much that can be said about the Kelly criterion (particularly in 2005, before the latest burst of interest in it due to evolutionary & biological interpretations of the Kelly criterion & probability matching/Thompson sampling), he benefits tremendously from extensive access to Shannon’s papers and Thorp’s reminiscences about his mob connections while trying to beat the casinos. Indeed, some of the reviews criticize the characterization of Thorp as almost forgettable and perhaps insufficiently critical due to Poundstone’s dependency. What is a little remarkable to me is how well Shannon did financially by 3 early venture capital investments, and how little Shannon contributed intellectually after his information theory paper; I had always somehow assumed that Claude Shannon, a genius who had offhandedly made a major contribution to genetics simply because his advisor forced him to work on genetics, and had created fully-formed information theory, had died in the 1950s or something, because how else would such a genius have not made further major contributions? But no! Shannon died in 2001! Ramsay died on the operating table; von Neumann had cancer; Kelly himself dropped dead of a stroke on a NYC sidewalk; Pitts was mentally ill and died of alcoholism; but Shannon was rich, tenured, sound as a bell in mind & body, and infinitely respected - what was his excuse? Poundstone explains that Shannon was simply too unambitious (and perfectionist) to work hard on any big topics or write up and publish properly any of his findings! (Instead, he worked on an endless succession of hobbies like juggling or Rubik’s cube or discovering that the smallest ride-able unicycle is >18 inches.) One of the more depressing demonstrations that raw genius is not enough.

I did not notice any major errors (asides from perhaps a confusion of Euler and Gauss, and overstating the obscurity of Louis Bachelier's life). One downside is that despite the involvement of Jimmy Savage, Poundstone never mentions the connections to subjective Bayesianism, personal interpretations of probability, or Thompson sampling. (Which would, if nothing else, have partially explained why Savage's career was so peripatetic - it wasn't just his acerbic opinions as Poundstone claims.)