How Not to Be Wrong: The Power of Mathematical Thinking

by Jordan Ellenberg

“Mathematics is not just a sequence of computations to be carried out by rote until your patience or stamina runs out—although it might seem that way from what you’ve been taught in courses called mathematics. Those integrals are to mathematics as weight training and calisthenics are to soccer. If you want to play soccer—I mean, really play, at a competitive level—you’ve got to do a lot of boring, repetitive, apparently pointless drills. Do professional players ever use those drills? Well, you won’t see anybody on the field curling a weight or zigzagging between traffic cones. But you do see players using the strength, speed, insight, and flexibility they built up by doing those drills, week after tedious week. Learning those drills is part of learning soccer.

The Statistical Research Group (SRG), where Wald spent much of World War II, was a classified program that yoked the assembled might of American statisticians to the war effort—something like the Manhattan Project, except the weapons being developed were equations, not explosives. And the SRG was actually in Manhattan, at 401 West 118th Street in Morningside Heights, just a block away from Columbia University. The building now houses Columbia faculty apartments and some doctor’s offices, but in 1943 it was the buzzing, sparking nerve center of wartime math. At the Applied Mathematics Group−Columbia, dozens of young women bent over Marchant desktop calculators were calculating formulas for the optimal curve a fighter should trace out through the air in order to keep an enemy plane in its gunsights. In another apartment, a team of researchers from Princeton was developing protocols for strategic bombing. And Columbia’s wing of the atom bomb project was right next door.

The mathematical talent at hand was equal to the gravity of the task. In Wallis’s words, the SRG was “the most extraordinary group of statisticians ever organized, taking into account both number and quality.” Frederick Mosteller, who would later found Harvard’s statistics department, was there. So was Leonard Jimmie Savage, the pioneer of decision theory and great advocate of the field that came to be called Bayesian statistics.* Norbert Wiener, the MIT mathematician and the creator of cybernetics, dropped by from time to time. This was a group where Milton Friedman, the future Nobelist in economics, was often the fourth-smartest person in the room. The smartest person in the room was usually Abraham Wald. Wald had been Allen Wallis’s teacher at Columbia, and functioned as a kind of mathematical eminence to the group.

The armor, said Wald, doesn’t go where the bullet holes are. It goes where the bullet holes aren’t: on the engines. Wald’s insight was simply to ask: where are the missing holes? The ones that would have been all over the engine casing, if the damage had been spread equally all over the plane? Wald was pretty sure he knew. The missing bullet holes were on the missing planes. The reason planes were coming back with fewer hits to the engine is that planes that got hit in the engine weren’t coming back. Whereas the large number of planes returning to base with a thoroughly Swiss-cheesed fuselage is pretty strong evidence that hits to the fuselage can (and therefore should) be tolerated. If you go to the recovery room at the hospital, you’ll see a lot more people with bullet holes in their legs than people with bullet holes in their chests. But that’s not because people don’t get shot in the chest; it’s because the people who get shot in the chest don’t recover. Here’s an old mathematician’s trick that makes the picture perfectly clear: set some variables to zero.

Wald’s recommendations were quickly put into effect, and were still being used by the navy and the air force through the wars in Korea and Vietnam. I can’t tell you exactly how many American planes they saved, though the data-slinging descendants of the SRG inside today’s military no doubt have a pretty good idea. One thing the American defense establishment has traditionally understood very well is that countries don’t win wars just by being braver than the other side, or freer, or slightly preferred by God. The winners are usually the guys who get 5% fewer of their planes shot down, or use 5% less fuel, or get 5% more nutrition into their infantry at 95% of the cost. That’s not the stuff war movies are made of, but it’s the stuff wars are made of. And there’s math every step of the way.

We tend to teach mathematics as a long list of rules. You learn them in order and you have to obey them, because if you don’t obey them you get a C-. This is not mathematics. Mathematics is the study of things that come out a certain way because there is no other way they could possibly be. Now let’s be fair: not everything in mathematics can be made as perfectly transparent to our intuition as addition and multiplication. You can’t do calculus by common sense. But calculus is still derived from our common sense—Newton took our physical intuition about objects moving in straight lines, formalized it, and then built on top of that formal structure a universal mathematical description of motion. Once you have Newton’s theory in hand, you can apply it to problems that would make your head spin if you had no equations to help you. In the same way, we have built-in mental systems for assessing the likelihood of an uncertain outcome. But those systems are pretty weak and unreliable, especially when it comes to events of extreme rarity. That’s when we shore up our intuition with a few sturdy, well-placed theorems and techniques, and make out of it a mathematical theory of probability. The specialized language in which mathematicians converse with one another is a magnificent tool for conveying complex ideas precisely and swiftly. But its foreignness can create among outsiders the impression of a sphere of thought totally alien to ordinary thinking. That’s exactly wrong. Math is l...

John von Neumann, in his 1947 essay “The Mathematician,” warned: As a mathematical discipline travels far from its empirical source, or still more, if it is a second and third generation only indirectly inspired by ideas coming from “reality” it is beset with very grave dangers. It becomes more and more purely aestheticizing, more and more purely l’art pour l’art. This need not be bad, if the field is surrounded by correlated subjects, which still have closer empirical connections, or if the discipline is under the influence of men with an exceptionally well-developed taste. But there is a grave danger that the subject will develop along the line of least resistance, that the stream, so far from its source, will separate into a multitude of insignificant branches, and that the discipline will become a disorganized mass of details and complexities. In other words, at a great distance from its empirical source, or after much “abstract” inbreeding, a mathematical subject is in danger of degeneration.*

But as long as you believe there’s such a thing as too much welfare state and such a thing as too little, you know the linear picture is wrong. Some principle more complicated than “More government bad, less government good” is in effect. The generals who consulted Abraham Wald faced the same kind of situation: too little armor meant planes got shot down, too much meant the planes couldn’t fly. It’s not a question of whether adding more armor is good or bad; it could be either, depending on how heavily armored the planes are to start with. If there’s an optimal answer, it’s somewhere in the middle, and deviating from it in either direction is bad news. Nonlinear thinking means which way you should go depends on where you already are.

The square in the picture is called the inscribed square; each of its corners just touches the circle, but it doesn’t extend beyond the circle’s boundary. Why do this? Because circles are mysterious and intimidating, and squares are easy. If you have before you a square whose side has length X, its area is X times X—indeed, that’s why we call the operation of multiplying a number by itself squaring! A basic rule of mathematical life: if the universe hands you a hard problem, try to solve an easier one instead, and hope the simple version is close enough to the original problem that the universe doesn’t object. The inscribed square breaks up into four triangles, each of which is none other than the isosceles triangle we just drew.* So the square’s area is four times the area of the triangle. That triangle, in turn, is what you get when you take a 1 x 1 square and cut it diagonally in half like a tuna fish sandwich.

So the area of the circle is trapped in between 2.83 and 3.31. Why stop there? You can stick points in between the corners of the octagon (whether inscribed or circumscribed) to make a 16-gon; after some more trigonometric figuring, that tells you that the area of the circle is in between 3.06 and 3.18. Do it again, to make a 32-gon; and again, and again, and pretty soon you have something that looks like this: Wait, isn’t that just the circle? Of course not! It’s a regular polygon with 65,536 sides. Couldn’t you tell? The great insight of Eudoxus and Archimedes was that it doesn’t matter whether it’s a circle or a polygon with very many very short sides. The two areas will be close enough for any purpose you might have in mind.

The reason the 0.999 . . . problem is difficult is that it brings our intuitions into conflict. We would like the sum of an infinite series to play nicely with arithmetic manipulations like the ones we carried out on the previous pages, and this seems to demand that the sum equal 1. On the other hand, we would like each number to be represented by a unique string of decimal digits, which conflicts with the claim that the same number can be called either 1 or 0.999 . . . , as we like. We can’t hold on to both of these desires at once; one must be discarded. In Cauchy’s approach, which has amply proved its worth in the two centuries since he invented it, it’s the uniqueness of the decimal expansion that goes out the window. We’re untroubled by the fact that the English language sometimes uses two different strings of letters (i.e., two words) to refer synonymously to the same thing in the world; in the same way, it’s not so bad that two different strings of digits can refer to the same number.

Two words referring to the same thing: kettle and pot

An important rule of mathematical hygiene: when you’re field-testing a mathematical method, try computing the same thing several different ways. If you get several different answers, something’s wrong with your method. For example: the 2004 bombings at the Atocha train station in Madrid killed almost 200 people. What would be an equivalently deadly bombing at Grand Central Station? The United States has almost seven times the population of Spain. So if you think of 200 people as 0.0004% of the Spanish population, you find that an equivalent attack would kill 1,300 people in the United States. On the other hand, 200 people is 0.006% of the population of Madrid; scaling up to New York City, which is two and a half times as large, gives you 463 victims. Or should we compare the province of Madrid with the state of New York? That gives you something closer to 600. This multiplicity of conclusions should be a red flag. Something is fishy with the method of proportions.

As you can see, the fraction of heads converges inexorably toward 50% as you flip more and more coins, as if squeezed by an invisible vise. You can see the same effect in the simulations. The proportions of heads in the first group of tries, the Smalls, range from 30% to 90%. With a hundred flips at a time, the range narrows: just 40% to 60%. And with a thousand flips, the range of proportions is only 46.2% to 53.7%. Something is pushing those numbers closer and closer to 50%. That something is the cold, strong hand of the Law of Large Numbers. I won’t state that theorem precisely (though it is stunningly handsome!), but you can think of it as saying the following: the more coins you flip, the more and more extravagantly unlikely it is that you’ll get 80% heads. In fact, if you flip enough coins, there’s only the barest chance of getting as many as 51%!

It’s the very same effect that makes political polls less reliable when fewer voters are polled. And it’s the same, too, for brain cancer. Small states have small sample sizes—they are thin reeds whipped around by the winds of chance, while the big states are grand old oaks that barely bend. Measuring the absolute number of brain cancer deaths is biased toward the big states; but measuring the highest rates—or the lowest ones!—puts the smallest states in the lead. That’s how South Dakota can have one of the highest rates of brain cancer death while North Dakota claims one of the lowest. It’s not because Mount Rushmore or Wall Drug is somehow toxic to the brain; it’s because smaller populations are inherently more variable. That’s a mathematical fact you already know, even if you don’t know you already know it. Who’s the most accurate shooter in the NBA? A month into the 2011−12 season, five players were locked in a tie for the highest shooting percentage in the league: Armon Johnson, DeAndre Liggins, Ryan Reid, Hasheem Thabeet, and Ronny Turiaf. Who? That’s the point. These were not the five best shooters in the NBA. These were people who barely ever played. Armon Johnson, for instance, appeared in one game for the Portland Trail Blazers. He took one shot. He made it.

The bell curve/gendarme’s hat is tall in the middle and very flat near the edges, which is to say that the farther a discrepancy is from zero, the less likely it is to be encountered. And this can be precisely quantified. If you flip N coins, the chance that you’ll end up being off by at most the square root of N from 50% heads is about 95.45%. The square root of 1,000 is about 31; indeed, eighteen of our twenty big thousand-coin trials above, or 90%, were within 31 heads of 500. If I kept playing the game, the fraction of times I ended up somewhere between 469 and 531 heads would get closer and closer to that 95.45% figure.* It feels like something is making it happen. Indeed, de Moivre himself might have felt this way. By many accounts, he viewed the regularities in the behavior of repeated coin flips (or any other experiment subject to chance) as the work of God’s hand itself,

The way the overall proportion settles down to 50% isn’t that fate favors tails to compensate for the heads that have already landed; it’s that those first ten flips become less and less important the more flips we make. If I flip the coin a thousand more times, and get about half heads, then the proportion of heads in the first 1,010 flips is also going to be close to 50%. That’s how the Law of Large Numbers works: not by balancing out what’s already happened, but by diluting what’s already happened with new data, until the past is so proportionally negligible that it can safely be forgotten.

This is a perfect example of the soup you get into when you start reporting percentages of numbers, like net job gains, that might be either positive or negative. Wisconsin added ninety-five hundred jobs, which is good; but neighboring Minnesota, under Democratic governor Mark Dayton, added more than thirteen thousand in the same month. Texas, California, Michigan, and Massachusetts also outpaced Wisconsin’s job gains. Wisconsin had a good month, that’s true—but it didn’t contribute as many jobs as the rest of the country put together, as the Republican messaging suggested. In fact, what was going on is that job losses in other states almost exactly balanced out the jobs created in places like Wisconsin, Massachusetts, and Texas. That’s how Wisconsin’s governor could claim his state accounted for half the nation’s job growth, and Minnesota’s governor, if he’d cared to, could have said that his own state was responsible for 70% of it, and they could both, in this technically correct but fundamentally misleading way, be right.

What does this mean for you, if you’re fortunate enough to have some money to invest? It means you’re best off resisting the lure of the hot new fund that made 10% over the last twelve months. Better to follow the deeply unsexy advice you’re probably sick of hearing, the “eat your vegetables and take the stairs” of financial planning: instead of hunting for a magic system or an adviser with a golden touch, put your money in a big dull low-fee index fund and forget about it. When you sink your savings into the incubated fund with the eye-popping returns, you’re like the newsletter getter who invests his life savings with the Baltimore stockbroker; you’ve been swayed by the impressive results, but you don’t know how many chances the broker had to get those results.

It’s massively improbable to get hit by a lightning bolt, or to win the lottery; but these things happen to people all the time, because there are a lot of people in the world, and a lot of them buy lottery tickets, or go golfing in a thunderstorm, or both. Most coincidences lose their snap when viewed from the appropriate distance. On July 9, 2007, the North Carolina Cash 5 lottery numbers came up 4, 21, 23, 34, 39. Two days later, the same five numbers came up again. That seems highly unlikely, and it seems that way because it is. The chance of those two lottery draws matching by pure chance was tiny, less than two in a million. But that’s not the relevant question, if you’re deciding how impressed to be. After all, the Cash 5 game had already been going on for almost a year, offering many opportunities for coincidence; it turns out the chance some three-day period would have seen two identical Cash 5 draws was a much less miraculous one in a thousand. And Cash 5 isn’t the only game in town. There are hundreds of five-number lottery games running all over the country, and have been for years; when you put them all together, it’s not at all surprising that you get a coincidence like two identical draws in three days. That doesn’t make each individual coincidence any less improbable. But here comes the chorus again: improbable things happen a lot. Aristotle, as usual, was here first: despite lacking any formal notion of probability, he was able to understand that “it is pr...

Australian lottery numbers 4,5,6,7,8 are questioned as being faked

It is very unlikely that any given set of rabbinic appellations is well matched to birth and death dates in the book of Genesis. But with so many ways of choosing the names, it’s not at all improbable that among all the choices there would be one that made the Torah look uncannily prescient. Given enough chances, finding codes is a cinch. It’s especially easy if you use Michael Drosnin’s less scientific approach to code-finding. Drosnin said of code skeptics, “When my critics find a message about the assassination of a prime minister encrypted in Moby-Dick, I’ll believe them.” McKay quickly found equidistant letter sequences in Moby-Dick referring to the assassination of John F. Kennedy, Indira Gandhi, Leon Trotsky, and, for good measure, Drosnin himself. As I write this, Drosnin remains alive and well despite the prophecy. He is on his third Bible code book, the last of which he advertised by taking out a full-page ad in a December 2010 edition of the New York Times, warning President Obama that, according to letter sequences hidden in Scripture, Osama bin Laden might already have a nuclear weapon.

The more chances you give yourself to be surprised, the higher your threshold for surprise had better be. If a random Internet stranger who eliminated all North American grains from his food intake reports that he dropped fifteen pounds and his eczema went away, you shouldn’t take that as powerful evidence in favor of the maize-free plan. Somebody’s selling a book about that plan, and thousands of people bought that book and tried it, and the odds are very good that, by chance alone, one among them will experience some weight loss and clear skin the next week. And that’s the guy who’s going to log in as saygoodbye2corn452 and post his excited testimonial, while the people for whom the diet failed stay silent.

The computation tells us there are exactly two times when the missile is at ground level. One time is 0.4939 . . . seconds ago. That’s when the missile was launched. The other time is 40.4939 . . . seconds from now. That’s when the missile lands. Perhaps it doesn’t seem so troubling, especially if you’re used to the quadratic formula, to get two answers to the same question. But when you’re twelve it represents a real philosophical shift. You’ve spent six long years in grade school figuring out what the answer is, and now, suddenly, there is no such thing. And those are just quadratic equations! What if you have to solve x3 + 2x2 − 11x= 12?

This is a cubic equation, which is to say it involves x raised to the third power. Fortunately, there is a cubic formula that allows you to figure out, by a direct computation, what values of x could have gone in the box to make 12 fall out when you turn the crank. But you didn’t learn the cubic formula in school, and the reason you didn’t learn it in school is that it’s kind of a mess, and wasn’t worked out until the late Renaissance, when itinerant algebraists roamed across Italy, engaging one another in fierce public equation-solving battles with money and status on the line. The few people who knew the cubic formula kept it to themselves or wrote it down in cryptic rhymed verse. Long story. The point is, reverse engineering is hard. The problem of inference, which is what the Bible coders were wrestling with, is hard because it’s exactly this kind of problem. When we are scientists, or Torah scholars, or toddlers gaping at the clouds, we are presented with observations and asked to build theories—what went into the box to produce the world that we see? Inference is a hard thing, maybe the hardest thing. From the shape of the clouds and the way they move we struggle to go backward, to solve for x, the system that made them.

If you’re the researcher who developed the new drug, the null hypothesis is the thing that keeps you up at night. Unless you can rule it out, you don’t know whether you’re on the trail of a medical breakthrough or just barking up the wrong metabolic pathway. So how do you rule it out? The standard framework, called the null hypothesis significance test, was developed in its most commonly used form by R. A. Fisher, the founder of the modern practice of statistics,* in the early twentieth century.

So here’s the procedure for ruling out the null hypothesis, in executive bullet-point form: Run an experiment. Suppose the null hypothesis is true, and let p be the probability (under that hypothesis) of getting results as extreme as those observed. The number p is called the p-value. If it is very small, rejoice; you get to say your results are statistically significant. If it is large, concede that the null hypothesis has not been ruled out. How small is “very small”? There’s no principled way to choose a sharp dividing line between what is significant and what is not; but there’s a tradition, which started with Fisher himself and is now widely adhered to, of taking p= 0.05, or 1/20, to be the threshold. Null hypothesis significance testing is popular because it captures our intuitive way of reasoning about uncertainty. Why do we find the Bible codes compelling, at least at first glance? Because codes like the ones Witztum uncovered are very unlikely under the null hypothesis that the Torah doesn’t know the future. The value of p—the likelihood of finding so many equidistant letter sequences, so accurate in their demographic profiling of notable rabbis—is very close to 0.

A significance test is a scientific instrument, and like any other instrument, it has a certain degree of precision. If you make the test more sensitive—by increasing the size of the studied population, for example—you enable yourself to see ever-smaller effects. That’s the power of the method, but also its danger. The truth is, the null hypothesis, if we take it literally, is probably just about always false. When you drop a powerful drug into a patient’s bloodstream, it’s hard to believe the intervention has exactly zero effect on the probability that the patient will develop esophageal cancer, or thrombosis, or bad breath. Every part of the body speaks to every other, in a complex feedback loop of influence and control. Everything you do either gives you cancer or prevents it. In principle, if you carry out a powerful enough study, you can find out which it is. But those effects are usually so minuscule that they can be safely ignored. Just because we can detect them doesn’t always mean they matter. If only we could go back in time to the dawn of statistical nomenclature and declare that a result passing Fisher’s test with a p-value of less than 0.05 was “statistically noticeable” or “statistically detectable” instead of “statistically significant”!

He had what basketball fans call “the hot hand”—the apparent inability to miss a shot, no matter how great the distance or how fierce the defense. Except there’s supposed to be no such thing. In 1985, in one of the most famous contemporary papers in cognitive psychology, Thomas Gilovich, Robert Vallone, and Amos Tversky (hereafter GVT) did to basketball fans what B. F. Skinner had done to lovers of the Bard. They obtained records of every shot taken by the 1980−81 Philadelphia 76ers in their forty-eight home games and analyzed them statistically. If players tended toward hot streaks and cold streaks, you might expect a player to be more likely to hit a shot following a basket than a shot following a miss. And when GVT surveyed NBA fans, they found this theory had broad support; nine out of ten fans agreed that a player is more likely to sink a shot when he’s just hit two or three baskets in a row. But nothing of the kind was going on in Philadelphia.

Joseph Berkson, the longtime head of the medical statistics division at the Mayo Clinic, who cultivated (and loudly broadcast) a vigorous skepticism about methodology he thought shaky, offered a famous example demonstrating the pitfalls of the method. Suppose you have a group of fifty experimental subjects, who you hypothesize (H) are human beings. You observe (O) that one of them is an albino. Now, albinism is extremely rare, affecting no more than one in twenty thousand people. So given that H is correct, the chance you’d find an albino among your fifty subjects is quite small, less than 1 in 400,* or 0.0025. So the p-value, the probability of observing O given H, is much lower than .05. We are inexorably led to conclude, with a high degree of statistical confidence, that H is incorrect: the subjects in the sample are not human beings. It’s tempting to think of “very improbable” as meaning “essentially impossible,” and, from there, to utter the word “essentially” more and more quietly in our mind’s voice until we stop paying attention to it.* But impossible and improbable are not the same—not even close. Impossible things never happen. But improbable things happen a lot. That means we’re on quivery logical footing when we try to make inferences from an improbable observation, as reductio ad unlikely asks us to. That time in North Carolina when the lottery combo 4, 21, 23, 34, 39 came up twice in a week raised a lot of questions; was something wrong with the game? But eac...

One of the first theorems ever proved in number theory is that of Euclid, which tells us that the primes are infinite in number; we will never run out, no matter how far along the number line we let our minds range. But mathematicians are greedy types, not inclined to be satisfied with a mere assertion of infinitude. After all, there’s infinite and then there’s infinite. There are infinitely many powers of 2, but they’re very rare. Among the first one thousand numbers, there are only ten of them: 1, 2, 4, 8, 16, 32, 64, 128, 256, and 512. There are infinitely many even numbers, too, but they’re much more common: exactly 500 out of the first 1,000 numbers. In fact, it’s pretty apparent that out of the first N numbers, just about (1/2)N will be even. Primes, it turns out, are intermediate—more common than the powers of 2 but rarer than even numbers. Among the first N numbers, about N/log N are prime; this is the Prime Number Theorem, proven at the end of the nineteenth century by the number theorists Jacques Hadamard and Charles-Jean de la Vallée Poussin.

Modern medicine and social science are not haruspicy. But a steadily louder drum circle of dissident scientists has been pounding out an uncomfortable message in recent years: there’s probably a lot more entrail reading in the sciences than we’d like to admit. The loudest drummer is John Ioannidis, a Greek high school math star turned biomedical researcher whose 2005 paper “Why Most Published Research Findings Are False” touched off a fierce bout of self-criticism (and a second wave of self-defense) in the clinical sciences. Some papers plead for attention with a title more dramatic than the claims made in the body, but not this one. Ioannidis takes seriously the idea that entire specialties of medical research are “null fields,” like haruspicy, in which there are simply no actual effects to be found. “It can be proven,” he writes, “that most claimed research findings are false.”

Most scientific studies don’t consist of blundering around the genome at random; they test hypotheses that the researchers have some preexisting reason to think might be true, so the bottom row of the box is not quite so enormously dominant over the top. But the crisis of replicability is real. In a 2012 study, scientists at the California biotech company Amgen set out to replicate some of the most famous experimental results in the biology of cancer, fifty-three studies in all. In their independent trials, they were able to reproduce only six. How can this have happened? It’s not because genomicists and cancer researchers are dopes. In part, the replicability crisis is simply a reflection of the fact that science is hard and that most ideas we have are wrong—even most of those ideas that survive a first round of prodding.

That slope is the shape of p-hacking. It tells you that a lot of experimental results that belong over on the unpublishable side of the p= .05 boundary have been cajoled, prodded, tweaked, or just plain tortured until, at last, they end up just on the happy side of the line. That’s good for the scientists who need publications, but it’s bad for science.

But the culture is changing. Reformers with loud voices like Ioannidis and Simonsohn, who speak both to the scientific community and to the broader public, have generated a new sense of urgency about the danger of descent into large-scale haruspicy. In 2013, the Association for Psychological Science announced that they would start publishing a new genre of article, called Registered Replication Reports. These reports, aimed at reproducing the effects reported in widely cited studies, are treated differently from usual papers in a crucial way: the proposed experiment is accepted for publication before the study is carried out. If the outcomes support the initial finding, great news, but if not, they’re published anyway, so the whole community can know the full state of the evidence. Another consortium, the Many Labs project, revisits high-profile findings in psychology and attempts to replicate them in large multinational samples. In November 2013, psychologists were cheered when the first suite of Many Labs results came back, finding that 10 of the 13 studies addressed were successfully replicated.

But some problems are more like predicting the weather. That’s another situation where having plenty of fine-grained data, and the computational power to plow through it quickly, can really help. In 1950, it took the early computer ENIAC twenty-four hours to simulate twenty-four hours of weather, and that was an astounding feat of space-age computation. In 2008, the computation was reproduced on a Nokia 6300 mobile phone in less than a second. Forecasts aren’t just faster now; they’re longer-range and more accurate, too. In 2010, a typical five-day forecast was as accurate as a three-day forecast had been in 1986.

It’s tempting to imagine that predictions will just get better and better as our ability to gather data gets more and more powerful; won’t we eventually have the whole atmosphere simulated to a high precision in a server farm somewhere under The Weather Channel’s headquarters? Then, if you wanted to know next month’s weather, you could just let the simulation run a little bit ahead. It’s not going to be that way. Energy in the atmosphere burbles up very quickly from the tiniest scales to the most global, with the effect that even a minuscule change at one place and time can lead to a vastly different outcome only a few days down the road. Weather is, in the technical sense of the word, chaotic. In fact, it was in the numerical study of weather that Edward Lorenz discovered the mathematical notion of chaos in the first place. He wrote, “One meteorologist remarked that if the theory were correct, one flap of a sea gull’s wing would be enough to alter the course of the weather forever. The controversy has not yet been settled, but the most recent evidence seems to favor the sea gulls.” There’s a hard limit to how far in advance we can predict the weather, no matter how much data we collect. Lorenz thought it was about two weeks, and so far the concentrated efforts of the world’s meteorologists have given us no cause to doubt that boundary.

Then the math starts. Do the terrorists tend to make more status updates per day than the general population, or fewer, or on this metric do they look basically the same? Are there words that appear more frequently in their updates? Bands or teams or products they’re unusually prone or disinclined to like? Putting all this stuff together, you can assign to each user a score,* which represents your best estimate for the probability that the user has ties, or will have ties, to terrorist groups. It’s more or less the same thing Target does when they cross-reference your lotion and vitamin purchases to estimate how likely it is that you’re pregnant. There’s one important difference: pregnancy is very common, while terrorism is very rare. In almost all cases, the estimated probability that a given user would be a terrorist would be very small. So the result of the project wouldn’t be a Minority Report−style precrime center, where Facebook’s panoptic algorithm knows you’re going to do some crime before you do. Think of something much more modest: say, a list of a hundred thousand users about whom Facebook can say, with some degree of confidence, “People drawn from this group are about twice as likely as the typical Facebook user to be terrorists or terrorism supporters.” What would you do if you found out a guy on your block was on that list? Would you call the FBI?

The attraction of lotteries is no novelty. The practice dates back to seventeenth-century Genoa, where it seems to have evolved by accident from the electoral system. Every six months, two of the city’s governatori were drawn from the members of the Petty Council. Rather than hold an election, Genoa carried out the election by lot, drawing two slips from a pile containing the names of all 120 councilors. Before long, the city’s gamblers began to place extravagant side bets on the election outcome. The bets became so popular that gamblers started to chafe at having to wait until

Election Day for their enjoyable game of chance; and they quickly realized that if they wanted to bet on paper slips drawn from a pile, there was no need for an election at all. Numbers replaced names of politicians, and by 1700 Genoa was running a lottery that would look very familiar to modern Powerball players. Bettors tried to guess five randomly drawn numbers, with a bigger payoff the more numbers a player matched. Lotteries quickly spread throughout Europe, and from there to North America. During the Revolutionary War, both the Continental Congress and the governments of the states established lotteries to fund the fight against the British. Harvard, back in the days before it enjoyed a nine-figure endowment, ran lotteries in 1794 and 1810 to fund two new college buildings. (They’re still used as dorms for first-year students today.)

Almost 90% of the tickets for the drawing were held by Harvey’s team. They were standing in front of the money spigot, all alone. And when the drawing was over, Random Strategies had made $700,000 on their $1.4 million investment, a cool 50% profit. This trick wasn’t going to work twice. Once the lottery realized what had happened, they put an early-warning system in place to notify top management if it looked like one of the teams was trying to push the jackpot over the roll-down line single-handedly. When Random Strategies tried again in late December, the lottery was ready. On the morning of December 24, three days before the drawing, the chief of staff of the lottery got an e-mail from his team saying “Cash WinFall guys are trying to make it roll again.” If Harvey was betting on the lottery being off-duty for the holiday, he wagered wrong; early Christmas morning, the lottery updated its estimated jackpot to announce to the world that a roll-down was coming. The other cartels, still smarting from their August snookering, canceled their Christmas vacations and bought hundreds of thousands of tickets, bringing profits back down to normal levels. At any rate, the game was almost up. Sometime shortly afterward, a friend of Boston Globe reporter Andrea Estes noticed something funny in the “20-20 list” of winners that the lottery makes public: there were a lot of people in Michigan winning prizes, and they were all winning one particular game, Cash WinFall. Did Estes think t...

James Harvey wasn’t the first person to take advantage of a poorly designed state lottery. Gerald Selbee’s group made millions on Michigan’s original WinFall game before the state got wise and shut it down in 2005. And the practice goes back much further. In the early eighteenth century, France financed government spending by selling bonds, but the interest rate they offered wasn’t enticing enough to drive sales. To spice the pot, the government attached a lottery to the bond sales. Every bond gave its holder the right to buy a ticket for a lottery with a 500,000-livre prize, enough money to live on comfortably for decades. But Michel Le Peletier des Forts, the deputy finance minister who conceived the lottery plan, had botched the computations; the prizes to be disbursed substantially exceeded the money to be gained in ticket receipts. In other words, the lottery, like Cash WinFall on roll-down days, had a positive expected value for the players, and anyone who bought enough tickets was due for a big score.

One person who figured this out was the mathematician and explorer Charles-Marie de La Condamine; just as Harvey would do almost three centuries later, he gathered his friends into a ticket-buying cartel. One of these was the young writer François-Marie Arouet, better known as Voltaire. While he might not have contributed to the mathematics of the scheme, Voltaire placed his stamp on it. Lottery players were to write a motto on their ticket, to be read aloud when a ticket won the jackpot; Voltaire, characteristically, saw this as a perfect opportunity to epigrammatize, writing cheeky slogans like “All men are equal!” and “Long live M. Peletier des Forts!” on his tickets for public consumption when the cartel won the prize. Eventually, the state caught on and canceled the program, but not before La Condamine and Voltaire had taken the government for enough money to be rich men for the rest of their lives. What—you thought Voltaire made a living writing perfectly realized essays and sketches? Then, as now, that’s no way to get rich.

One of the first people to think clearly about expected value was Blaise Pascal; puzzled by some questions posed to him by the gambler Antoine Gombaud (self-styled the Chevalier de Méré), Pascal spent half of 1654 exchanging letters with Pierre de Fermat, trying to understand which bets, repeated over and over, would tend to be profitable in the long run, and which would lead to ruin. In modern terminology, he wished to understand which kinds of bets had positive expected value and which kinds were negative. The Pascal-Fermat correspondence is generally thought of as marking the beginning of probability theory.

For Ellsberg, the answer to the paradox is simply that expected utility theory is incorrect. As Donald Rumsfeld would later put it, there are known unknowns and there are unknown unknowns, and the two are to be processed differently. The “known unknowns” are like RED—we don’t know which ball we’ll get, but we can quantify the probability that the ball will be the color we want. BLACK, on the other hand, subjects the player to an “unknown unknown”—not only are we not sure whether the ball will be black, we don’t have any knowledge of how likely it is to be black. In the decision-theory literature, the former kind of unknown is called risk, the latter uncertainty. Risky strategies can be analyzed numerically; uncertain strategies, Ellsberg suggested, were beyond the bounds of formal mathematical analysis, or at least beyond the bounds of the flavor of mathematical analysis beloved at RAND.

What we’re up against here is the dreaded phenomenon known by computer-science types as “the combinatorial explosion.” Put simply: very simple operations can change manageably large numbers into absolutely impossible ones. If you want to know which of the fifty states is the most advantageous place to site your business, that’s easy; you just have to compare fifty different things. But if you want to know which route through the fifty states is the most efficient—the so-called traveling salesman problem—the combinatorial explosion goes off, and you face difficulty on a totally different scale. There are about 30 vigintillion routes to choose from. In more familiar terms, that’s 30 thousand trillion trillion trillion trillion trillion.

How exactly Florentine artists like Filippo Brunelleschi came to develop the modern theory of perspective has occasioned a hundred quarrels among art historians, into which we won’t enter here. What we know for sure is that the breakthrough joined aesthetic concerns with new ideas from mathematics and optics. A central point was the understanding that the images we see are produced by rays of light that bounce off objects and subsequently strike our eye. This sounds obvious to a modern ear, but believe me, it wasn’t obvious then. Many of the ancient scientists, most famously Plato, argued that vision must involve a kind of fire that emanated from the eye. This view goes at least as far back as Alcmaeon of Croton, one of the Pythagorean weirdos we met in chapter 2. The eye must generate light, Alcmaeon argued: what other source could there be for the phosphene, the stars you see when you shut your eyes and press down on your eyeball? The theory of vision by reflected rays was worked out in great detail by the eleventh-century Cairene mathematician Abu ‘Ali al-Hasan ibn al-Haytham (but let’s call him Alhazen, as most Western writers do). His treatise on optics, the Kitab al-Manazir, was translated into Latin and taken up eagerly by philosophers and artists

The Mars orbiter Mariner 9 sent pictures of the Martian surface back to Earth using one such code, the Hadamard code. Compact discs are encoded with the Reed-Solomon code, which is why you can scratch them and they still sound perfect. (Readers born after, say, 1990 who are unfamiliar with compact discs can just think of flash drives, which use among other things the similar Bose-Chaudhuri-Hocquenghem codes to avoid data corruption.) Your bank’s routing number is encoded using a simple code called a checksum. This one is not quite an error-correcting code, but merely an error-detecting code like the “repeat each bit twice” protocol; if you type one digit wrong, the computer executing the transfer may not be able to puzzle out what number you actually meant, but it can at least figure out something’s wrong and avoid sending your money to the wrong bank.

For a code to be an error-correcting code, no string—no point, if we’re to take this geometric analogy seriously—can be within distance 1 of two different code words; in other words, we ask that no two of the Hamming spheres centered at the code words share any points. So the problem of constructing error-correcting codes has the same structure as a classical geometric problem, that of sphere packing: how do we fit a bunch of equal-sized spheres as tightly as possible into a small space, in such a way that no two spheres overlap? More succinctly, how many oranges can you stuff into a box? The sphere-packing problem is a lot older than error-correcting codes; it goes back to the astronomer Johannes Kepler, who wrote a short booklet in 1611 called Strena Seu De Nive Sexangula, or “The Six-Cornered Snowflake.” Despite the rather specific title, Kepler’s book contemplates the general question of the origin of natural form. Why do snowflakes and the chambers of honeycombs form hexagons, while the seed chambers of an apple are more apt to come in fives? Most relevantly for us right now: why do the seeds of pomegranates tend to have twelve flat sides? Here’s Kepler’s explanation. The pomegranate wants to fit as many seeds as possible inside its skin; in other words, it is carrying out a sphere-packing problem. If we believe nature does as good a job as can be done, then these spheres ought to be arranged in the densest possible fashion. Kepler argued that the tightest possible pa...

The next layer is going to look just the same as this one, but cunningly placed so that each seed sits in the little triangular divot formed by three seeds below it. Then just keep adding more layers in the same way. It’s best to be a little careful here: only half the divots are going to support spheres in the next layer up, and at each stage you have a choice of which half of the divots you want to fill. The most customary choice, called the face-centered cubic lattice, has the nice property that every layer has the spheres placed directly over the spheres three layers below. According to Kepler, there is no denser way to pack spheres in space. And in the face-centered cubic lattice, each sphere touches exactly twelve others. As the pomegranate seeds grew, Kepler reasoned, each one would press against its twelve neighbors, flattening its surface near the point of contact and producing the twelve-sided figures he observed. Whether Kepler was right about pomegranates I have no idea,* but his claim that the face-centered cubic lattice was the densest possible sphere packing became a topic of intense mathematical interest for centuries. Kepler offered no proof of his statement; apparently it just seemed right to him that the face-centered cubic lattice couldn’t be beat. Generations of grocers, who stack oranges in a face-centered cubic configuration without any worry as to whether their method is the absolute best possible, agree

So in 1933, when Secrist was ready to reveal the results of his analysis, people in both academia and business were inclined to listen. All the more so when he revealed the striking nature of his results in a 468-page volume, thickly marbled with tables and graphs. Secrist pulled no punches: he called his book The Triumph of Mediocrity in Business. “Mediocrity tends to prevail in the conduct of competitive business,” Secrist wrote. “This is the conclusion to which this study of the costs (expenses) and profits of thousands of firms unmistakably points. Such is the price which industrial (trade) freedom brings.” How did Secrist arrive at such a doomy conclusion? First, he stratified the businesses in each sector, carefully segregating the winners (high income, low expenses) from the inefficient duds. The 120 clothing stores Secrist studied, for instance, were first ranked by ratio of sales to expenses in 1916, then divided into six groups, or “sextiles,” of twenty shops each. Secrist expected to see the shops in the top sextile consolidate their gains over time, growing ever more superior as they honed their already top-of-market skills. What he found was precisely the opposite. By 1922, the clothing stores in the highest sextile had lost most of their advantage over the typical shop; they were still better-than-average stores, but by and large they were no longer the standouts. What’s more, the lowest sextile—the worst stores—experienced the same effect in the opposite dir...

As Galton and everyone else already knew, tall parents tend to have tall children. When a six-foot-two man and a five-foot-ten woman get married, their sons and daughters are likely to be taller than average. But now here is Galton’s remarkable discovery: those children are not likely to be as tall as their parents. The same goes for short parents, in the opposite direction; their kids will tend to be short, but not as short as they themselves are. Galton had discovered the phenomenon now called regression to the mean. His data left no doubt that it was real. “However paradoxical it may appear at first sight,” Galton wrote in his 1889 book Natural Inheritance, “it is theoretically a necessary fact,* and one that is clearly confirmed by observation, that the Stature of the adult offspring must on the whole, be more mediocre than the stature of their Parents.” So, too, Galton reasoned, must it be for mental achievement. And this conforms with common experience; the children of a great composer, or scientist, or political leader, often excel in the same field, but seldom so much so as their illustrious parent. Galton was observing the same phenomenon that Secrist would uncover in the operations of business. Excellence doesn’t persist; time passes, and mediocrity asserts itself.*