Loonshots: How to Nurture the Crazy Ideas That Win Wars, Cure Diseases, and Transform Industries

by Safi Bahcall

I’ll describe the science briefly (skipping the boring stuff). And then we’ll see how small changes in structure, rather than culture, can transform the behavior of groups, the same way a small change in temperature can transform rigid ice to flowing water. Which will give all of us the tools to become the initiators, not the victims, of innovative surprise.

MORE IS DIFFERENT The pattern of sudden changes in the behavior of teams and companies—of the same people suddenly behaving in very different ways—is a mystery in business and social science. Entrepreneurs, for example, often say that big companies fail because big-corporate types are conservative and risk-averse. The most exciting ideas come from small companies, because—we tell ourselves—we are the truly passionate risk-takers. But put that big-corporate type in a startup, and the tie will come off and he’ll be pounding the table supporting some wild idea. The same person can act like a project-killing conservative in one context and a flag-waving entrepreneur in another.

The change in behavior may be a mystery in business, but a similar pattern is the essence of a strange quirk of matter called a phase transition. Imagine a large bathtub filled with water. Hit the surface with a hammer: a splash, and the hammer slips through the liquid. Then lower the temperature until the water freezes. Strike again, and the surface shatters. The same molecule behaves like a liquid in one context and a rigid solid in another. Why? How do molecules “know” to suddenly change their behavior? To put it another way, which brings us even closer to the mystery of our supposedly risk-averse, big-corporate type: If we drop a molecule of water onto a block of ice, what happens? It freezes. If we drop that same molecule into a pool of water, what happens? It slushes around with all the other molecules. How can we explain this? The physicist and Nobel laureate Phil Anderson once captured the core idea underlying the answers to these questions with the phrase more is different: “The whole becomes not only more than but very different from the sum of its parts.”

I will show you that the same holds true for teams and companies. There’s no way to analyze the behavior of any individual and explain the group. Being good at nurturing loonshots is a phase of human organization, in the same way that being liquid is a phase of matter.

All phase transitions are the result of two competing forces, like the tug-of-war between binding and entropy in water. When people organize into a team, a company, or any kind of group with a mission they also create two competing forces—two forms of incentives. We can think of the two competing incentives, loosely, as stake and rank.

When groups are small, for example, everyone’s stake in the outcome of the group project is high. At a small biotech, if the drug works, everyone will be a hero and a millionaire. If it fails, everyone will be looking for a job. The perks of rank—job titles or the increase in salary from being promoted—are small compared to those high stakes.

By the time Vannevar Bush, dean of engineering at MIT, quit his job, moved to Washington, and talked his way into a meeting with the president in the summer of 1940, the US Navy already held the key to winning that race. They’d had it for eighteen years. They just didn’t know it. To find that key and win that race, Bush invented a new system for nurturing radical breakthroughs.

In other words, Bush understood intuitively that being good at franchises and being good at loonshots are phases of organization. And the same organization can’t be in two phases at the same time, for the same reason water can’t be both solid and liquid at the same time—under ordinary conditions.

“All of [my] recent ancestors were sea captains, and they have a way of running things without any doubt,” Bush said years later. “So it may have been partly that, and partly my association with my grandfather, who was a whaling skipper, [which] left me with some inclination to run a show, once I was in it.” Bush quit his job, accepted the Carnegie offer, and moved to Washington.

LIFE AT 32 FAHRENHEIT Imagine bringing that bathtub to the brink of freezing. A little bit one way or the other and the whole thing freezes or liquefies. But right on the cusp, blocks of ice coexist with pockets of liquid. The coexistence of two phases, on the edge of a phase transition, is called phase separation. The phases break apart—but stay connected. The connection between the two phases takes the form of a balanced cycling back and forth: Molecules in ice patches melt into adjacent pools of liquid. Molecules of liquid swimming by an ice patch lock onto a surface and freeze. That cycling, in which neither phase overwhelms the other, is called dynamic equilibrium.

As we will see, phase separation and dynamic equilibrium were the key ingredients in Bush’s recipe. “The essence of a sound military organization is that it should be tight. But a tight organization does not lend itself to innovations,” Bush wrote. “And loosening it in time of war … would be fraught with danger.” But, Bush continued, there “should be close collaboration between the military and [some] organization, made loose in its structure on purpose.” Life on the edge In other words, the two phases must break apart while staying connected.

Bush moved quickly. By the end of 1940, six months after his meeting with the president, the OSRD had 126 research contracts in place with 19 industrial labs and 32 academic institutions.

Bush’s report, called Science: The Endless Frontier, presented to President Truman in June 1945, two months after FDR’s death, and released the following month, caused a sensation. The country had no national science policy, he declared. Philanthropy and private industry could not be relied upon to fund the basic research that is “the pacemaker of technological progress,” essential for national security, economic growth, and the fight against disease. The report outlined the architecture of a new national research system.

In 1907, a banking group led by the financier J. P. Morgan took control of the company, by then renamed AT&T, and got rid of its management. Morgan installed Theodore Vail, age 62, as its new chief executive.

Vail’s goal required technologies that did not yet exist, based on science that was not yet known. Vail persuaded his new board of directors that to solve these problems, the company should create a quarantined group working on “fundamental” research. Like Bush, he understood the need for separating and sheltering radical ideas—the need for a department of loonshots run by loons, free to explore the bizarre.

Vail put a physicist from MIT, Frank Jewett, in charge. Over the next several years, Jewett’s group worked through the science and eventually solved the problem of the fading signals. They invented the vacuum tube: the world’s first amplifier, the forerunner of all modern electronics.

Over the next 50 years, Vail’s organization—eventually called the Bell Telephone Laboratories—produced the transistor, the solar cell, the CCD chip (used inside every digital camera), the first continuously operating laser, the Unix operating system, the C programming language, and eight Nobel Prizes. Vail created the most successful industrial research lab in history, and AT&T grew into the country’s largest business.

Many of the principles that Bush applied during the war had first been applied by Vail and Jewett at Bell Labs.

The magic of Bush and Vail was in engineering the forces of genius and serendipity to work for them rather than against them. Luck is the residue of design.

Vannevar Bush, James Conant, Karl Compton, and Alfred Loomis at UC Berkeley (1940)

But the ones who truly succeed—the engineers of serendipity—play a more humble role. Rather than champion any individual loonshot, they create an outstanding structure for nurturing many loonshots. Rather than visionary innovators, they are careful gardeners.

The structures that these gardeners create share a common set of principles. I’ll call these principles the Bush-Vail rules. The first two rules are the ones mentioned above, the key to life at 32 Fahrenheit: separate the phases (the groups working on loonshots and on franchises) and create dynamic equilibrium (ensure that projects and feedback travel easily between the two groups). Break apart while staying connected.

SEPARATE THE PHASES Separate your artists and soldiers People responsible for developing high-risk, early-stage ideas (call them “artists”) need to be sheltered from the “soldiers” responsible for the already-successful, steady-growth part of an organization. Early-stage projects are fragile. “Although military officers became avid for a new development once it had thoroughly proved itself in the field,” Bush wrote, they dismissed any weapon “in embryo”—as they did with radar, with the DUKW truck, and with nearly every early innovation, which almost always arrives covered in warts. Without a strong cocoon to protect those early-stage ideas, they will be shut down or buried, like Young and Taylor’s early discovery of radar.

Tailor the tools to the phase Just separating loonshot and franchise groups is not enough.

Bush quarantined the team working on radar in anonymous office buildings at MIT. He recognized that the tight organization needed by the military, mentioned earlier, is not conducive to scientists exploring the bizarre, just as “a good organization for a research laboratory would not work well for a combat regiment in the field.”

DYNAMIC EQUILIBRIUM Love your artists and soldiers equally Maintaining balance so that neither phase overwhelms the other requires something that sounds soft and fuzzy but is very real and often overlooked. Artists working on loonshots and soldiers working on franchises have to feel equally loved.

The trap for most groups, however, is that soldiers naturally favor soldiers and artists naturally favor artists.

Equal-opportunity respect is a rare and valuable skill. Vannevar Bush, although a veteran academic at the start of the war, genuinely respected the military. “I have enjoyed associating with military men more than with any other group, scientists, businessmen, professors,” he wrote years later. The deference with which Bush treated officers helped him understand, and ultimately influence, the military far more than the many scientists and engineers who had tried, and failed, before him.

When Jobs returned twelve years later, he had learned to love his artists (Jony Ive) and soldiers (Tim Cook) equally.

Manage the transfer, not the technology Bush, although a brilliant inventor and engineer, pointedly stayed out of the details of any one loonshot. “I made no technical contribution whatever to the war effort,” he wrote. “Not a single technical idea of mine ever amounted to shucks.

Vail similarly stayed out of the details of the technical program. Both Bush and Vail saw their jobs as managing the touch and the balance between loonshots and franchises—between scientists exploring the bizarre and soldiers assembling munitions; between the blue-sky research of Bell Labs and the daily grind of telephone operations. Rather than dive deep into one or the other, they focused on the transfer between the two.

When the balance broke down, they intervened.

Bush and Vail zeroed in on that link. A radar detection device buried in a building full of physicists would sink no U-boats.

A flawed transfer from inventors to the field is not the only danger. Transfer in the other direction is equally important. No product works perfectly the first time. If feedback from the field is ignored by inventors, initial enthusiasm can rapidly fade, and a promising program will be dropped. Early aircraft radar, for example, was practically useless; pilots ignored it. Bush made sure that pilots went back to the scientists and explained why they weren’t using it. The reason had nothing to do with the technology: pilots in the heat of battle didn’t have time to fiddle with the complicated switches on the early radar boxes. The user interface was lousy. Scientists quickly created a custom display technology—the sweeping line and moving dots now called a PPI display. Pilots started using radar.

In some cases, as with the radar-controlled fuse on artillery shells mentioned earlier, Bush acted alone when he sensed a weak link. The Army initially paid little attention to the fuse, so Bush got on a plane and flew straight to battlefield headquarters in Europe. He was received by General Walter Bedell Smith, Eisenhower’s chief of staff.

Key to that dynamic equilibrium—and Bush’s ability to speak freely to generals—was support from the top. In the middle of managing a difficult conflict, Bush wrote, “I told FDR that he had handed me a hot potato, and I might have to bump some heads together. I remember well his answer. He said, ‘You go ahead and bump, and I will back you up.’”

Not long after, one of the bumped heads came to FDR and launched into a tirade about Bush and his operation. The president, according to an aide who was present, was in the middle of signing letters. FDR paused for a while to listen, went back to signing letters, then said, “Look, Mac, I put that in Bush’s hands. He’s running it, and you get the hell out of here.”

Many companies, however, especially when faced with a crisis, try to legislate creativity and innovation everywhere (“The CEO must be the CIO—the Chief Innovation Officer!”). This usually results in chaos, the top-left quadrant.

The most common trap, however, is to head straight to the bottom-right quadrant. As mentioned earlier, leaders proudly draw a box on an org chart, rent a new building, and hang a shingle advertising a new research lab.

In the real world, ideas are ridiculed, experiments fail, budgets are cut, and good people are fired for stupid reasons. Companies fall apart and their best projects remain buried, sometimes forever. The Three Deaths tells the honest history, as opposed to the revisionist history, of nearly every important breakthrough I’m aware of or have personally experienced (the Three often stretches to Four, Five, or Ten). The need to nurture and protect fragile loonshots so they can survive those stumbles and setbacks, whether self-inflicted or caused by others, is the central idea behind the systems of Bush and Vail.

Ancel Keys, a scientist at the University of Minnesota. His famous study of 10,000 people across seven countries confirmed that elevated blood cholesterol correlated with heart disease. But Keys went further, and implicated diet. Consuming fat, specifically saturated fat, he said, was the problem. Keys was not one for nuance. Obesity was “disgusting,” he said. “Maybe if the idea got around again that obesity is immoral, the fat man would start to think.” Keys’s advocacy eventually led to official guidelines recommending low-fat, high-carbohydrate diets, despite the lack of any more rigorous evidence (it would be six decades before official guidelines backed away from that diet, now recognized not to be a good idea).

In New York, Endo made the connection firsthand: He was surprised both by the high incidence of heart disease, and the rich American diet (“I saw many overweight people, like sumo wrestlers”). He concluded, like Keys, that as Japan became more Westernized, heart disease would become more common. He returned to Japan determined to find a drug that lowered cholesterol.

Since fungi are such great chemists, Endo reasoned, that’s where he would begin his search. Bacteria, Endo knew, are natural predators of molds and mushrooms. To defend themselves, fungi have evolved many ways to kill bacteria. Penicillium notatum, for example, kills bacteria by secreting a compound that causes bacterial cell walls to collapse. That’s how its extract, penicillin, works.

Brown and Goldstein dedicated a recent historical review “to Akira Endo, discoverer of a ‘penicillin’ for cholesterol,” and concluded, “The millions of people whose lives will be extended through statin therapy owe it all to Akira Endo and his search through fungal extracts at the Sankyo Co.” Endo’s story is more than a wild anecdote. The twisted paths leading to great discoveries are the rule rather than the exception. And so are their revisionist histories: victors don’t just write history; they rewrite history.

At one point, Folkman discussed with his wife, Paula, whether he should quit research, close the lab, and return full-time to surgery. With Paula’s encouragement, which he later called “Spouse Activation Factor” (SAF), Folkman did the opposite. He quit clinical work and began full-time research. He recruited a handful of star students, overcoming warnings they had heard to stay away from Folkman and his work “by reminding them that they were so good that even if things didn’t work out and they left after a year, their careers wouldn’t be harmed,” he recalled years later. Folkman joined them in the lab, working nights and weekends.

Later, Folkman would say, “You can tell a leader by counting the number of arrows in his ass.”

LESSONS FROM THE SURPRISING FRAGILITY Beware the False Fail The Endo and Folkman stories illustrate not only the Three Deaths but also a specific type of death, one common to loonshots. The failure of Endo’s drug in rat models (Death #2), for example, nearly terminated his program at Sankyo. The same failure permanently killed a similar program at another company, Beecham Pharmaceuticals. Beecham later merged with SmithKline & French, and then Glaxo Wellcome, becoming today’s GlaxoSmithKline. Had Beecham persisted, they might have shared in the $300 billion of revenues from statins. Even a small piece of $300 billion is pretty good. But they gave up and ended up with nothing. The negative result in the rat experiment was a False Fail—a result mistakenly attributed to the loonshot but actually a flaw in the test. Sankyo persisted through that fail, because of Endo.

We will see the False Fail over and over, both in science and in business. There are many reasons projects can die: funding dwindles, a competitor wins, the market changes, a key person leaves. But the False Fail is common to loonshots. Risks of this type of death can never be fully eliminated—negative results don’t come with a neon sign that lights up “your idea is flawed” or “your test is flawed.” But those risks can be reduced, which is exactly what Endo and Folkman did, as we will discuss below. People may think of Endo and Folkman as great inventors, but arguably their greatest skill was investigating failure. They learned to separate False Fails from true fails.

Peter Thiel and Ken Howery at Founders Fund, however, reached out to their friends behind the scenes at Friendster. They dug into why users were leaving the site. Like other users, Thiel and Howery knew that Friendster crashed often. They also knew that the team behind Friendster had received, and ignored, crucial advice on how to scale their site—how to transform a system built for a few thousand users into one that could support millions of users. They asked for and received a copy of Friendster’s data on user retention. They were stunned by how long users stayed with the site, despite the irritating crashes.