The Coming Wave: AI, Power, and Our Future

by Mustafa Suleyman

Question: What does the coming wave of technology mean for humanity? In the annals of human history, there are moments that stand out as turning points, where the fate of humanity hangs in the balance. The discovery of fire, the invention of the wheel, the harnessing of electricity—all of these were moments that transformed human civilization, altering the course of history forever. And now we stand at the brink of another such moment as we face the rise of a coming wave of technology that includes both advanced AI and biotechnology. Never before have we witnessed technologies with such transformative potential, promising to reshape our world in ways that are both awe-inspiring and daunting. On the one hand, the potential benefits of these technologies are vast and profound. With AI, we could unlock the secrets of the universe, cure diseases that have long eluded us, and create new forms of art and culture that stretch the bounds of imagination. With biotechnology, we could engineer life to tackle diseases and transform agriculture, creating a world that is healthier and more sustainable. But on the other hand, the potential dangers of these technologies are equally vast and profound. With AI, we could create systems that are beyond our control and find ourselves at the mercy of algorithms that we don’t understand. With biotechnology, we could manipulate the very building blocks of life, potentially creating unintended consequences for both individuals and entire ecosystem...

Between Miracle and Misstep: Humanity at the AI–Biotech Threshold

A single overriding trend has stood the test of time since the discovery of fire and stone tools, the first technologies harnessed by our species. Almost every foundational technology ever invented, from pickaxes to plows, pottery to photography, phones to planes, and everything in between, follows a single, seemingly immutable law: it gets cheaper and easier to use, and ultimately it proliferates, far and wide. This proliferation of technology in waves is the story of Homo technologicus—of the technological animal. Humanity’s quest to improve—ourselves, our lot, our abilities, and our influence over our environment—has powered a relentless evolution of ideas and creation. Invention is an unfolding, sprawling, emergent process driven by self-organizing and highly competitive inventors, academics, entrepreneurs, and leaders, each surging forward with their own motivations. This ecosystem of invention defaults to expansion. It is the inherent nature of technology.

Cheaper, Easier, Everywhere: The Proliferation Law of Technology

Language—the foundation of our social interactions, of our cultures, of our political organizations, and perhaps of what it means to be human—is another product, and driver, of our intelligence. Every principle and abstract concept, every small creative endeavor or project, every encounter in your life, has been mediated by our species’ unique and endlessly complex capacity for imagination, creativity, and reason. Human ingenuity is an astonishing thing. Only one other force is so omnipresent in this picture: biological life itself. Before the modern age, aside from a few rocks and minerals, most human artifacts—from wooden houses to cotton clothes to coal fires—came from things that were once alive. Everything that has entered the world since then flows from us, flows from the fact that we are biological beings. It’s no exaggeration to say the entirety of the human world depends on either living systems or our intelligence. And yet both are now in an unprecedented moment of exponential innovation and upheaval, an unparalleled augmentation that will leave little unchanged. Starting to crash around us is a new wave of technology. This wave is unleashing the power to engineer these two universal foundations: a wave of nothing less than intelligence and life. The coming wave is defined by two core technologies: artificial intelligence (AI) and synthetic biology. Together they will usher in a new dawn for humanity, creating wealth and surplus unlike anything ever seen. And yet...

Rewriting Life, Recasting Mind: The Twin Revolutions of AI and Synthetic Biology

AI has been climbing the ladder of cognitive abilities for decades, and it now looks set to reach human-level performance across a very wide range of tasks within the next three years.

T-Minus Three: AI’s Sprint to Parity

AI is everywhere, on the news and in your smartphone, trading stocks and building websites. Many of the world’s largest companies and wealthiest nations barrel forward, developing cutting-edge AI models and genetic engineering techniques, fueled by tens of billions of dollars in investment. Once matured, these emerging technologies will spread rapidly, becoming cheaper, more accessible, and widely diffused throughout society. They will offer extraordinary new medical advances and clean energy breakthroughs, creating not just new businesses but new industries and quality of life improvements in almost every imaginable area. And yet alongside these benefits, AI, synthetic biology, and other advanced forms of technology produce tail risks on a deeply concerning scale. They could present an existential threat to nation-states—risks so profound they might disrupt or even overturn the current geopolitical order. They open pathways to immense AI-empowered cyberattacks, automated wars that could devastate countries, engineered pandemics, and a world subject to unexplainable and yet seemingly omnipotent forces. The likelihood of each may be small, but the possible consequences are huge. Even a slim chance of outcomes like these requires urgent attention. Some countries will react to the possibility of such catastrophic risks with a form of technologically charged authoritarianism to slow the spread of these new powers. This will require huge levels of surveillance along with massiv...

The Narrow Path: Steering Between Catastrophe and Panopticon

pessimism-aversion trap: the misguided analysis that arises when you are overwhelmed by a fear of confronting potentially dark realities, and the resulting tendency to look the other way.

Comfortable Blindness: The Pessimism-Aversion Trap

Pessimism aversion is an emotional response, an ingrained gut refusal to accept the possibility of seriously destabilizing outcomes. It tends to come from those in secure and powerful positions with entrenched worldviews, people who can superficially cope with change but struggle to accept any real challenge to their world order. Many of those whom I accuse of being stuck in the pessimism-aversion trap fully embrace the growing critiques of technology. But they nod along without actually taking any action. We’ll manage, we always do, they say. Spend time in tech or policy circles, and it quickly becomes obvious that head-in-the-sand is the default ideology. To believe and act otherwise risks becoming so crippled by fear of and outrage against enormous, inexorable forces that everything feels futile. So the strange intellectual half-world of pessimism aversion rumbles on.

The Nodding Class: How Pessimism Aversion Blocks Action

we look at the long history of technology and how it spreads—waves building over millennia. What drives them? What makes them truly general? We also ask if there are examples of societies consciously saying no to a new technology. Instead of turning away from technologies, the past is marked by a pronounced pattern of proliferation, resulting in sprawling chains of both intended and unintended consequences. I call this “the containment problem.” How do we keep a grip on the most valuable technologies ever invented as they get cheaper and spread faster than any in history?

Holding Back the Wave: The Containment Problem of General Technologies

Part 2 gets into the details of the coming wave itself. At its heart lie two general-purpose technologies of immense promise, power, and peril: artificial intelligence and synthetic biology. Both have been long heralded, and yet, if anything, I believe the scope of their impact is still often understated. Around them grow a host of associated technologies like robotics and quantum computing whose development will intersect in complex and turbulent ways. In this section, we look at not only how they all emerged and what they can do but also why they are so hard to contain. The various technologies I’m speaking of share four key features that explain why this isn’t business as usual: they are inherently general and therefore omni-use, they hyper-evolve, they have asymmetric impacts, and, in some respects, they are increasingly autonomous. Their creation is driven by powerful incentives: geopolitical competition, massive financial rewards, and an open, distributed culture of research. Scores of state and non-state actors will race ahead to develop them regardless of efforts to regulate and control what’s coming, taking risks that affect everyone, whether we like it or not.

Uncontainable by Design: AI & SynBio’s General, Hyper, Asymmetric, Autonomous Surge

Technologies can fail in the mundane sense of not working: the engine doesn’t start; the bridge falls down. But they can also fail in a wider sense. If technology damages human lives, or produces societies filled with harm, or renders them ungovernable because we empower a chaotic long tail of bad (or unintentionally dangerous) actors—if, in the aggregate, technology is damaging—then it can be said to have failed in another, deeper sense, failing to live up to its promise. Failure in this sense isn’t intrinsic to technology; it is about the context within which it operates, the governance structures it is subject to, the networks of power and uses to which it is put. That impressive ingenuity giving rise to so much now means we are better at avoiding the first kind of failure. Fewer planes crash, cars are cleaner and safer, computers are more powerful and yet more secure. Our great challenge is that we still haven’t reckoned with the latter mode of failure.

Beyond Breakdowns: Technology’s Social Failure Mode

Technology has a clear, inevitable trajectory: mass diffusion in great roiling waves. This is true from the earliest flint and bone tools to the latest AI models. As science produces new discoveries, people apply these insights to make cheaper food, better goods, and more efficient transport. Over time demand for the best new products and services grows, driving competition to produce cheaper versions bursting with yet more features. This in turn drives yet more demand for the technologies that create them, and they also become easier and cheaper to use. Costs continue to fall. Capabilities rise. Experiment, repeat, use. Grow, improve, adapt. This is the inescapable evolutionary nature of technology.

Waves That Never Recede: Technology’s Evolutionary Trajectory

Stonework and fire were proto-general-purpose technologies, meaning they were pervasive, in turn enabling new inventions, goods, and organizational behaviors. General-purpose technologies ripple out over societies, across geographies, and throughout history. They open the doors of invention wide, enabling scores of downstream tools and processes. They are often built on some kind of general-purpose principle, whether the power of steam to do work or the information theory behind a computer’s binary code. The irony of general-purpose technologies is that, before long, they become invisible and we take them for granted. Language, agriculture, writing—each was a general-purpose technology at the center of an early wave. These three waves formed the foundation of civilization as we know it. Now we take them for granted. One major study pegged the number of general-purpose technologies that have emerged over the entire span of human history at just twenty-four, naming inventions ranging from farming, the factory system, the development of materials like iron and bronze, through to printing presses, electricity, and of course the internet. There aren’t many of them, but they matter; it’s why in the popular imagination we still use terms like the Bronze Age and the Age of Sail.

Invisible Engines of History: The Ripple Power of General-Purpose Technologies

The more tools you have, the more you can do and the more you can imagine new tools and processes beyond them. As the Harvard anthropologist Joseph Henrich points out, the wheel arrived surprisingly late in human life. But once invented, it became a building block of everything from chariots and wagons to mills, presses, and flywheels. From the written word to sailing vessels, technology increases interconnectedness, helping to boost its own flow and spread. Each wave hence lays the groundwork for successive waves. Over time, this dynamic accelerated. Beginning around the 1770s in Europe, the first wave of the Industrial Revolution combined steam power, mechanized looms, the factory system, and canals. In the 1840s came the age of railways, telegraphs, and steamships, and a bit later steel and machine tools; together they formed the First Industrial Revolution. Then, just a few decades later, came the Second Industrial Revolution. You’ll be familiar with its greatest hits: the internal combustion engine, chemical engineering, powered flight, and electricity. Flight needed combustion, and mass production of combustion engines demanded steel and machine tools, and so on. Beginning with the Industrial Revolution, immense change became measured in decades rather than centuries or millennia. This isn’t, however, an orderly process. Technological waves don’t arrive with the neat predictability of the tides. Over the long term, waves erratically intersect and intensify. The ten t...

Interlocking Waves: How Tools Beget Tools and Collapse Time

Proliferation is catalyzed by two forces: demand and the resulting cost decreases, each of which drives technology to become even better and cheaper. The long and intricate dialogue of science and technology produces a chain of insights, breakthroughs, and tools that build and reinforce over time, productive recombinations that drive the future. As you get more and cheaper technology, it enables new and cheaper technologies downstream. Uber was impossible without the smartphone, which itself was enabled by GPS, which was enabled by satellites, which were enabled by rockets, which were enabled by combustion techniques, which were enabled by language and fire.

The Demand–Cost Flywheel: How Breakthroughs Beget Breakthroughs

In the early 1970s there were about half a million computers. Back in 1983, only 562 computers total were connected to the primordial internet. Now the number of computers, smartphones, and connected devices is estimated at 14 billion. It took smartphones a few years to go from niche product to utterly essential item for two-thirds of the planet. With this wave came email, social media, online videos—each a fundamentally new experience enabled by the transistor and another general-purpose technology, the internet. This is what pure, uncontained technological proliferation looks like. It created a yet more mind-boggling proliferation: data, up twenty times in the decade 2010–2020 alone. Just a few decades ago data storage was the domain of books and dusty archives. Now humans produce hundreds of billions of emails, messages, images, and videos daily and store them in the cloud. Eighteen million gigabytes of data are added to the global sum every single minute of every day. Billions of hours of raw human life are consumed, shaped, distorted, and enriched by these technologies. They dominate our businesses and our leisure time. They occupy our minds and every crevice of our worlds, from fridges, timers, garage doors, and hearing aids to wind turbines. They form the very architecture of modern life. Our phones are the first thing we see in the morning and the last at night. Every aspect of human life is affected: they help us find love and new friends while turbocharging suppl...

From Niche to Nervous System: Computing’s Total Envelopment

Alan Turing and Gordon Moore could never have predicted, let alone altered the rise of, social media, memes, Wikipedia, or cyberattacks. Decades after their invention, the architects of the atomic bomb could no more stop a nuclear war than Henry Ford could stop a car accident. Technology’s unavoidable challenge is that its makers quickly lose control over the path their inventions take once introduced to the world. Technology exists in a complex, dynamic system (the real world), where second-, third-, and nth-order consequences ripple out unpredictably. What on paper looks flawless can behave differently out in the wild, especially when copied and further adapted downstream. What people actually do with your invention, however well intentioned, can never be guaranteed. Thomas Edison invented the phonograph so people could record their thoughts for posterity and to help the blind. He was horrified when most people just wanted to play music. Alfred Nobel intended his explosives to be used only in mining and railway construction. Gutenberg just wanted to make money printing Bibles. Yet his press catalyzed the Scientific Revolution and the Reformation, and so became the greatest threat to the Catholic Church since its establishment. Fridge makers didn’t aim to create a hole in the ozone layer with chlorofluorocarbons (CFCs), just as the creators of the internal combustion and jet engines had no thought of melting the ice caps. In fact early enthusiasts for automobiles argued f...

The Unruly Afterlife of Inventions

As technology proliferates, more people can use it, adapt it, shape it however they like, in chains of causality beyond any individual’s comprehension. As the power of our tools grows exponentially and as access to them rapidly increases, so do the potential harms, an unfolding labyrinth of consequences that no one can fully predict or forestall. One day someone is writing equations on a blackboard or fiddling with a prototype in the garage, work seemingly irrelevant to the wider world. Within decades, it has produced existential questions for humanity. As we have built systems of increasing power, this aspect of technology has felt more and more pressing to me. How do we guarantee that this new wave of technologies does more good than harm? Technology’s problem here is a containment problem. If this aspect cannot be eliminated, it might be curtailed. Containment is the overarching ability to control, limit, and, if need be, close down technologies at any stage of their development or deployment. It means, in some circumstances, the ability to stop a technology from proliferating in the first place, checking the ripple of unintended consequences (both good and bad). The more powerful a technology, the more ingrained it is in every facet of life and society. Thus, technology’s problems have a tendency to escalate in parallel with its capabilities, and so the need for containment grows more acute over time. Does any of this get technologists off the hook? Not at all; more th...

Rudder Against the Tide: Containment and Responsibility in Runaway Tech

Technology is not an adversary; it’s a basic property of human society. Containing technology needs to be a much more fundamental program, a balance of power not between competing actors but between humans and our tools. It’s a necessary prerequisite for the survival of our species over the next century. Containment encompasses regulation, better technical safety, new governance and ownership models, and new modes of accountability and transparency, all as necessary (but not sufficient) precursors to safer technology. It’s an overarching lock uniting cutting-edge engineering, ethical values, and government regulation. Containment shouldn’t be seen as the final answer to all technology’s problems; it is rather the first, critical step, a foundation on which the future is built. Think of containment, then, as a set of interlinked and mutually reinforcing technical, cultural, legal, and political mechanisms for maintaining societal control of technology during a time of exponential change; an architecture up to the task of containing what would have once been centuries or millennia of technological change happening now in a matter of years or even months, where consequences ricochet around the world in seconds. Technical containment refers to what happens in a lab or an R&D facility. In AI, for example, it means air gaps, sandboxes, simulations, off switches, hard built-in safety and security measures—protocols for verifying the safety or integrity or uncompromised nature of ...

Humanity’s Circuit Breaker: Technical, Cultural, and Legal Containment

In hindsight, waves might appear smooth and inevitable. But there is an almost infinite array of small, local, and often arbitrary factors that affect a technology’s trajectory. Indeed, no one should imagine diffusion is easy. It can be costly, slow, and risky, or require wrenching changes in behavior feasible over only decades or lifetimes. It has to fight existing interests, established knowledge, and those who jealously hold both. Fear and suspicion of anything new and different are endemic. Everyone from guilds of skilled craftsmen to suspicious monarchs has reason to push back. Luddites, the groups that violently rejected industrial techniques, are not the exception to the arrival of new technologies; they are the norm.

Inevitable, Not Easy: Technology’s Rugged Diffusion

People throughout history have attempted to resist new technologies because they felt threatened and worried their livelihoods and way of life would be destroyed. Fighting, as they saw it, for the future of their families, they would, if necessary, physically destroy what was coming. If peaceful measures failed, Luddites wanted to take apart the wave of industrial machinery.

Breaking the Looms: Luddism’s Desperate Stand

Where there is demand, technology always breaks out, finds traction, builds users. Once established, waves are almost impossible to stop. As the Ottomans discovered when it came to printing, resistance tends to be ground down with the passage of time. Technology’s nature is to spread, no matter the barriers. Plenty of technologies come and go. You don’t see too many penny-farthings or Segways, listen to many cassettes or minidiscs. But that doesn’t mean personal mobility and music aren’t ubiquitous; older technologies have just been replaced by new, more efficient forms. We don’t ride on steam trains or write on typewriters, but their ghostly presence lives on in their successors, like Shinkansens and MacBooks. Think of how, as parts of successive waves, fire, then candles and oil lamps, gave way to gas lamps and then to electric lightbulbs, and now LED lights, and the totality of artificial light increased even as the underlying technologies changed. New technologies supersede multiple predecessors. Just as electricity did the work of candles and steam engines alike, so smartphones replaced satnavs, cameras, PDAs, computers, and telephones (and invented entirely new classes of experience: apps). As technologies let you do more, for less, their appeal only grows, along with their adoption.

Resistance Wanes, Replacement Reigns: Why Waves Keep Spreading

On July 16, 1945, under the auspices of the Manhattan Project, the U.S. Army detonated a device code-named Trinity in the New Mexico desert. Weeks later a Boeing B-29 Superfortress, the Enola Gay, dropped a device code-named Little Boy containing sixty-four kilograms of uranium-235 over the city of Hiroshima, killing 140,000 people. In an instant, the world had changed. Yet from there, against the wider pattern of history, nuclear weapons did not endlessly proliferate. Nuclear weapons have been detonated only twice in wartime. To date only nine countries have acquired them. Indeed, South Africa relinquished the technology altogether in 1989. As far as we know, no non-state actors have acquired nuclear weapons, and today the total number of warheads stands at around ten thousand, frighteningly large, but lower than Cold War highs, when that figure hovered at more than sixty thousand. So what happened? Nuclear weapons clearly confer a significant strategic advantage. At the end of World War II many unsurprisingly assumed they would proliferate widely. After the successful development of early nuclear bombs, the United States and Russia had been on a path of developing ever more destructive weapons, like thermonuclear hydrogen bombs. The biggest explosion ever recorded was a test of an H-bomb called the Tsar Bomba. Detonated over a remote archipelago in the Barents Sea in 1961, the explosion created a three-mile fireball and a mushroom cloud fifty-nine miles wide. The blast w...

The Exception That Held: Nuclear Nonproliferation’s Uneasy Success

A turning point came in 1968 with the Treaty on the Non-proliferation of Nuclear Weapons, a landmark moment when nations explicitly agreed never to develop nuclear weapons. The world had come together to decisively arrest the proliferation of nuclear weapons to new states. From the first test, their destructive power was clear. Popular revulsion at the possibility of a thermonuclear apocalypse was a powerful motivator for signing the treaty. But these weapons have also been contained by cold calculation. Mutually assured destruction hemmed in possessors since it soon became clear that using them in anger is a quick way of ensuring your own destruction. They’re also eye-wateringly expensive and difficult to manufacture. Not only do they require rare and difficult-to-handle materials like enriched uranium-235, but maintaining and ultimately decommissioning them is also challenging. Lack of widespread demand has meant little pressure to reduce costs and grow access; they are not subject to the classic cost curves of modern consumer technology. These were never going to spread like transistors or flat-screen TVs; producing fissile material is not like rolling aluminum. Nonproliferation is in no small part a function of the fact that building a nuke is one of the largest, most expensive, and most complicated endeavors a state can embark on.

MAD, Scarcity, and the NPT: Why Nuclear Tech Bucked the Cost Curve

There are always good reasons to resist or curtail technology. Although its history is one of enabling people to do more, increasing capabilities, driving improvements in well-being, it’s not a one-sided story: Technology creates more lethal and destructive weapons as well as better tools. It produces losers, eliminates some jobs and ways of life, and creates harm up to the planetary, existential scale of climate change. New technologies can be unsettling and destabilizing, alien and invasive. Technology causes problems, and always has. And yet none of that seems to matter. It might take time, but the pattern is unmistakable: proliferating, cheaper, and more efficient technologies, wave upon wave of them. As long as a technology is useful, desirable, affordable, accessible, and unsurpassed, it survives and spreads and those features compound.

The Gravity of Utility: Why Useful Tech Prevails

In the space of around a hundred years, successive waves took humanity from an era of candles and horse carts to one of power stations and space stations. Something similar is going to occur in the next thirty years. In the coming decades, a new wave of technology will force us to confront the most foundational questions our species has ever faced. Do we want to edit our genomes so that some of us can have children with immunity to certain diseases, or with more intelligence, or with the potential to live longer? Are we committed to holding on to our place at the top of the evolutionary pyramid, or will we allow the emergence of AI systems that are smarter and more capable than we can ever be? What are the unintended consequences of exploring questions like these?

From Candles to CRISPR: The Thirty-Year Reckoning

Thanks to deep learning, computer vision is now everywhere, working so well it can classify dynamic real-world street scenes with visual input equivalent to twenty-one full-HD screens, or about 2.5 billion pixels per second, accurately enough to weave an SUV through busy city streets. Your smartphone recognizes objects and scenes, while vision systems automatically blur the background and highlight people in your videoconference calls. Computer vision is the basis of Amazon’s checkout-less supermarkets and is present in Tesla’s cars, pushing them toward increasing autonomy. It helps the visually impaired navigate cities, guides robots in factories, and powers the facial recognition systems that increasingly monitor urban life from Baltimore to Beijing. It’s in the sensors and cameras on your Xbox, your connected doorbell, and the scanner at the airport gate. It helps fly drones, flags inappropriate content on Facebook, and diagnoses a growing list of medical conditions: at DeepMind, one system my team developed read eye scans as accurately as world-leading expert doctors.

Eyes Everywhere: Machine Vision’s Quiet Conquest

A big part of what makes humans intelligent is that we look at the past to predict what might happen in the future. In this sense intelligence can be understood as the ability to generate a range of plausible scenarios about how the world around you may unfold and then base sensible actions on those predictions.

Past as Prologue: Intelligence as Scenario-Making

When GPT-4 launched in March 2023, results were again impressive. As with its predecessors, you can ask GPT-4 to compose poetry in the style of Emily Dickinson and it obliges; ask it to pick up from a random snippet of The Lord of the Rings and you are suddenly reading a plausible imitation of Tolkien; request start-up business plans and the output is akin to having a roomful of executives on call. Moreover, it can ace standardized tests from the bar exam to the GRE. It can also work with images and code, create 3-D computer games that run in desktop browsers, build smartphone apps, debug your code, identify weaknesses in contracts, and suggest compounds for novel drugs, even offering ways of modifying them so they are not patented. It will produce websites from hand-drawn images and understand the subtle human dynamics in complex scenes; show it a fridge and it will come up with recipes based on what’s in it; write a rough presentation and it will polish and design a professional-looking version. It appears to “understand” spatial and causal reasoning, medicine, law, and human psychology. Within days of its release people had built tools that automated lawsuits, helped co-parent children, and offered real-time fashion advice. Within weeks they’d created add-ons so that GPT-4 could accomplish complex tasks like creating mobile apps or researching and writing detailed market reports. All of this is just the start. We are only beginning to scratch at the profound impact larg...

From Prompt to Platform: LLMs Become the New Internet

learn from carefully hand-labeled data. Quite often the quality of the AI’s predictions depends on the quality of the labels in the training data. However, a key ingredient of the LLM revolution is that for the first time very large models could be trained directly on raw, messy, real-world data, without the need for carefully curated and human-labeled data sets. As a result almost all textual data on the web became useful. The more the better. Today’s LLMs are trained on trillions of words. Imagine digesting Wikipedia wholesale, consuming all the subtitles and comments on YouTube, reading millions of legal contracts, tens of millions of emails, and hundreds of thousands of books. This kind of vast, almost instantaneous consumption of information is not just difficult to comprehend; it’s truly alien. Pause here for a moment. Consider the unfathomable number of words that these models consume during training. If we assume that the average person can read about two hundred words per minute, in an eighty-year lifetime that would be about eight billion words, assuming they did absolutely nothing else twenty-four hours per day. More realistically, the average American reads a book for about fifteen minutes per day, which over the year amounts to reading about a million words. That’s roughly six orders of magnitude less than what these models consume in a single monthlong training run.

The Unlabeled Feast: How LLMs Devour the Internet

Researchers meanwhile see more and more evidence for “the scaling hypothesis,” which predicts that the main driver of performance is, quite simply, to go big and keep going bigger. Keep growing these models with more data, more parameters, more computation, and they’ll keep improving—potentially all the way to human-level intelligence and beyond. No one can say for sure whether this hypothesis will hold, but so far at least it has. I think that looks set to continue for the foreseeable future. Our brains are terrible at making sense of the rapid scaling of an exponential, and so in a field like AI it’s not always easy to grasp what is actually happening. It’s inevitable that in the next years and decades many orders of magnitude more compute will be used to train the largest AI models, and so, if the scaling hypothesis is at least partially true, there is an inevitability about what this means. Sometimes people seem to suggest that in aiming to replicate human-level intelligence, AI chases a moving target or that there is always some ineffable component forever out of reach. That’s just not the case. The human brain is said to contain around 100 billion neurons with 100 trillion connections between them—it is often said to be the most complex known object in the universe. It’s true that we are, more widely, complex emotional and social beings. But humans’ ability to complete given tasks—human intelligence itself—is very much a fixed target, as large and multifaceted as it ...

Exponentials vs. the Brain: Scaling Past Human Level

It wasn’t until the autumn of 2019 that I started paying attention to GPT-2. I was impressed. This was the first time I had encountered evidence that language modeling was making real progress, and I quickly became fixated, reading hundreds of papers, deeply immersing myself in the burgeoning field. By the summer of 2020, I was convinced that the future of computing was conversational. Every interaction with a computer is already a conversation of sorts, just using buttons, keys, and pixels to translate human thoughts to machine-readable code. Now that barrier was starting to break down. Machines would soon understand our language. It was, and still is, a thrilling prospect.

When the Interface Spoke Back

There’s a recurrent problem with making sense of progress in AI. We quickly adapt, even to breakthroughs that astound us initially, and within no time they seem routine, even mundane. We no longer gasp at AlphaGo or GPT-3. What seems like near-magic engineering one day is just another part of the furniture the next. It’s easy to become blasé and many have. In the words of John McCarthy, who coined the term “artificial intelligence”: “As soon as it works, no one calls it AI anymore.” AI is—as those of us building it like to joke—“what computers can’t do.” Once they can, it’s just software.

When Magic Becomes Mundane: AI’s Disappearing Act

So, where does AI go next as the wave fully breaks? Today we have narrow or weak AI: limited and specific versions. GPT-4 can spit out virtuoso texts, but it can’t turn around tomorrow and drive a car, as other AI programs do. Existing AI systems still operate in relatively narrow lanes. What is yet to come is a truly general or strong AI capable of human-level performance across a wide range of complex tasks—able to seamlessly shift among them. But this is exactly what the scaling hypothesis predicts is coming and what we see the first signs of in today’s systems. AI is still in an early phase. It may look smart to claim that AI doesn’t live up to the hype, and it’ll earn you some Twitter followers. Meanwhile, talent and investment pour into AI research nonetheless. I cannot imagine how this will not prove transformative in the end. If for some reason LLMs show diminishing returns, then another team, with a different concept, will pick up the baton, just as the internal combustion engine repeatedly hit a wall but made it in the end. Fresh minds, new companies, will keep working at the problem. Then as now, it takes only one breakthrough to change the trajectory of a technology. If AI stalls, it will have its Otto and Benz eventually. Further progress—exponential progress—is the most likely outcome.

Past the Plateau: Why General AI Still Feels Inevitable

In a paper published in 1950, the computer scientist Alan Turing suggested a legendary test for whether an AI exhibited human-level intelligence. When AI could display humanlike conversational abilities for a lengthy period of time, such that a human interlocutor couldn’t tell they were speaking to a machine, the test would be passed: the AI, conversationally akin to a human, deemed intelligent. For more than seven decades this simple test has been an inspiration for many young researchers entering the field of AI. Today, as the LaMDA-sentience saga illustrates, systems are already close to passing the Turing test. But, as many have pointed out, intelligence is about so much more than just language (or indeed any other single facet of intelligence taken in isolation). One particularly important dimension is in the ability to take actions. We don’t just care about what a machine can say; we also care about what it can do. What we would really like to know is, can I give an AI an ambiguous, open-ended, complex goal that requires interpretation, judgment, creativity, decision-making, and acting across multiple domains, over an extended time period, and then see the AI accomplish that goal? Put simply, passing a Modern Turing Test would involve something like the following: an AI being able to successfully act on the instruction “Go make $1 million on Amazon in a few months with just a $100,000 investment.” It might research the web to look at what’s trending, finding what’s h...

Show, Don’t Tell: The Modern Turing Test

Rather than get too distracted by questions of consciousness, then, we should refocus the entire debate around near-term capabilities and how they will evolve in the coming years. As we have seen, from Hinton’s AlexNet to Google’s LaMDA, models have been improving at an exponential rate for more than a decade. These capabilities are already very real indeed, but they are nowhere near slowing down. While they are already having an enormous impact, they will be dwarfed by what happens as we progress through the next few doublings and as AIs complete complex, multistep end-to-end tasks on their own. I think of this as “artificial capable intelligence” (ACI), the point at which AI can achieve complex goals and tasks with minimal oversight. AI and AGI are both parts of the everyday discussion, but we need a concept encapsulating a middle layer in which the Modern Turing Test is achieved but before systems display runaway “superintelligence.” ACI is shorthand for this point. The first stage of AI was about classification and prediction—it was capable, but only within clearly defined limits and at preset tasks. It could differentiate between cats and dogs in images, and then it could predict what came next in a sequence to produce pictures of those cats and dogs. It produced glimmers of creativity, and could be quickly integrated into tech companies’ products. ACI represents the next stage of AI’s evolution. A system that not only could recognize and generate novel images, audio,...

From Knowing to Doing: Enter ACI

The future of AI is, at least in one sense, fairly easy to predict. Over the next five years, vast resources will continue to be invested. Some of the smartest people on the planet are working on these problems. Orders of magnitude more computation will train the top models. All of this will lead to more dramatic leaps forward, including breakthroughs toward AI that can imagine, reason, plan, and exhibit common sense. It won’t be long before AI can transfer what it “knows” from one domain to another, seamlessly, as humans do. What are now only tentative signs of self-reflection and self-improvement will leap forward. These ACI systems will be plugged into the internet, capable of interfacing with everything we humans do, but on a platform of deep knowledge and ability. It will be not just language they’ve mastered but a bewildering array of tasks, too. AI is far deeper and more powerful than just another technology. The risk isn’t in overhyping it; it’s rather in missing the magnitude of the coming wave. It’s not just a tool or platform but a transformative meta-technology, the technology behind technology and everything else, itself a maker of tools and platforms, not just a system but a generator of systems of any and all kinds. Step back and consider what’s happening on the scale of a decade or a century. We really are at a turning point in the history of humanity.

When the Tool Makes Tools: ACI’s Decisive Decade

Thanks to lifesaving treatments like vaccines, we are already accustomed to the idea of intervening in our biology to help us fight disease. The field of systems biology aims to understand the “larger picture” of a cell, tissue, or organism by using bioinformatics and computational biology to see how the organism works holistically; such efforts could be the foundation for a new era of personalized medicine. Before long the idea of being treated in a generic way will seem positively medieval; everything, from the kind of care we receive to the medicines we are offered, will be precisely tailored to our DNA and specific biomarkers. Eventually, it might be possible to reconfigure ourselves to enhance our immune responses. That, in turn, might open the door to even more ambitious experimentation like longevity and regenerative technologies, already a burgeoning area of research.

Made-to-Measure Medicine: From Systems Biology to Longevity

In 1837, John Deere was a blacksmith working in Grand Detour, Illinois. This was prairie country, with its dense black soil and wide-open spaces. It had potential as some of the world’s best arable land—great for crops but incredibly tough to plow. Then one day Deere saw a broken steel saw at a mill. Steel being scarce, he took his find home and fashioned the blade into a plow. Strong and smooth, steel was the perfect material for plowing through the dense, sticky soil. Although others had seen steel as an alternative to the coarser iron plows, Deere’s breakthrough was to ramp up mass production. Before long farmers from across the Midwest were flocking to his workshop. His invention opened the prairie to a flood of settlers. The Midwest duly became the breadbasket of the world; John Deere quickly became synonymous with agriculture; and a techno-geographic revolution was instigated. The John Deere company still makes agricultural technology today. You might be thinking tractors, sprinklers, and combines, and it’s true that John Deere does make all these things. Increasingly, though, the company builds robots. The future of agriculture, as John Deere sees it, involves autonomous tractors and combines that operate independently, following a field’s GPS coordinates and using an array of sensors to make automatic, real-time alterations to harvesting, maximizing yield and minimizing waste. The company is producing robots that can plant, tend, and harvest crops, with levels of p...

Steel to Silicon: John Deere and the Robot Farm

Emerging technologies have always created new threats, redistributed power, and removed barriers to entry. Cannons meant a small force could destroy castles and level armies. A few colonial soldiers with advanced weapons could massacre thousands of indigenous people. The printing press meant a single workshop might produce thousands of pamphlets—spreading ideas with an ease that medieval monks copying books by hand could scarcely fathom. Steam power enabled single factories to do the work of entire towns. The internet took this capacity to a new peak: a single tweet or image might travel the world in minutes or seconds; a single algorithm could help a small start-up to grow into a vast, globe-spanning corporation.

From Cannons to Tweets: Tech Flattens Power

Dual-use technologies are those with both civilian and military applications. In World War I, the process of synthesizing ammonia was seen as a way of feeding the world. But it also allowed for the creation of explosives, and helped pave the way for chemical weapons. Complex electronics systems for passenger aircraft can be repurposed for precision missiles. Conversely, the Global Positioning System was originally a military system, but now has countless everyday consumer uses. At launch, the PlayStation 2 was regarded by the U.S. Department of Defense as so powerful that it could potentially help hostile militaries usually denied access to such hardware. Dual-use technologies are both helpful and potentially destructive, tools and weapons. What the concept captures is how technologies tend toward the general, and a certain class of technologies come with a heightened risk because of this. They can be put toward many ends—good, bad, everywhere in between—often with difficult-to-predict consequences.

Double-Edged by Design: How Tech Becomes Tool and Weapon

Technological evolution has been speeding up for centuries. Omni-use features and asymmetric impacts are magnified in the coming wave, but to some extent they’re inherent properties of all technology. That isn’t the case for autonomy. For all of history technology has been “just” a tool, but what if the tool comes to life? Autonomous systems are able to interact with their surroundings and take actions without the immediate approval of humans. For centuries the idea that technology is somehow running out of control, a self-directed and self-propelling force beyond the realms of human agency, remained a fiction. Not anymore. Technology has always been about allowing us to do more, but crucially with humans still doing the doing. It has leveraged our existing abilities and automated precisely codified tasks. Until now, constant oversight and management have been the default. Technology remained to greater or lesser degrees under meaningful human control. Full autonomy is qualitatively different. Take autonomous vehicles. In certain conditions today, they can drive on roads with minimal or no direct input from the driver. Researchers in the field categorize autonomy from level 0, no autonomy whatsoever, to level 5, where a vehicle can drive itself under all conditions and the driver simply inputs a destination and then can fall happily asleep. You won’t find level 5 vehicles on the roads anytime soon, not least for legal and insurance reasons. The new wave of autonomy heralds...

From Hand to Helm: When Tools Take the Wheel

Will new inventions be beyond our grasp? Previously creators could explain how something worked and why it did what it did, even if this required vast detail. That’s increasingly no longer true. Many technologies and systems are becoming so complex that they’re beyond the capacity of any one individual to truly understand: quantum computing and other technologies operate toward the limits of what can be known. A paradox of the coming wave is that its technologies are largely beyond our ability to comprehend at a granular level yet still within our ability to create and use. In AI, the neural networks moving toward autonomy are, at present, not explainable. You can’t walk someone through the decision-making process to explain precisely why an algorithm produced a specific prediction. Engineers can’t peer beneath the hood and easily explain what caused something to happen. GPT-4, AlphaGo, and the rest are black boxes, their outputs and decisions based on opaque and intricate chains of minute signals. Autonomous systems can and may be explainable, but the fact that so much of the coming wave operates at the edge of what we can understand should give us pause. We won’t always be able to predict what these autonomous systems will do next; that’s the nature of autonomy.

Built Beyond Understanding: The Black-Box Paradox

Humans dominate our environment because of our intelligence. A more intelligent entity could, it follows, dominate us. The AI researcher Stuart Russell calls it the “gorilla problem”: gorillas are physically stronger and tougher than any human being, but it is they who are endangered or living in zoos; they who are contained. We, with our puny muscles but big brains, do the containment. By creating something smarter than us, we could put ourselves in the position of our primate cousins. With a long-term view in mind, those focusing on AGI scenarios are right to be concerned. Indeed, there is a strong case that by definition a superintelligence would be fully impossible to control or contain. An “intelligence explosion” is the point at which an AI can improve itself again and again, recursively making itself better in ever faster and more effective ways. Here is the definitive uncontained and uncontainable technology. The blunt truth is that nobody knows when, if, or exactly how AIs might slip beyond us and what happens next; nobody knows when or if they will become fully autonomous or how to make them behave with awareness of and alignment with our values, assuming we can settle on those values in the first place. Nobody really knows how we can contain the very features being researched so intently in the coming wave. There comes a point where technology can fully direct its own evolution; where it is subject to recursive processes of improvement; where it passes beyond ex...

When Minds Outrun Masters: Facing the Gorilla Problem

Technology is pushed on by all too rudimentary and fundamentally human drivers. From curiosity to crisis, fortune to fear, at its heart technology emerges to fill human needs. If people have powerful reasons to build and use it, it will get built and used. Yet in most discussions of technology people still get stuck on what it is, forgetting why it was created in the first place. This is not about some innate techno-determinism. This is about what it means to be human.

Motives Make Machines: The Human Engine of Technology

Postwar America took its technological supremacy for granted. Sputnik woke it up. In the fall of 1957, the Soviets launched Sputnik, the world’s first artificial satellite, humanity’s first encroachment on space. About the size of a beach ball, it was still impossibly futuristic. Sputnik was up there for the world to see, or rather hear, its extraterrestrial beeps broadcasting around the planet. Pulling it off was an undeniable feat. This was a crisis for America, a technological Pearl Harbor. Policy reacted. Science and technology, from high schools to advanced laboratories, became national priorities, with new funding and new agencies like NASA and DARPA. Massive resources were plowed into major technology projects, not least the Apollo missions. These spurred many important advances in rocketry, microelectronics, and computer programming. Nascent alliances like NATO were strengthened. Twelve years later, it was the United States, not the USSR, that succeeded in putting a human on the moon. The Soviets almost bankrupted themselves trying to keep up. With Sputnik, Russia had blown past the United States, a historic technical achievement with enormous geopolitical ramifications. But when America needed to step up, it did.

The Beep Heard ’Round the World: Sputnik and America’s Tech Awakening

The truth is that the curiosity of academic researchers or the will of motivated governments is insufficient to propel new breakthroughs into the hands of billions of consumers. Science has to be converted into useful and desirable products for it to truly spread far and wide. Put simply: most technology is made to earn money. If anything, this is perhaps the most persistent, entrenched, dispersed incentive of all. Profit drives the Chinese entrepreneur to develop moldings for a radically redesigned phone; it pushes the Dutch farmer to find new robotics and greenhouse technologies to grow tomatoes year-round in the cool climate of the North Sea; it leads suave investors on Palo Alto’s Sand Hill Road to invest millions of dollars in untested young entrepreneurs. While the motivations of their individual contributors may vary, Google is building AI, and Amazon is building robots, because as public companies with shareholders to please, they see them as ways to make a profit. And this, the potential for profit, is built on something even more long-lasting and robust: raw demand. People both want and need the fruits of technology. People need food, or refrigeration, or telecoms to live their lives; they might want AC units, or a new kind of shoe design requiring some intricate new manufacturing technique, or some kind of revolutionary new food-coloring method for cupcakes, or any of the innumerable everyday ends to which technology is put to use. Either way, technology helps p...

From Lab to Ledger: Demand Turns Ideas into Industries

Think about the impact of the new wave of AI systems. Large language models enable you to have a useful conversation with an AI about any topic in fluent, natural language. Within the next couple of years, whatever your job, you will be able to consult an on-demand expert, ask it about your latest ad campaign or product design, quiz it on the specifics of a legal dilemma, isolate the most effective elements of a pitch, solve a thorny logistical question, get a second opinion on a diagnosis, keep probing and testing, getting ever more detailed answers grounded in the very cutting edge of knowledge, delivered with exceptional nuance. All of the world’s knowledge, best practices, precedent, and computational power will be available, tailored to you, to your specific needs and circumstances, instantaneously and effortlessly. It is a leap in cognitive potential at least as great as the introduction of the internet.

Expertise on Tap: The Post-Internet Leap

For most of history simply feeding yourself and your family was the dominant challenge of human life. Farming has always been a hard, uncertain business. But especially prior to the improvements of the twentieth century, it was much, much harder. Any variation in weather conditions—too cold, hot, dry, or wet—could be catastrophic. Almost everything was done by hand, maybe with the help of some oxen if you were lucky. At some times of the year there was little to do; at others, there were weeks of unceasing, backbreaking physical labor. Crops could be ruined by disease or pests, spoil after harvesting, or get stolen by invading armies. Most farmers lived hand to mouth, often working as serfs, giving up much of their scant crop. Even in the most productive parts of the world, yields were low and fragile. Life was tough, lived on the edge of disaster. When Thomas Malthus argued in 1798 that a fast-growing population would quickly exhaust the carrying capacity of agriculture and lead to a collapse, he wasn’t wrong; static yields would and often did follow this rule. What he hadn’t accounted for was the scale of human ingenuity. Assuming favorable weather conditions and using the latest techniques, in the thirteenth century each hectare of wheat in England yielded around half a ton. There it remained for centuries. Slowly the arrival of new techniques and technologies changed all that: from crop rotation to selective breeding, mechanized plows, synthetic fertilizer, pesticides,...

Sixteenfold Harvest: From Scarcity to Surplus

Technology can and will improve lives and solve problems. Think of a world populated by trees that are longer lived and absorb much greater amounts of CO2. Or phytoplanktons that help the oceans become a greater and more sustainable carbon sink. AI has helped design an enzyme that can break down the plastic clogging our oceans. It will also be an important part of how we predict what is coming, from guessing where a wildfire might hit suburbia to tracking deforestation through public data sets. This will be a world of cheap, personalized drugs; fast, accurate diagnoses; and AI-generated replacements for energy-intensive fertilizers. Sustainable, scalable batteries need radical new technologies. Quantum computers paired with AI, with their ability to model down to the molecular level, could play a critical role in finding substitutes to conventional lithium batteries that are lighter, cheaper, cleaner, easier to produce and recycle, and more plentiful. Likewise on work with photovoltaic materials, or drug discovery, that enables molecular-level simulations to identify new compounds—far more precise and powerful than using the slow experimental techniques of the past. This is hyper-evolution in action, and it promises to save billions in R&D while going far beyond the present research paradigm. A school of naive techno-solutionism sees technology as the answer to all of the world’s problems. Alone, it’s not. How it is created, used, owned, and managed all make a difference. ...

Tools, Not Talismans: Realistic Optimism at Planet Scale

The Los Alamos director J. Robert Oppenheimer was a highly principled man. But above all else he was a curiosity-driven problem solver. Consider these words, in their own way as chilling as his famous Bhagavad Gita quotation (on seeing the first nuclear test, he recalled some lines from Hindu scripture: “Now I am become Death, the destroyer of worlds”): “When you see something that is technically sweet, you go ahead and do it, and you argue about what to do about it only after you have had your technical success.” It was an attitude shared by his colleague on the Manhattan Project, the brilliant, polymathic Hungarian American John von Neumann. “What we are creating now,” he said, “is a monster whose influence is going to change history, provided there is any history left, yet it would be impossible not to see it through, not only for military reasons, but it would also be unethical from the point of view of the scientists not to do what they know is feasible, no matter what terrible consequences it may have.”

When Curiosity Outruns Conscience