The Coming Wave: AI, Power, and Our Future

by Mustafa Suleyman

The coming wave is defined by two core technologies: artificial intelligence (AI) and synthetic biology.

So, what is a wave? Put simply, a wave is a set of technologies coming together around the same time, powered by one or several new general-purpose technologies with profound societal implications.

Technological waves don’t arrive with the neat predictability of the tides. Over the long term, waves erratically intersect and intensify. The ten thousand years up to 1000 BCE saw seven general-purpose technologies emerge. The two hundred years between 1700 and 1900 marked the arrival of six, from steam engines to electricity. And in the last hundred years alone there were seven.

As you get more and cheaper technology, it enables new and cheaper technologies downstream. Uber was impossible without the smartphone, which itself was enabled by GPS, which was enabled by satellites, which were enabled by rockets, which were enabled by combustion techniques, which were enabled by language and fire.

Containment is the overarching ability to control, limit, and, if need be, close down technologies at any stage of their development or deployment.

Nuclear is an exception to the unstoppable spread of technology, but only because of the tremendous costs and complexity involved, the decades of tough multilateral effort, the fear-inducing enormity of its lethal potential, and pure luck.

The coming wave of technology is built primarily on two general-purpose technologies capable of operating at the grandest and most granular levels alike: artificial intelligence and synthetic biology. For the first time core components of our technological ecosystem directly address two foundational properties of our world: intelligence and life.

In 1996, thirty-six million people used the internet; this year it will be well over five billion. That’s the kind of trajectory we should expect for these tools, only much faster. Over the next few years, I believe, AI will become as ubiquitous as the internet itself: just as available, and yet even more consequential.

I think of this as “artificial capable intelligence” (ACI), the point at which AI can achieve complex goals and tasks with minimal oversight.

Agents :-)

the Carlson curve: the epic collapse in costs for sequencing DNA. Thanks to ever-improving techniques, the cost of human genome sequencing fell from $1 billion in 2003 to well under $1,000 by 2022.

In the 1960s computer chips were still largely hand built, just as—until recently—most biotech research was still a manual process, slow, unpredictable, messy in every sense. Now semiconductor fabrication is a hyperefficient atomic-scale manufacturing process churning out some of the world’s most complex products. Biotech is following a similar trajectory, only at a much earlier phase; organisms will soon be designed and produced with the precision and scale of today’s computer chips and software.

Previous experiments had delivered the structure of about 190,000 proteins to the European Bioinformatics Institute’s database, about 0.1 percent of known proteins in existence. DeepMind uploaded some 200 million structures in one go, representing almost all known proteins. Whereas once it might have taken researchers weeks or months to determine a protein’s shape and function, that process can now begin in a matter of seconds. This is what we mean by exponential change. This is what the coming wave makes possible.

Nanomachines would work at speeds far beyond anything at our scale, delivering extraordinary outputs: an atomic-scale nanomotor, for example, could rotate forty-eight billion times a minute. Scaled up, it could power a Tesla with material equivalent in volume to about twelve grains of sand.

The coming wave is, however, characterized by a set of four intrinsic features compounding the problem of containment. First among them is the primary lesson of this section: hugely asymmetric impact. You don’t need to hit like with like, mass with mass; instead, new technologies create previously unthinkable vulnerabilities and pressure points against seemingly dominant powers. Second, they are developing fast, a kind of hyper-evolution, iterating, improving, and branching into new areas at incredible speed. Third, they are often omni-use; that is, they can be used for many different purposes. And fourth, they increasingly have a degree of autonomy beyond any previous technology.

A more appropriate term for the technologies of the coming wave is “omni-use,” a concept that grasps at the sheer levels of generality, the extreme versatility on display. Omni-use technologies like steam or electricity have wider societal effects and spillovers than narrower technologies.

In the early years of the twenty-first century, China’s R&D spending was just 12 percent of America’s. By 2020, it was 90 percent. On current trends it will be significantly ahead by the mid-2020s, as it already is on patent applications.

In developed economies, people work far less than they used to for far more reward. In Germany, for example, annual working hours have decreased by nearly 60 percent since 1870.

Is this true for founders?

Assuming favorable weather conditions and using the latest techniques, in the thirteenth century each hectare of wheat in England yielded around half a ton. There it remained for centuries. Slowly the arrival of new techniques and technologies changed all that: from crop rotation to selective breeding, mechanized plows, synthetic fertilizer, pesticides, genetic modifications, and now even AI-optimized planting and weeding. In the twenty-first century, yields are now at about eight tons per hectare. The very same small, innocuous patch of ground, the same geography and soil that was reaped in the thirteenth century, can now deliver sixteen times the crop.

Electric vehicles may not emit carbon when being driven, but they are resource hungry nonetheless: materials for just one EV require extracting around 225 tons of finite raw materials, demand for which is already spiking unsustainably.

Imagine the average tomato soaked in five tablespoons of oil. That’s how much went into growing it. What’s more, to meet global demand, agriculture will need to produce almost 50 percent more food by 2050 just as yields decline in the face of climate change. If we are to stand any chance of keeping global warming under two degrees Celsius, then the world’s scientists working under the UN’s Intergovernmental Panel on Climate Change have been clear: carbon capture and storage is an essential technology. And yet it’s largely not been invented or is still to be deployed at scale. To meet this global challenge, we’ll have to reengineer our agricultural, manufacturing, transport, and energy systems from the ground up with new technologies that are carbon neutral or probably even carbon negative. These are not inconsiderable tasks. In practice it means rebuilding the entire infrastructure of modern society while hopefully also offering quality-of-life improvements to billions.

Re climate goals ad food systems

New jobs might be created in the long term, but for millions they won’t come quick enough or in the right places.

On automation and job creation

This is a world where billionaires and latter-day prophets can build and run microstates; where non-state actors from corporations to communes to algorithms begin to overshadow the state from above but also from below.

The coming wave will only deepen and recapitulate the exact same contradictory dynamics of the last wave. The internet does precisely this: centralizes in a few key hubs while also empowering billions of people. It creates behemoths and yet gives everyone the opportunity to join in. Social media created a few giants and a million tribes.

China is a major part of the story of technology in the coming decades, but by the century’s end the Shanghai Academy of Social Sciences predicts the country could have only 600 million people, a staggering reversal of nearly a century’s population increases. China’s total fertility rate is one of the lowest in the world, matched only by neighbors like South Korea and Taiwan.

There is an unbridgeable gulf between the desire to rein in the coming wave and the desire to shape and own it, between the need for protections against technologies and the need for protections against others. Advantage and control point in opposing directions.

A key driver behind this progress is called reinforcement learning from human feedback. To fix their bias-prone LLMs, researchers set up cunningly constructed multi-turn conversations with the model, prompting it to say obnoxious, harmful, or offensive things, seeing where and how it goes wrong. Flagging these missteps, researchers then reintegrate these human insights into the model, eventually teaching it a more desirable worldview, in a way not wholly dissimilar from how we try to teach children not to say inappropriate things at the dinner table.

This seems problematic, prone to bias.

In AI, the lion’s share of the most advanced GPUs essential to the latest models are designed by one company, the American firm NVIDIA. Most of its chips are manufactured by one company, TSMC, in Taiwan, the most advanced in just a single building, the world’s most sophisticated and expensive factory. TSMC’s machinery to make these chips comes from a single supplier, the Dutch firm ASML, by far Europe’s most valuable and important tech company. ASML’s machines, which use a technique known as extreme ultraviolet lithography and produce chips at levels of astonishing atomic precision, are among the most complex manufactured goods in history. These three companies have a choke hold on cutting-edge chips, a technology so physically constrained that one estimate argues they cost up to $10 billion per kilogram.

It was a foundational lesson for me: shareholder capitalism works because it is simple and clear, and governance models too have a tendency to default to the simple and clear. In the shareholder model, lines of accountability and performance tracking are quantified and very transparent. It may be possible to design more modern structures in theory, but operating them in practice is another story.

For understandable reasons, we don’t let any business build or operate nuclear reactors in any way they see fit. In practice, the state is intimately involved in—and closely watching, licensing, and governing—every aspect of their existence. Over time this will and should become more true of technology in general. Today anyone can build AI. Anyone can set up a lab. We should instead move to a more licensed environment.

Delicate alliances and international cooperation can be pulled off, and they can change history. Consider these examples, some of which we discussed earlier: the Treaty on the Non-proliferation of Nuclear Weapons; the Montreal Protocol outlawing CFCs; the invention, trialing, and rollout of a polio vaccine across a Cold War divide; the Biological Weapons Convention, a disarmament treaty effectively banning biological weapons; bans on cluster munitions, land mines, genetic editing of human beings, and eugenics policies; the Paris Agreement, aiming to limit carbon emissions and the worst impacts of climate change; the global effort to eradicate smallpox; phasing out lead in gasoline; and putting an end to asbestos. Countries no more like giving up power than companies like missing out on profit, and yet these are precedents to learn from, shards of hope in a landscape riven with resurgent techno-competition. Each had specific conditions and challenges that both helped it come about and hindered perfect compliance. But each, crucially, is a precious example of the world’s nations uniting and compromising to face a major challenge, offering hints and frameworks for tackling the coming wave.

Safety relies on things not failing, not getting into the wrong hands, forever. Some level of policing the internet, DNA synthesizers, AGI research programs, and so on is going to be essential. It’s painful to write. As a young twentysomething, I started out from a privacy maximalist position, believing spaces of communication and work completely free from oversight were foundational rights and important parts of healthy democracy. Over the years, though, as the arguments became clearer and the technology more and more developed, I’ve updated that view. It’s just not acceptable to create situations where the threat of catastrophic outcomes is ever present. Intelligence, life, raw power—these are not playthings, and should be treated with the respect, care, and control they deserve. Technologists and the general public alike will have to accept greater levels of oversight and regulation than have ever been the case before. Just as most of us wouldn’t want to live in societies without laws and police, most of us wouldn’t want to live in a world of unrestricted technology either.