The Coming Wave: AI, Power, and Our Future

by Mustafa Suleyman

A ragtag volunteer band of drone hobbyists, software engineers, management consultants, and soldiers, they were amateurs, designing, building, and modifying their own drones in real time, much like a start-up. A lot of their equipment was crowdsourced and crowdfunded.

A thousand-strong group of nonmilitary elite programmers and computer scientists banded together in an organization called Delta to bring advanced AI and robotics capabilities to the army, using machine learning to identify targets, monitor Russian tactics, and even suggest strategies.

A precision missile in a conventional military costs hundreds of thousands of dollars; with AI and consumer-grade drones, with custom software and 3-D printed parts, something similar has now been battle-tested in Ukraine at a cost of around $15,000.

Switchblade loitering munitions,

drones that wait around a target until an optimal ...

When technology confers a cost and tactical advantage like this, it will, of course, inevitably proliferate and be taken up by all sides.

The real question is what this means for conflict when production costs fall by another order of magnitude and capabilities multiply. Conventional militaries and governments are already struggling to contain them. What comes next will be much harder to contain.

The coming wave is, however, characterized by a set of four intrinsic features compounding the problem of containment. First among them is the primary lesson of this section: hugely asymmetric impact.

Second, they are developing fast, a kind of hyper-evolution, iterating, improving, and branching into new areas at incredible speed.

Third, they are often omni-use;

fourth, they increasingly have a degree of autonomy beyond any ...

These features define...

Understanding them is vital in identifying what benefits and risks arise from their creation; together they escalate containment and control ...

ASYMMETRY: A COLOSSAL TRANSF...

This new wave of technology has unlocked powerful capabilities that are cheap, easy to access and use, targeted, and scalable. This clearly brings risks.

nascent

These developments represent a colossal transfer of power away from traditional states and militaries toward anyone with the capacity, and motivation, to deploy these devices. There is no obvious reason why a single operator, with enough wherewithal, could not control a swarm of thousands of drones.

One viable quantum computer could render the world’s entire encryption infrastructure redundant.

AI creates asymmetric risks beyond those of a bad batch of food, a plane accident, or a faulty product. Its risks extend to entire societies,

Network scale makes containing damage,

HYPER-EVOLUTION: ENDLESS ACCELERATION

If you want to contain technology, you might hope it develops at a manageable pace, giving society time and space to understand and adapt to it.

Consider what Moore’s law alone will deliver over the next decade. Should it hold, in ten years a dollar will buy you a hundred times the compute of today.

Quantum technologies, many millions of times more powerful than the most powerful classical computers, could let this play out at a molecular level. This is what we mean by hyper-evolution—a fast, iterative platform for creation.

OMNI-USE: MORE IS MORE

Defying conventional wisdom, progress in health care was one of the areas that slowed in the recent stagnation of innovation

In 2020 an AI system sifted through 100 million molecules to create the first machine-learning-derived antibiotic—called halicin (yes, after HAL from 2001: A Space Odyssey)—which can potentially help fight tuberculosis.

Exscientia,

asking their molecule-generating AI to find poisons. In six hours it identified more than forty thousand molecules with toxicity comparable to the most dangerous chemical weapons,

Omni-use technologies like steam or electricity have wider societal effects and spillovers than narrower technologies.

Containing something like this is always going to be much harder than containing a constrained, single-task technology,

Engineering life is a completely general technique whose potential uses are near limitless; it might create material for construction, tackle disease, and store data.

What’s different about the coming wave is how quickly it is being embedded, how globally it spreads, how easily it can be componentized into swappable parts, and just how powerful and above all broad its applications could be.

This is the containment problem supersized.

AUTONOMY AND BEYOND: WILL HUMANS BE IN THE LOOP?

Researchers in the field categorize autonomy from level 0, no autonomy whatsoever, to level 5, where a vehicle can drive itself under all conditions and the driver simply inputs a destination and then can fall happily asleep.

we just specify a high-level goal and rely on a machine to figure out the optimal way of getting there.

Internal research on GPT-4 concluded that it was “probably” not capable of acting autonomously or self-replicating, but within days of launch users had found ways of getting the system to ask for its own documentation and to write scripts for copying itself and taking over other machines.

reverberate

We humans face a singular challenge: Will new inventions be beyond our grasp? Previously creators could explain how something worked and why it did what it did, even if this required vast detail. That’s increasingly no longer true.

A paradox of the coming wave is that its technologies are largely beyond our ability to comprehend at a granular level yet still within our ability to create and use.

some AI researchers want to automate every aspect of building AI systems, feeding that hyper-evolution, but potentially with radical degrees of independence through self-improvement.

AIs are already finding ways to improve their own algorithms.

the “gorilla problem”: gorillas are physically stronger and tougher than any human being, but it is they who are endangered or living in zoos; they who are contained.

By creating something smarter than us, we could put ourselves in the position of our primate cousins.

a superintelligence would be fully impossible to control or contain.

“intelligence explosion”

Homo technologicus may end up being threatened by its own creation.

It was a significant moment in tech history—for those who care about such things.

all. The significance was lost on no one. The challenger, a Western firm, London based, American owned, had just marched into an ancient, iconic, cherished game, literally put its flag in the turf, and obliterated the home team.