The Coming Wave: AI, Power, and Our Future

by Mustafa Suleyman

Aum built dummy companies and infiltrated university labs to procure material, purchased land in Australia with the intent of prospecting for uranium to build nuclear weapons, and embarked on a huge biological and chemical weapons program in the hilly countryside outside Tokyo. The group experimented with phosgene, hydrogen cyanide, soman, and other nerve agents.

Members obtained the neurotoxin C. botulinum and sprayed it on Narita International Airport, the National Diet Building, the Imperial Palace, the headquarters of another religious group, and two U.S. naval bases. Luckily, they made a mistake in its manufacture and no harm ensued.

In 1994, Aum Shinrikyo sprayed the nerve agent sarin from a truck, killing eight and wounding two hundred. A year later they struck the Tokyo subway, releasing more sarin, killing thirteen and injuring some six thousand people.

Aum Shinrikyo combined an unusual degree of organization with a frightening level of ambition. They wanted to initiate World War III and a global collapse by murdering at shocking scale and began building an infrastructure to do so.

coterie

A new phase of history is here. With zombie governments failing to contain technology, the next Aum Shinrikyo, the next industrial accident, the next mad dictator’s war, the next tiny lab leak, will have an impact that is difficult to contemplate.

Containment is about the ability to control technology. Further back, that means the ability to control the people and societies behind it. As catastrophic impacts unfurl or their possibility becomes unignorable, the terms of debate will change. Calls for not just control but crackdowns will grow. The potential for unprecedented levels of vigilance will become ever more appealing. Perhaps it might be possible to spot and then stop emergent threats? Wouldn’t that be for the best—the right thing to do?

the inevitable reaction will be a tightening of the grip on power. The question is, at what cost?

The greater the catastrophe, the greater the stakes, the greater the need for countermeasures.

To counter calamity in the face of the unprecedented dynamics of the coming wave means an unprecedented response. It means not just watching everything but reserving the capacity to stop it and control it whenever and wherever necessary.

Differences between screening / scanning / monitoring and controlling will be very important. So will be the privacy walls.

many nations will convince themselves that the only way of truly ensuring this is to install the kind of blanket surveillance we saw in the last chapter: total control, backed by hard power.

a self-reinforcing “AI-tocracy” of steadily increasing data collection and coercion.

tendril

Trading off liberty and security is an ancient dilemma.

The consequences for liberty, sovereignty, and privacy have never been so potentially painful.

With the architecture of monitoring and coercion being built in China and elsewhere, the first steps have arguably been taken.

Lewis Mumford

exuded

affordances

Civilizations that collapse are not the exception; they are the rule. A survey of sixty civilizations suggests they last about four hundred years on average before falling apart.

The civilizations are a continuum. They don't collapse. Their rulers or the name of the state or their population mix changes. With AI, the entirety of the human civilization may be at risk.

Our entire edifice is premised on the idea of long-term economic growth. And long-term economic growth is ultimately premised on the introduction and diffusion of new technologies.

The development of new technologies is, as we’ve seen, a critical part of meeting our planet’s grand challenges. Without new technologies, these challenges will simply not be met.

without new technologies it will be impossible to maintain living standards.

“the governing models of the post–World War II era do not simply go broke, they become societal suicide pacts.”

trilemma.

“For progress there is no cure,”

the coming decades will see complex, painful trade-offs between prosperity, surveillance, and the threat of catastrophe growing ever more acute.

even an existential risk to the species.

The only coherent approach to technology is to see both sides at the same time.

Exponential change is coming. It is inevitable.

governments face multiple crises independent of the coming wave—declining

Going into 2020, the Global Health Security Index ranked the United States number one in the world and the U.K. not far behind in terms of pandemic readiness. Yet a catalog of disastrous decisions delivered mortality rates and financial costs materially worse than in peer countries like Canada and Germany. Despite what looked like excellent expertise, institutional depth, planning, and resources, even those best prepared on paper were sideswiped.

This, meanwhile, is an age of surprises.

High-risk AI must be “transparent, secure, subject to human control and properly documented.”

Consider how motorized transport is regulated. There isn’t a single regulator, or even just a few laws. Instead, we have regulations around traffic, roads, parking, seatbelts, emissions, driver training, and so on.

Complex regulations refined over decades made roads and vehicles incrementally safer and more ordered, enabling their growth and spread. And yet 1.35 million people a year still die in traffic accidents. Regulation may lessen the negative effects, but it can’t erase bad outcomes like crashes, pollution, or sprawl.

Regulation doesn’t just rely on the passing of a new law. It is also about norms, structures of ownership, unwritten codes of compliance and honesty, arbitration procedures, contract enforcement, oversight mechanisms. All of this needs to be integrated and the public needs to buy in.

a best-case scenario of strong, reasonably competent, cohesive (liberal democratic) nation-states capable of working coherently as units internally and coordinating well internationally.

For containment to be possible, rules need to work well in places as diverse as the Netherlands and Nicaragua, New Zealand and Nigeria.

DeFi

LLMs

China

regulation is matched by an unparalleled deployment of technology as a tool of authoritarian government power.

In effect, Chinese AI policy has two tracks: a regulated civilian path and a freewheeling military-industrial one.

the four features of the coming wave: asymmetry, hyper-evolution, omni-use, and autonomy.

Specific technologies are easier to regulate than omni-use technologies, but regulating omni-use is more important.

the more potential for offensive actions or autonomy, the greater the requirement for containment.

“Technology” is not, on the face of it, a problem in the same sense as a heating planet. And yet it might be.

Biotech and pharma have operated under safety standards far beyond those of most software businesses for decades.

The main monitor of bioweapons, for example, the Biological Weapons Convention, has a budget of just $1.4 million and only four full-time employees—fewer than the average McDonald’s.