The Coming Wave: AI, Power, and Our Future

by Mustafa Suleyman

A system like this—called an air gap—could, in theory, stop an AI from engaging with the wider world or somehow “escaping.”

Compared with the magnitude of what could go wrong, safety and ethics research on AI is marginal. Only a handful of institutions, owing to the challenges of resources, take technical safety issues seriously. And yet safety decisions made today will alter the future course of technology and humanity. There’s a clear must-do here: encourage, incentivize, and directly fund much more work in this area. It’s time for an Apollo program on AI safety and biosafety. Hundreds of thousands should be working on it. Concretely, a good proposal for legislation would be to require that a fixed portion—say, a minimum of 20 percent—of frontier corporate research and development budgets should be directed toward safety efforts, with an obligation to publish material findings to a government working group so that progress can be tracked and shared.

Pandemic preparedness could, for example, be greatly enhanced by using low-wavelength lightbulbs that kill viruses. Giving off light with a wavelength between 200 and 230 nanometers, close to the ultraviolet spectrum, they can kill viruses while not penetrating the outer layer of the skin: a powerful weapon against pandemics and the spread of disease more widely. And if the COVID-19 pandemic taught us one thing, it’s the value of an integrated, accelerated approach across research, rollout, and regulation for novel vaccines.

We’re designing Pi to express self-doubt, solicit feedback frequently and constructively, and quickly give way assuming the human, not the machine, is right. We and others are also working on an important track of research that aims to fact-check a statement by an AI using third-party knowledge bases we know to be credible. Here it’s about making sure AI outputs provide citations, sources, and interrogable evidence that a user can further investigate when a dubious claim arises.

Another interesting example is “red teaming”—that is, proactively hunting for flaws in AI models or software systems. This means attacking your systems in controlled ways to probe for weaknesses

and other failure modes.

Another promising example of a new oversight mechanism is SecureDNA, a not-for-profit program started by a group of scientists and security specialists. At present only a fraction of synthesized DNA is screened for potentially dangerous elements, but a global effort like the SecureDNA program to plug every synthesizer—benchtop

benchtop at home or large and remote—into a centralized, secure, and encrypted system that can scan for pathogenic sequences is a great start. If people are printing potentially harmful sequences, they’re flagged. Cloud based, free, cryptographically secure, it updates in real time.

Some 80 percent of the high-quality quartz essential to things like photovoltaic panels and silicon chips comes from a single mine in North Carolina. DNA synthesizers and quantum computers are not commonplace consumer goods. Skills, too, are a choke point: the number of people working on all the frontier technologies discussed in this book is probably no more than 150,000.