More on this book
Community
Kindle Notes & Highlights
Read between
January 7 - February 6, 2025
Recall the four features of the coming wave: asymmetry, hyper-evolution, omni-use, and autonomy. Each feature must be viewed through the lens of containability.
Specific technologies are easier to regulate than omni-use technologies, but regulating omni-use is more important.
“Technology” now mostly means social media platforms and wearable gadgets to measure our steps and heart rate.
Technology isn’t just a way to store your selfies; it represents access to the world’s accumulated culture and wisdom. Technology is not a niche; it is a hyper-object dominating human existence.
amorphous
As recently as the 1970s, global atmospheric carbon was around the low 300s ppm. In 2022 it was at 420 ppm.
Data brings clarity.
The ultimate control is hard physical control, of servers, microbes, drones, robots, and algorithms.
“Boxing” an AI is the original and basic form of technological containment. This would involve no internet connections, limited human contact, a small, constricted external interface.
A system like this—called an air gap—could, in theory, stop an AI from engaging with the wider ...
This highlight has been truncated due to consecutive passage length restrictions.
Bodies like the Institute of Electrical and Electronics Engineers maintain more than two thousand technical safety standards on technologies ranging from autonomous robot development to machine learning.
a good proposal for legislation would be to require that a fixed portion—say, a minimum of 20 percent—of frontier corporate research and development budgets should be directed toward safety efforts, with an obligation to publish material findings to a government working group so that progress can be tracked and shared.
Giving off light with a wavelength between 200 and 230 nanometers, close to the ultraviolet spectrum, they can kill viruses while not penetrating the outer layer of the skin: a powerful weapon against pandemics and the spread of disease more widely.
Recall that at present no one can explain why, precisely, a model produces the outputs it does.
The highest-level challenge, whether in synthetic biology, robotics, or AI, is building a bulletproof off switch, a means of closing down any technology threatening to run out of control.
Melding social and technological containment mechanisms like this is critical.
Skills, too, are a choke point: the number of people working on all the frontier technologies discussed in this book is probably no more than 150,000.
People often ask me, given all this, why work in AI and build AI companies and tools? Aside from the huge positive contribution they can make, my answer is that I don’t just want to talk about and debate containment. I want to proactively help make it happen, on the front foot, ahead of where the technology is going. Containment needs technologists utterly focused on making it a reality.
Indeed, at times shrill criticism just becomes part of the same hype cycle as technology itself.
IP core to Google’s search business would stay with Google, but the rest would be available for us to advance DeepMind’s social mission, working on new drugs, better health care, climate change, and so on. It would mean investors could be rewarded, but also ensured that social purpose was in the company’s legal DNA.
U.S. federal government expenditure on R&D is at an all-time-low share of the total—just 20 percent—but still amounts to a not inconsiderable $179 billion per year.
