More on this book
Community
Kindle Notes & Highlights
Read between
September 18 - October 31, 2023
CHAPTER 1
pessimism-aversion trap: the misguided analysis that arises when you are overwhelmed by a fear of confronting potentially dark realities, and the resulting tendency to look the other way.
The various technologies I’m speaking of share four key features that explain why this isn’t business as usual: they are inherently general and therefore omni-use, they hyper-evolve, they have asymmetric impacts, and, in some respects, they are increasingly autonomous.
PART I
CHAPTER 2
CHAPTER 3
THE CONTAINMENT PROBLEM
Technology’s unavoidable challenge is that its makers quickly lose control over the path their inventions take once introduced to the world.
Containment encompasses regulation, better technical safety, new governance and ownership models, and new modes of accountability and transparency, all as necessary (but not sufficient) precursors to safer technology.
PART II
CHAPTER 4
Then something remarkable happened. DQN appeared to discover a new, and very clever, strategy. Instead of simply knocking out bricks steadily, row by row, DQN began targeting a single column of bricks. The result was the creation of an efficient route up to the back of the block of bricks. DQN had tunneled all the way to the top, creating a path that then enabled the ball to simply bounce off the back wall, steadily destroying the entire set of bricks like a frenzied ball in a pinball machine.
For the first time I’d witnessed a very simple, very elegant system that could learn valuable knowledge, arguably a strategy that wasn’t obvious to many humans.
ALPHAGO AND THE BEGINNING OF THE FUTURE
AlphaGo initially learned by watching 150,000 games played by human experts. Once we were satisfied with its initial performance, the key next step was creating lots of copies of AlphaGo and getting it to play against itself over and over.
AlphaGo won again. Go strategy was being rewritten before our eyes. Our AI had uncovered ideas that hadn’t occurred to the most brilliant players in thousands of years.
with just a day’s training, AlphaZero was capable of learning more about the game than the entirety of human experience could teach it.
First bits and then increasingly genes supplanted atoms as the building blocks of invention.
This is key to understanding the coming wave.
Each technology described here intersects with, buttresses, and boosts the others in ways that make it difficult to predict their impact in advance.
Deep learning uses neural networks loosely modeled on those of the human brain. In simple terms, these systems “learn” when their networks are “trained” on large amounts of data.
AUTOCOMPLETE EVERYTHING: THE RISE OF LARGE LANGUAGE MODELS
LLMs take advantage of the fact that language data comes in a sequential order. Each unit of information is in some way related to data earlier in a series.
The model reads very large numbers of sentences, learns an abstract representation of the information contained within them, and then, based on this, generates a prediction about what should come next.
The challenge lies in designing an algorithm that “knows where to look” for signals in a given sentence. What are the key words, the most salient elements of a sentence, and how do they relate to one another? ...
This highlight has been truncated due to consecutive passage length restrictions.
(GPT stands for generative pre-trained transformer.)
a key ingredient of the LLM revolution is that for the first time very large models could be trained directly on raw, messy, real-world data, without the need for carefully curated and human-labeled data sets.
But humans’ ability to complete given tasks—human intelligence itself—is very much a fixed target, as large and multifaceted as it is. Unlike the scale of available compute, our brains do not radically change year by year. In time this gap will be closed.
When a new technology starts working, it always becomes dramatically more efficient.
I was convinced that the future of computing was conversational.
Every interaction with a computer is already a conversation of sorts, just using buttons, keys, and pixels to translate human thoughts to machine-readable code. Now that barrier was starting to break down.
CHAPTER 5 THE TECHNOLOGY OF LIFE
Like AI, synthetic biology is on a sharp trajectory of falling costs and rising capabilities.
CHAPTER 6 THE WIDER WAVE
CHAPTER 7 FOUR FEATURES OF THE COMING WAVE
If you want to contain technology, you might hope it develops at a manageable pace, giving society time and space to understand and adapt to
You can’t walk someone through the decision-making process to explain precisely why an algorithm produced a specific prediction.
CHAPTER 8 UNSTOPPABLE INCENTIVES
As long as these incentives are in place, the important question of “should we?” is moot.
Technological rivalry is a geopolitical reality.
China was already committed to investing heavily in science and technology, but AlphaGo helped focus government minds even more acutely on AI.
Xi Jinping, speaking to the Twentieth CCP Congress in 2022, “to meet strategic needs” the country “must adhere to science and technology as the number-one productive force, talent as the number-one resource, [and] innovation as the number-one driving force.”
China’s top-down model means it can marshal the state’s full resources behind technological ends.
Chinese institutions have published a whopping four and a half times more AI papers than U.S. counterparts since 2010, and comfortably more than the United States, the U.K., India, and Germany combined.
nearly double the number of STEM PhDs as the United States every year.
The openness imperative saturates research culture. Academia is built around peer review; any paper not subject to critical scrutiny by credible peers doesn’t meet the gold standard. Funders don’t like supporting work that stays locked away.
At DeepMind we learned early that opportunities to publish were a key factor when leading researchers decided where to work.
Since the start of the twenty-first century global energy use is up 45 percent, but the share coming from fossil fuels only fell from 87 to 84 percent—meaning fossil fuel use is greatly up despite all the moves into clean electricity as a power source.
Everything leaks. Everything is copied, iterated, improved. And because everyone is watching and learning from everyone else, with so many people all scratching around in the same areas, someone is inevitably going to figure out the next big breakthrough.
There is only one entity that could, perhaps, provide the solution, one that anchors our political system and takes final responsibility for the technologies society produces: the nation-state.
