More on this book
Community
Kindle Notes & Highlights
Read between
January 7 - February 6, 2025
Tech is by far the biggest single category in the S&P 500, constituting 26 percent of the index.
Investment in AI technologies alone has hit $100 billion a year.
PwC forecasts AI will add $15.7 trillion to the global economy by 2030. McKinsey forecasts a $4 trillion boost from biotech over the same period.
AI has helped design an enzyme that can break down the plastic clogging our oceans.
A school of naive techno-solutionism sees technology as the answer to all of the world’s problems. Alone, it’s not. How it is created, used, owned, and managed all make a difference.
The idea that technology alone can solve social and political problems is a dangerous delusion. But the idea that they can be solved without technology is also wrongheaded.
Those countries leaning into nationalism are, in part, experiencing a turning away from the bright twentieth-century promise that greater interconnectedness would accelerate the spread of wealth and democracy.
A meta-analysis published in the journal Nature reviewed the results of nearly five hundred studies, concluding there is a clear correlation between growing use of digital media and rising distrust in politics, populist movements, hate, and polarization.
While technology doesn’t simplistically push people in a predetermined direction, it’s not naive techno-determinism to recognize its tendency to afford certain capabilities or see how it prompts some outcomes over others.
perhaps the single overriding characteristic of the coming wave is that it will democratize access to power.
The past wave enabled us to sequence, or read, DNA. The coming wave will make DNA synthesis universally available.
Democratizing access necessarily means democratizing risk.
The fully omni-use nature of the coming wave means it is found at every level, in every sector, every business, or subculture, or group, or bureaucracy, in every corner of our world.
It produces trillions of dollars in new economic value while also destroying certain existing sources of wealth. Some individuals are greatly enabled; others stand to lose everything. Militarily it empowers some nation-states and militias alike. This is not, then, confined to amplifying specific points of fragility; it is, in the slightly longer term, about a transformation of the very ground on which society is built.
Soon after the stirrup was introduced into Europe, Charles Martel, leader of the Franks, saw its potential. Using it to devastating effect, he defeated and expelled the Saracens from France.
The story of stirrups and feudalism highlights an important truth: new technologies help create new centers of power with new social infrastructures both enabling them and supporting them.
Technologies can reinforce social structures, hierarchies, and regimes of control as well as upend them.
This ungovernable “post-sovereign” world, in the words of the political scientist Wendy Brown, will go far beyond a sense of near-term fragility; it will be instead a long-term macro-trend toward deep instability grinding away over decades. The first result will be massive new concentrations of power and wealth that reorder society.
People often like to measure progress in AI by comparing it with how well an individual human can perform a certain task. Researchers talk about achieving superhuman performance in language translation, or on real-world tasks like driving. But what this misses is that the most powerful forces in the world are actually groups of individuals coordinating to achieve shared goals.
the combined revenues of companies in Fortune’s Global 500 are already at 44 percent of world GDP.
Unlike with rockets, satellites, and the internet, the frontier of this wave is found in corporations, not in government organizations or academic labs.
The world’s top fifty cities have the lion’s share of wealth and corporate power (45 percent of big company HQs; 21 percent of world GDP) despite having only 8 percent of the world’s population.
The top 10 percent of global firms take 80 percent of the total profits.
Samsung Group revenue represents up to 20 percent of the Korean economy.
In a few decades, I predict most physical products will look like services. Zero marginal cost production and distribution will make it possible. The migration to the cloud will become all-encompassing, and the trend will be spurred by the ascendancy of low-code and no-code software, the rise of bio-manufacturing, and the boom in 3-D printing. When you combine all the facets of the coming wave, from the design, management, and logistical capabilities of AI to the modeling of chemical reactions enabled by quantum computing to the fine-grained assembly capabilities of robotics, you get a
...more
Those with the resources to invent or adopt new technologies fastest—those that can pass my updated Turing test, for example—will enjoy rapidly compounding returns.
However, another inevitable reaction of nation-states will be to use the tools of the coming wave to tighten their grip on power, taking full advantage to entrench their dominance.
This is not some far-off dystopia. I’m describing daily reality for millions in a city like London.
Consider that a combination of AI, cheap robotics, and advanced biotech coupled with clean energy sources might, for the first time in modernity, make living “off-grid” nearly equivalent to being plugged-in.
Every individual, every business, every church, every nonprofit, every nation, will eventually have its own AI and ultimately its own bio and robotics capability.
Pandemics feature widely. Two killed up to 30 percent of the world population: the sixth-century Plague of Justinian and the fourteenth-century Black Death. England’s population was seven million in 1300, but by 1450, crushed by waves of the plague, it was down to just two million.
Over time, then, the implications of these technologies will push humanity to navigate a path between the poles of catastrophe and dystopia.
This is the essential dilemma of our age.
Solving the question of AI alignment doesn’t mean doing so once; it means doing it every time a sufficiently powerful AI is built, wherever and whenever that happens.
philosopher of technology Lewis Mumford talked about the “megamachine,” where social systems combine with technologies to form “a uniform, all-enveloping structure” that is “controlled for the benefit of depersonalized collective organizations.” In the name of security, humanity could unleash the megamachine to, literally, stop other megamachines from coming into being.
Might it be time, however improbable, to have a moratorium on technology itself?
Reminds me or makes me think of Dune or Silo.
A survey of sixty civilizations suggests they last about four hundred years on average before falling apart. Without new technologies, they hit hard limits to development—in available energy, in food, in social complexity—that bring them crashing down.
Unpalatable as it is to some, it’s worth repeating: solving problems like climate change, or maintaining rising living and health-care standards, or improving education and opportunity is not going to happen without delivering new technologies as part of the package.
Demand for lithium, cobalt, and graphite is set to rise 500 percent by 2030.
Maintaining, let alone improving, standards of living needs technology.
In 1955, toward the end of his life, the mathematician John von Neumann wrote an essay called “Can We Survive Technology?” Foreshadowing the argument here, he believed that global society was “in a rapidly maturing crisis—a crisis attributable to the fact that the environment in which technological progress must occur has become both undersized and underorganized.” At the end of the essay, von Neumann puts survival as only “a possibility,” as well he might in the shadow of the mushroom cloud his own computer had made a reality.
My profound worry is that technology is demonstrating the real possibility to sharply move net negative, that we don’t have answers to arrest this shift, and that we’re locked in with no way out.
The only coherent approach to technology is to see both sides at the same time.
As we’ve seen, governments face multiple crises independent of the coming wave—declining trust, entrenched inequality, polarized politics, to name a few. They’re overstretched, their workforces under-skilled and unprepared for the kinds of complex and fast-moving challenges that lie ahead.
Twenty years on from the dawn of social media, there’s no consistent approach to the emergence of a powerful new platform (and besides, is privacy, polarization, monopoly, foreign ownership, or mental health the core problem—or all of the above?). The coming wave will worsen this dynamic.
Talking about the ethics of machine learning systems is a world away from, say, the technical safety of synthetic bio. These discussions happen in isolated, echoey silos. They rarely break out.
It’s not enough to have dozens of separate conversations about algorithmic bias or bio-risk or drone warfare or the economic impact of robotics or the privacy implications of quantum computing. It completely underplays how interrelated both causes and effects are. We need an approach that unifies these disparate conversations, encapsulating all those different dimensions of risk, a general-purpose concept for this general-purpose revolution.
Right now, scattered insights are all we’ve got: hundreds of distinct programs across distant parts of the technosphere, chipping away at well-meaning but ad hoc efforts without an overarching plan or direction.
Whether it’s facing an emergent AGI or a strange but useful new life-form, the goal has to be unified: containment.
EU’s AI Act, first proposed in 2021. As of this writing in 2023, the act is going through the lengthy process of becoming European law. If it is enacted, AI research and deployment will be categorized on a risk-based scale. Technologies with “unacceptable risk” of causing direct harm will be prohibited.
