The Coming Wave: AI, Power, and Our Future

by Mustafa Suleyman

At its heart, the nation-state, the central unit of the world’s political order today, offers its citizens a simple and highly persuasive bargain: that not only is centralization of power in the sovereign, territorial state possible but its benefits far outweigh the risks. History suggests that a monopoly over violence—that is, entrusting the state with wide latitude to enforce laws and develop its military powers—is the surest way to enable peace and prosperity. That, moreover, a well-managed country is a key foundation of economic growth, security, and well-being. Over the last five hundred years centralizing power in a singular authority has been essential to keeping the peace, unleashing the creative talents of billions of people to work hard, seek out education, invent, trade, and, in doing so, drive progress. Even as it grows more powerful and entangled with everyday life, the grand bargain of the nation-state, therefore, is that not only can centralized power enable peace and prosperity, but this power can be contained using a series of checks, balances, redistributions, and institutional forms. We often take for granted the delicate balance that has to be struck between extremes to maintain this. On the one hand the most dystopian excesses of centralized power must be avoided, and on the other we must accept regular intervention to maintain order. Today, more so than at any time in history, the technologies of the coming wave threaten to unsettle this fragile equil...

Sovereignty Under Strain: Technology vs. the Grand Bargain

Technological breakthroughs will help us meet the challenges hinted at in the last section: grow food amid unsustainable temperatures; detect floods, earthquakes, and fires ahead of time; and increase the standard of living for everyone. At a time of spiraling costs and deteriorating services, I see AI and synthetic biology as critical levers to help accelerate progress. They will make health care both higher-quality and more affordable. They will help us invent tools to bring about the transition to renewable energy and combat climate change at a time when politics has stalled, and support teachers, helping to increase the effectiveness of underfunded education systems. This is the real potential of the next wave.

Applied Hope: AI & SynBio for Health, Energy, and Education

Global living conditions are objectively better today than at any time in the past. We take running water and plentiful food supplies for granted. Most people enjoy warmth and shelter all year round. Literacy rates, life expectancy, and gender equality sit at all-time highs. The sum of thousands of years of human scholarship and inquiry is available at the touch of a button. For most people in developed countries, life is marked by an ease and abundance that would have seemed unbelievable in bygone eras. And yet, under the surface, there’s a nagging feeling that something isn’t quite right. Western societies in particular are mired in a deep-seated anxiety; they are “nervous states,” impulsive and fractious. This persistent unease is partly the result of previous shocks—multiple financial crises, the pandemic, violence (everything from 9/11 to the Ukraine war)—and partly the effect of long-term and growing pressures like declining public trust, rising inequality, and a warming climate.

Ease and Edginess: Prosperity in a Nervous Age

Trust in government, particularly in America, has collapsed. Postwar presidential administrations like those of Eisenhower and Johnson were trusted to do “what is right” by more than 70 percent of Americans, according to a Pew survey. For recent presidents such as Obama, Trump, and Biden, this measure of confidence has cratered, all falling below 20 percent. Quite remarkably, a 2018 study of democracy in America found that as many as one in five believe “army rule” is a good idea! No less than 85 percent of Americans feel the country is “heading in the wrong direction.” Distrust extends to nongovernment institutions, with growing levels of distrust in the media, the scientific establishment, and the idea of expertise in general. The problem is not limited to the United States. Another Pew survey found that across twenty-seven countries, a majority were dissatisfied with their democracies. A Democracy Perception Index poll found that across fifty nations two-thirds of respondents felt the government “rarely” or “never” acted in the public interest. That so many people profoundly feel society is failing is itself a problem: Distrust breeds negativity and apathy. People decline to vote. Since 2010, more countries have slid backward on measures of democracy than have progressed, a process that appears to be accelerating. Rising nationalism and authoritarianism seem endemic, from Poland and China to Russia, Hungary, the Philippines, and Turkey. Populist movements range from the...

Trust on Empty: Inequality, Disillusion, and the Authoritarian Temptation

War, peace, commerce, political order, culture—these have always been fundamentally interlinked, and interlinked moreover with technology. Technologies are ideas, manifested in products and services that have profound and lasting consequences for people, social structures, the environment, and everything in between. Modern technology and the state evolved symbiotically, in constant dialogue. Think of how technology facilitated the state’s core working parts, helping construct the edifice of national identity and administration. Writing was invented as an administrative and accounting tool to keep track of debts, inheritances, laws, taxes, contracts, and records of ownership. The clock produced set times, first in limited spaces like monasteries but then in mechanical form across late medieval mercantile cities and eventually across nations, creating common, and ever larger, social units. The printing press helped standardize national languages from a chaos of dialects and thus helped produce a national “imagined community,” the unitary people behind a nation-state. Supplanting more fluid oral traditions, the printed word fixed geography, knowledge, and history in place, promulgating set legal codes and ideologies. Radio and TV turbocharged this process, creating moments of national and even international commonality experienced simultaneously, like FDR’s fireside chats or the World Cup. Weapons, too, are technologies central to the power wielded by nation-states. Indeed, t...

Cannons to Code: How Technology Rewrites the State

Power is “the ability or capacity to do something or act in a particular way;…to direct or influence the behavior of others or the course of events.” It’s the mechanical or electrical energy that underwrites civilization. The bedrock and central principle of the state. Power in one form or another shapes everything. And it, too, is about to be transformed. Technology is ultimately political because technology is a form of power. And perhaps the single overriding characteristic of the coming wave is that it will democratize access to power.

Power, Downloaded: Technology’s Democratizing Wave

In November 2020, Mohsen Fakhrizadeh was the head scientist and linchpin of Iran’s long effort to attain nuclear weapons. Patriotic, dedicated, highly experienced, he was a prime target for Iran’s adversaries. Cognizant of the risks, he kept his whereabouts and movements cloaked in secrecy with help from Iran’s security services. Driving in a heavily guarded convoy down a dusty road to his country house near the Caspian Sea, Fakhrizadeh’s motorcade suddenly screeched to a halt. The scientist’s vehicle was hit by a barrage of bullets. Wounded, Fakhrizadeh stumbled out of his car, only to be killed by a second burst of machine-gun fire that tore through him. His bodyguards, members of Iran’s Revolutionary Guard, scrambled to make sense of what was happening. Where was the shooter? A few moments later there was an explosion, and a nearby pickup truck erupted into flames. The truck, however, was empty save for a gun. There were no assassins on the ground that day. In the words of a New York Times investigation, this was a “debut test of a high-tech, computerized sharpshooter kitted out with artificial intelligence and multiple-camera eyes, operated via satellite and capable of firing 600 rounds a minute.” Mounted on a strategically parked but innocuous-looking pickup truck fitted with cameras, it was a kind of robot weapon assembled by Israeli agents. A human authorized the strike, but it was the AI that automatically adjusted the gun’s aim. Just fifteen bullets were fired and...

No Gunman in Sight: Assassination by Algorithm

Early in the COVID-19 pandemic a blizzard of disinformation had deadly consequences. A Carnegie Mellon study analyzed more than 200 million tweets discussing COVID-19 at the height of the first lockdown. Eighty-two percent of influential users advocating for “reopening America” were bots. This was a targeted “propaganda machine,” most likely Russian, designed to intensify the worst public health crisis in a century.

Algorithmic Agitprop: Botnets in a Pandemic

The rise of synthetic media at scale and minimal cost amplifies both disinformation (malicious and intentionally misleading information) and misinformation (a wider and more unintentional pollution of the information space) at once. Cue an “Infocalypse,” the point at which society can no longer manage a torrent of sketchy material, where the information ecosystem grounding knowledge, trust, and social cohesion, the glue holding society together, falls apart. In the words of a Brookings Institution report, ubiquitous, perfect synthetic media means “distorting democratic discourse; manipulating elections; eroding trust in institutions; weakening journalism; exacerbating social divisions; undermining public safety; and inflicting hard-to-repair damage on the reputation of prominent individuals, including elected officials and candidates for office.”

Truth on Trial: The Synthetic Media Infocalypse

Few areas of biology are as controversial as gain-of-function (GOF) research. Put simply, gain-of-function experiments deliberately engineer pathogens to be more lethal or infectious, or both. In nature, viruses usually trade off lethality for transmissibility. The more transmissible a virus, the less lethal it often is. But there is no absolute reason this must be so. One way of understanding how it might happen—that is, how viruses might become more lethal and transmissible at the same time—and how we might combat that is to, well, make it happen. That’s where gain-of-function research comes in. Researchers investigate disease incubation times, or how they evade vaccine resistance, or maybe how they can spread asymptomatically through a population. Work like this has been undertaken on diseases including Ebola, influenzas like H1N1, and measles. Such research efforts are generally credible and well intentioned. Work with an avian flu in Holland and the United States around a decade ago is a good example. This disease had shockingly high mortality rates but luckily was very difficult to catch. Researchers wanted to understand how that picture might change, how this disease could morph into a more transmissible form, and used ferrets to see how this might occur. In other words they made a deadly disease in principle easier to catch.

Teaching Viruses New Tricks: The Gain-of-Function Dilemma

Broadly speaking, when technology damaged old jobs and industries, it also produced new ones. Over time these new jobs tended toward service industry roles and cognitive-based white-collar jobs. As factories closed in the Rust Belt, demand for lawyers, designers, and social media influencers boomed. So far at least, in economic terms, new technologies have not ultimately replaced labor; they have in the aggregate complemented it. But what if new job-displacing systems scale the ladder of human cognitive ability itself, leaving nowhere new for labor to turn? If the coming wave really is as general and wide-ranging as it appears, how will humans compete? What if a large majority of white-collar tasks can be performed more efficiently by AI? In few areas will humans still be “better” than machines. I have long argued this is the more likely scenario. With the arrival of the latest generation of large language models, I am now more convinced than ever that this is how things will play out. These tools will only temporarily augment human intelligence. They will make us smarter and more efficient for a time, and will unlock enormous amounts of economic growth, but they are fundamentally labor replacing. They will eventually do cognitive labor more efficiently and more cheaply than many people working in administration, data entry, customer service (including making and receiving phone calls), writing emails, drafting summaries, translating documents, creating content, copywritin...

White-Collar Eclipse: The Coming Cognitive Displacement

Machines are rapidly imitating all kinds of human abilities, from vision to speech and language. Even without fundamental progress toward “deep understanding,” new language models can read, synthesize, and generate eye-wateringly accurate and highly useful text. There are literally hundreds of roles where this single skill alone is the core requirement, and yet there is so much more to come from AI. Yes, it’s almost certain that many new job categories will be created. Who would have thought that “influencer” would become a highly sought-after role? Or imagined that in 2023 people would be working as “prompt engineers”—nontechnical programmers of large language models who become adept at coaxing out specific responses? Demand for masseurs, cellists, and baseball pitchers won’t go away. But my best guess is that new jobs won’t come in the numbers or timescale to truly help. The number of people who can get a PhD in machine learning will remain tiny in comparison to the scale of layoffs. And, sure, new demand will create new work, but that doesn’t mean it all gets done by human beings. Labor markets also have immense friction in terms of skills, geography, and identity. Consider that in the last bout of deindustrialization the steelworker in Pittsburgh or the carmaker in Detroit could hardly just up sticks, retrain mid-career, and get a job as a derivatives trader in New York or a branding consultant in Seattle or a schoolteacher in Miami. If Silicon Valley or the City of Lo...

New Jobs, Narrow Doors: Labor Friction in the AI Era

Countries like India and the Philippines have seen a huge boom from business process outsourcing, creating comparatively high-paying jobs in places like call centers. It’s precisely this kind of work that will be targeted by automation. New jobs might be created in the long term, but for millions they won’t come quick enough or in the right places. At the same time, a jobs recession will crater tax receipts, damaging public services and calling into question welfare programs just as they are most needed. Even before jobs are decimated, governments will be stretched thin, struggling to meet all their commitments, finance themselves sustainably, and deliver services the public has come to expect. Moreover, all this disruption will happen globally, on multiple dimensions, affecting every rung of the development ladder from primarily agricultural economies to advanced service-based sectors. From Lagos to L.A., pathways to sustainable employment will be subject to immense, unpredictable, and fast-evolving dislocations. Even those who don’t foresee the most severe outcomes of automation still accept that it is on course to cause significant medium-term disruptions. Whichever side of the jobs debate you fall on, it’s hard to deny that the ramifications will be hugely destabilizing for hundreds of millions who will, at the very least, need to re-skill and transition to new types of work. Optimistic scenarios still involve troubling political ramifications from broken government fi...

From Call Centers to Crisis: Automation’s Global Jobs Shock

From the Mongols to the Mughals, for more than a thousand years the most powerful force in Asia was a traditional empire. By 1800 that had changed. It was rather a private company, owned by a relatively small number of shareholders, run by a handful of dusty accountants and administrators operating out of a building just five windows wide in a city thousands of miles away. At the turn of the nineteenth century, the British East India Company controlled huge swaths of the Indian subcontinent. It ruled more land and people than existed in all of Europe, collecting taxes and setting laws. It commanded a well-drilled standing army of 200,000 men, twice as large as Britain’s own army at home, and operated the world’s largest mercantile fleet. Its collective firepower was greater than that of any state in Asia. Its global trading relationships were fundamental in everything from the foundation of Hong Kong to the Boston Tea Party. Its customs, duties, and dividends were critical to the British economy; no less than half of Britain’s foreign trade at the time ran through the company. This was clearly no ordinary corporation. In truth it was a kind of empire.

Empire, Incorporated: The British East India Company

We are already in an era where megacorporations have trillion-dollar valuations and more assets, in every sense, than entire countries. Take Apple. It has produced one of the most beautiful, influential, and widely used products in the history of our species. The iPhone is genius. With its product used by more than 1.2 billion people worldwide, the company has deservedly collected rich rewards for its success: in 2022, Apple was valued at more than all the companies listed on the U.K.’s FTSE 100 stock exchange combined. With close to $200 billion of cash and investments in the bank and a captive audience largely locked into its ecosystem, Apple seems well placed to take advantage of this new wave.

iEmpire: When a Corporation Outgrows Countries

Similarly, a vast span of services, from very different sectors, across huge parts of the planet, have been collapsed into a single corporation, Google: mapping and location, reviews and business listings, advertising, video streaming, office tools, calendars, email, photo storage, videoconferencing, and so on. Big tech companies provide tools for everything from organizing a birthday to running multimillion-dollar businesses. The only equivalent organizations, touching so deeply into the lives of so many, are national governments. Call it “Googlization”: a range of services provided for free or at low cost leading to single entities functionally enabling massive sections of the economy and human experience.

Googlization: One Company, Many Worlds

I think we’ll see a group of private corporations grow beyond the size and reach of many nation-states. Consider the outsized influence of a sprawling corporate empire like the Samsung Group in South Korea. Founded as a noodle shop almost a century ago, it became a major conglomerate after the Korean War. As Korean growth accelerated in the 1960s and 1970s, Samsung was at the heart of it, not only a diversified manufacturing powerhouse but a major player in banking and insurance. The Korean economic miracle was a Samsung-powered miracle. By this point Samsung was the leading chaebol, the name given to a small group of massive firms dominating the country. Smartphones, semiconductors, and TVs are Samsung specialties. But so too are life insurance, ferry operators, and theme parks. Careers at Samsung are enormously prized. Samsung Group revenue represents up to 20 percent of the Korean economy. For Koreans today, Samsung is almost like a parallel government, a constant presence throughout people’s lives.

Beyond Borders, Beyond Ballots: Corporations as Parallel States

In the last wave, things dematerialized; goods became services. You don’t buy software or music on CDs anymore; it’s streamed. You just expect antivirus and security software as a by-product of using Google or Apple. Products break, get obsolete. Services less so. They are seamless and easy to use. For their part, companies are eager for you to subscribe to their software ecosystems; regular payments are alluring. All the big tech platforms either are mainly service businesses or have very large service businesses. Apple has the App Store, despite primarily selling devices, and Amazon, while operating as the world’s biggest retailer of physical goods, also provides e-commerce services to merchants and TV streaming to individuals, and hosts a good chunk of the internet on its cloud offering, Amazon Web Services. Everywhere you look, technology accelerates this dematerialization, reducing complexity for the end consumer by providing continuous consumption services rather than traditional buy-once products. Whether it’s services like Uber, DoorDash, and Airbnb, or open publishing platforms like Instagram and TikTok, the drift of mega-businesses is toward not participating in the market but being the market, not making the product but operating the service.

Dematerialized Dominance: When Platforms Become the Market

Whatever the end point, we are heading to a place where unprecedented powers and abilities are out there, in the hands of already powerful actors who’ll no doubt use them to amplify their reach and further their own agenda. Such concentrations will enable vast, automated megacorporations to transfer value away from human capital—work—and toward raw capital. Put all the inequalities resulting from concentration together, and it adds up to another great acceleration and structural deepening of an existing fracture. Little wonder there is talk of neo- or techno-feudalism—a direct challenge to the social order, this time built on something beyond even stirrups. In sum, returns on intelligence will compound exponentially. A select few artificial intelligences that we used to call organizations will massively benefit from a new concentration of ability—probably the greatest such concentration yet seen. Re-creating the essence of what’s made our species so successful into tools that can be reused and reapplied over and over, in myriad different settings, is a mighty prize, which corporations and bureaucracies of all kinds will pursue, and wield. How these entities are governed, how they will rub against, capture, and reengineer the state, is an open question. That they will challenge it seems certain.

The Lords of Compute: Techno-Feudal Power Consolidates

The coming wave will only deepen and recapitulate the exact same contradictory dynamics of the last wave. The internet does precisely this: centralizes in a few key hubs while also empowering billions of people. It creates behemoths and yet gives everyone the opportunity to join in. Social media created a few giants and a million tribes. Everyone can build a website, but there’s only one Google. Everyone can sell their own niche products, but there’s only one Amazon. And on and on. The disruption of the internet era is largely explained by this tension, this potent, combustible brew of empowerment and control.

Many Voices, Few Thrones

Catastrophes are also, of course, man-made. World War I killed around 1 percent of the global population; World War II, 3 percent. Or take the violence unleashed by Genghis Khan and the Mongol army across China and central Asia in the thirteenth century, which took the lives of up to 10 percent of the world’s population. With the advent of the atomic bomb, humanity now possesses enough lethal force to kill everyone on the planet several times over. Catastrophic events that once took place over years and decades could happen in minutes, at the push of a button.

Minutes to Midnight: Catastrophe at the Push of a Button

There is no instruction manual on how to build the technologies in the coming wave safely. We cannot build systems of escalating power and danger to experiment with ahead of time. We cannot know how quickly an AI might self-improve, or what would happen after a lab accident with some not yet invented piece of biotech. We cannot tell what results from a human consciousness plugged directly into a computer, or what an AI-enabled cyberweapon means for critical infrastructure, or how a gene drive will play out in the wild. Once fast-evolving, self-assembling automatons or new biological agents are released, out in the wild, there’s no rewinding the clock. After a certain point, even curiosity and tinkering might be dangerous. Even if you believe the chance of catastrophe is low, that we are operating blind should give you pause. Nor is building safe and contained technology in itself sufficient. Solving the question of AI alignment doesn’t mean doing so once; it means doing it every time a sufficiently powerful AI is built, wherever and whenever that happens. You don’t just need to solve the question of lab leaks in one lab; you need to solve it in every lab, in every country, forever, even while those same countries are under serious political strain. Once technology reaches a critical capability, it isn’t enough for early pioneers to just build it safely, as challenging as that undoubtedly is. Rather, true safety requires maintaining those standards across every single insta...

No Rewind: Safety at Planetary Scale

Looking at our vast cities, the sturdy civic buildings built of steel and stone, the great chains of roads and rails stitching them all together, the immense landscaping and engineering works that manage their environments, there’s a tempting sense of permanence exuded by our society. Despite the weightlessness of the digital world, there’s a solidness and a profusion to the material world around us. It shapes our everyday expectations. We go to the supermarket and expect it to be stuffed with fresh fruits and vegetables. We expect it to be kept cool in the summer, warm in the winter. Even despite constant turbulence, we assume that the supply chains and affordances of the twenty-first century are as robust as an old town hall. All the most historically extreme parts of our existence appear utterly banal, and so for the most part we carry on our lives as if they can go on indefinitely. Most of those around us, up to and including our leaders, do the same. And yet, nothing lasts forever. Throughout history societal collapses are legion: from ancient Mesopotamia to Rome, the Maya to Easter Island, again and again it’s not just that civilizations don’t last; it’s that unsustainability appears baked in. Civilizations that collapse are not the exception; they are the rule. A survey of sixty civilizations suggests they last about four hundred years on average before falling apart. Without new technologies, they hit hard limits to development—in available energy, in food, in soci...

The Illusion of Permanence: Technology’s Temporary Reprieve from Collapse

None of us can be sure how exactly all this unfolds. Within the broad parameters of the dilemma are an immense and unknowable range of specific outcomes. I am, however, confident that the coming decades will see complex, painful trade-offs between prosperity, surveillance, and the threat of catastrophe growing ever more acute. Even a system of states in the best possible health would struggle. We are facing the ultimate challenge for Homo technologicus. If this book feels contradictory in its attitude toward technology, part positive and part foreboding, that’s because such a contradictory view is the most honest assessment of where we are. Our great-grandparents would be astonished at the abundance of our world. But they would also be astonished at its fragility and perils. With the coming wave, we face a real threat, a cascade of potentially disastrous consequences—yes, even an existential risk to the species. Technology is the best and worst of us. There isn’t a neat one-sided approach that does it justice. The only coherent approach to technology is to see both sides at the same time. Over the last decade or so this dilemma has become even more pronounced, the task of tackling it more urgent. Look at the world and it seems that containment is not possible.

The Uncontainable Paradox

Regulating not just hyper-evolutionary but omni-use general-purpose technologies is incredibly challenging. Consider how motorized transport is regulated. There isn’t a single regulator, or even just a few laws. Instead, we have regulations around traffic, roads, parking, seatbelts, emissions, driver training, and so on. This comes not just from national legislatures but also from local governments, highway agencies, transport ministries issuing guidance, licensing bodies, offices of environmental standards. It relies not just on lawmakers but on police forces, traffic wardens, car companies, mechanics, city planners, and insurers. Complex regulations refined over decades made roads and vehicles incrementally safer and more ordered, enabling their growth and spread. And yet 1.35 million people a year still die in traffic accidents. Regulation may lessen the negative effects, but it can’t erase bad outcomes like crashes, pollution, or sprawl. We have decided that this is an acceptable human cost, given the benefits. That “we” is crucial. Regulation doesn’t just rely on the passing of a new law. It is also about norms, structures of ownership, unwritten codes of compliance and honesty, arbitration procedures, contract enforcement, oversight mechanisms. All of this needs to be integrated and the public needs to buy in.

Rules, Roads, and Reality: Why Regulation Mitigates but Can’t Eliminate Harm

For containment to be possible, rules need to work well in places as diverse as the Netherlands and Nicaragua, New Zealand and Nigeria. Where someone slows down, others will rush forward. Every country already brings its distinct legal and cultural customs to the development of technology. The EU heavily restricts genetically modified organisms in the food supply. Yet in the United States genetically modified organisms are a routine part of agribusiness. China, on the face of it, is a regulatory leader of sorts. The government has issued multiple edicts on AI ethics, seeking to impose wide-ranging restrictions. It proactively banned various cryptocurrencies and DeFi initiatives, and limits the time children under eighteen can spend on games and social apps to ninety minutes a day during the week, three hours on the weekend. Draft regulation of recommendation algorithms and LLMs in China far exceeds anything we’ve yet seen in the West. China is slamming on the brakes in some areas while also—as we’ve seen—charging ahead in others. Its regulation is matched by an unparalleled deployment of technology as a tool of authoritarian government power. Speak to Western defense and policy insiders and they’re adamant that although China talks a good game on AI ethics and limitations, when it comes to national security, there are no meaningful barriers. In effect, Chinese AI policy has two tracks: a regulated civilian path and a freewheeling military-industrial one.

Global Guardrails, Local Loopholes: China’s Dual Track

Earlier in the book I described containment as a foundation for controlling and governing technology, spanning technical, cultural, and regulatory aspects. At root, I believe this means having the power to drastically curtail or outright stop technology’s negative impacts, from the local and small scale up to the planetary and existential. Encompassing hard enforcement against misuse of proliferated technologies, it also steers the development, direction, and governance of nascent technologies. Contained technology is technology whose modes of failure are known, managed, and mitigated, a situation where the means to shape and govern technology escalate in parallel with its capabilities. It’s tempting to think of containment in an obvious, literal sense, a kind of magic box in which a given technology can be sealed away. At the outer limit—in the case of rogue malware or pathogens—such drastic steps might be needed. Generally, though, consider containment more as a set of guardrails, a way to keep humanity in the driver’s seat when a technology risks causing more harm than good. Picture those guardrails operating at different levels and with different modes of implementation. In the next chapter we’ll consider what they might look like at a more granular level, from AI alignment research to lab design, international treaties to best practice protocols. For now, the key point is that those guardrails need to be strong enough that, in theory, they could stop a runaway catastr...

Guardrails, Not Boxes: Containment That Scales

The original Apollo missions were expensive and onerous, but they showed the right immense level of ambition, and their can-do attitude in the face of daunting odds catalyzed the development of technologies from semiconductors and software to quartz clocks and solar panels.

Moonshot Dividends: Apollo’s Cascade of Innovation

Pandemic preparedness could, for example, be greatly enhanced by using low-wavelength lightbulbs that kill viruses. Giving off light with a wavelength between 200 and 230 nanometers, close to the ultraviolet spectrum, they can kill viruses while not penetrating the outer layer of the skin: a powerful weapon against pandemics and the spread of disease more widely. And if the COVID-19 pandemic taught us one thing, it’s the value of an integrated, accelerated approach across research, rollout, and regulation for novel vaccines.

Far-UVC & Fast Shots: A Playbook for Pandemic Readiness

how does an AI communicate when it might be wrong? One of the issues with LLMs is that they still suffer from the hallucination problem, whereby they often confidently claim wildly wrong information as accurate. This is doubly dangerous given they often are right, to an expert level. As a user, it’s all too easy to be lulled into a false sense of security and assume anything coming out of the system is true.

Confidence with Caution: Teaching AIs to Admit Uncertainty

In AI, the lion’s share of the most advanced GPUs essential to the latest models are designed by one company, the American firm NVIDIA. Most of its chips are manufactured by one company, TSMC, in Taiwan, the most advanced in just a single building, the world’s most sophisticated and expensive factory. TSMC’s machinery to make these chips comes from a single supplier, the Dutch firm ASML, by far Europe’s most valuable and important tech company. ASML’s machines, which use a technique known as extreme ultraviolet lithography and produce chips at levels of astonishing atomic precision, are among the most complex manufactured goods in history. These three companies have a choke hold on cutting-edge chips, a technology so physically constrained that one estimate argues they cost up to $10 billion per kilogram.

Atoms at $10B/kg: The NVIDIA–TSMC–ASML Chokehold

The physicist Richard Feynman famously said, “What I cannot create, I do not understand.” Today

To Build Is to Know

In the early days of jet engines, the 1950s, crashes—and fatalities—were worryingly common. By the early 2010s they were at just one death per 7.4 million passenger boardings. Years now go by with no fatal accidents whatsoever involving American commercial aircraft. Flying is just about the safest mode of transport there is: sitting thirty-five thousand feet in the sky is safer than sitting at home on your couch. Airlines’ impressive safety record comes down to numerous incremental technical and operational improvements over the years. But behind them is something just as important: culture. The aviation industry takes a vigorous approach to learning from mistakes at every level. Crashes are not just tragic accidents to mourn; they’re foundational learning experiences in determining how systems fail, opportunities for diagnosing problems, fixing them, and sharing that knowledge across the entire industry. Best practices are hence not corporate secrets, an edge over rival airlines: they’re enthusiastically implemented by competitors in the common interests of collective industry trust and safety.

From Wreckage to Wisdom: The Culture Behind Safe Skies

For millennia, the Hippocratic oath has been a moral lodestar for the medical profession. In Latin, Primum non nocere. First, do no harm. The Nobel Peace Prize winner and British-Polish scientist Joseph Rotblat, a man who left Los Alamos on the grounds of conscience, argued that scientists need something similar. Social and moral responsibility was, he believed, not something any scientist could ever set aside. I agree, and we should consider a contemporary version for technologists: ask not just what doing no harm means in an age of globe-spanning algorithms and edited genomes but how that can be enacted daily in what are often morally ambiguous circumstances. Precautionary principles like this are a good first step. Pause before building, pause before publishing, review everything, sit down and hammer out the second-, third-, nth-order impacts. Find all the evidence and look at it coldly. Relentlessly course correct. Be willing to stop. Do all this not just because it says so in some form, but because it’s what’s right, it’s what technologists do. Actions like this can’t just operate as laws or corporate mantras. Laws are only national, corporate mantras transitory, too often cosmetic. They must instead operate at a deeper level whereby the culture of technology is not that just-go-for-it “engineering mindset” but something more wary, more curious about what might happen. A healthy culture is one happy to leave fruit on the tree, say no, delay benefits for however long i...

First, Do No Harmware: Toward a Hippocratic Tech Culture

Too many visions of the future start with what technology can or might do and work from there. That’s completely the wrong foundation. Technologists should focus not just on the engineering minutiae but on helping to imagine and realize a richer, social, human future in the broadest sense, a complex tapestry of which technology is just one strand. Technology is central to how the future will unfold—that’s undoubtedly true—but technology is not the point of the future, or what’s really at stake. We are. Technology should amplify the best of us, open new pathways for creativity and cooperation, work with the human grain of our lives and most precious relationships. It should make us happier and healthier, the ultimate complement to human endeavor and life well lived—but always on our terms, democratically decided, publicly debated, with benefits widely distributed. Amid the turbulence, we must never lose sight of this: a vision even the most ardent of Luddites could embrace.

The Point Is People: Designing a Future on Our Terms

AI will no more produce a utopia than did steam or electricity. That’s a gross oversimplification. But new technology will be a critical piece of safeguarding the future and improving living standards. Too often the mistake of both critics and proponents of technology is to believe there is an easy answer. More technology, less technology. Acceleration versus deceleration. Utopia or dystopia. This is not how things work,

No Silver Bullets: Beyond Utopia vs. Doom

There is a voluminous academic debate over the precise definition of technology. In this book we go with a commonsense, everyday definition: the application of scientific knowledge (in the broadest possible sense) to produce tools or practical outcomes. However, the full, multifaceted complexity of the term is also acknowledged. Technology extends back into cultures and practices. It is not just transistors, screens, and keyboards. It is the explicit and tacit knowledge of coders, the social lives and societies that support them.

Beyond Hardware: Technology as Knowledge and Culture