More on this book
Community
Kindle Notes & Highlights
The eminent professor of complexity Melanie Mitchell rightly points out that present-day AI systems have many limitations: they can’t transfer knowledge from one domain to another, provide quality explanations of their decision-making process, and so on. Significant challenges with real-world applications linger, including material questions of bias and fairness, reproducibility, security vulnerabilities, and legal liability.
Today we have narrow or weak AI: limited and specific versions. GPT-4 can spit out virtuoso texts, but it can’t turn around tomorrow and drive a car, as other AI programs do. Existing AI systems still operate in relatively narrow lanes. What is yet to come is a truly general or strong AI capable of human-level performance across a wide range of complex tasks—able to seamlessly shift among them. But this is exactly what the scaling hypothesis predicts is coming and what we see the first signs of in today’s systems.
I believe the debate about whether and when the Singularity will be achieved is a colossal red herring. Debating timelines to AGI is an exercise in reading crystal balls. While obsessing about this one concept of superintelligence, people overlook the numerous nearer-term milestones being met with growing frequency.
For the time being, it doesn’t matter whether the system is self-aware, or has understanding, or has humanlike intelligence. All that matters is what the system can do. Focus on that, and the real challenge comes into view: systems can do more, much more, with every passing day.
In the words of the Stanford bioengineer Drew Endy, “Biology is the ultimate distributed manufacturing platform.” Synthetic biology’s true promise, then, is that it will “enable people to more directly and freely make whatever they need wherever they are.”
Theoretically, the entirety of the world’s data might be stored in just one kilogram of DNA.
The bio-revolution is coevolving with advances in AI,
Both fields are about re-creating, engineering these utterly foundational and interrelated concepts, two core attributes of humanity; change the view and they become one single project.
It likely won’t be too long before neural “laces” made from carbon nanotubes plug us directly into the digital world.
As with AI, robotics proved much more difficult in practice than early engineers assumed. The real world is a strange, uneven, unexpected, and unstructured environment, exquisitely sensitive to things like pressure: picking up an egg, an apple, a brick, a child, and a bowl of soup all require extraordinary dexterity, sensitivity, strength, and balance.
But as we’ve seen in so many other applications of machine learning, what starts with close human supervision ends up with the AI learning to do the task better by itself, eventually generalizing to new settings.
If AI represents the automation of information, robotics is the automation of the material, the physical instantiations of AI, a step change in what it is possible to do.
Like AI and biotech, quantum computing helps speed up other elements of the wave. And yet even the mind-bending quantum world is not the limit.
(Life + Intelligence) x Energy = Modern Civilization
Energy—expensive and dirty as it often is—is at present a limiter on technology’s rate of progress. Not for too much longer.
It may still be a decade or more, but a future with this clean and virtually limitless energy source is looking increasingly real.
As the elements of AI, advanced biotechnology, quantum computing, and robotics combine in new ways, prepare for breakthroughs like advanced nanotechnology, a concept that takes the ever-growing precision of technology to its logical conclusion.
In the words of the security expert Audrey Kurth Cronin, “Never before have so many had access to such advanced technologies capable of inflicting death and mayhem.”
Interlinked global systems are containment nightmares. And we already live in an age of interlinked global systems. In the coming wave a single point—a given program, a genetic change—can alter everything.
Just as today’s models produce detailed images based on a few words, so in decades to come similar models will produce a novel compound or indeed an entire organism with just a few natural language prompts.
a deep learning system might be designed for playing games yet capable of flying a fleet of bombers. The difference is not a priori obvious.
If AI is indeed the new electricity, then like electricity it will be an on-demand utility that permeates and powers almost every aspect of daily life, society, the economy: a general-purpose technology embedded everywhere. Containing something like this is always going to be much harder than containing a constrained, single-task technology, stuck in a tiny niche with few dependencies.
Previously creators could explain how something worked and why it did what it did, even if this required vast detail. That’s increasingly no longer true.
Many technologies and systems are becoming so complex that they’re beyond the capacity of any one individual to truly understand: quantum computing and other technologies operate toward the limits of what can be known.
gorillas are physically stronger and tougher than any human being, but it is they who are endangered or living in zoos; they who are contained. We, with our puny muscles but big brains, do the containment. By creating something smarter than us, we could put ourselves in the position of our primate cousins.
Technological rivalry is a geopolitical reality. Indeed it always has been. Nations feel the existential need to keep up with their peers. Innovation is power.
Just as Sputnik eventually put the United States on course to be a superpower in rocketry, space technology, computing, and all their military and civilian applications, so something similar is now taking place in China. AlphaGo was quickly labeled China’s Sputnik moment for AI.
In the words of Xi Jinping, speaking to the Twentieth CCP Congress in 2022, “to meet strategic needs” the country “must adhere to science and technology as the number-one productive force, talent as the number-one resource, [and] innovation as the number-one driving force.”
China is already ahead of the United States in green energy, 5G, and AI and is on a trajectory to overtake it in quantum and biotech in the next few years.
who builds, owns, and deploys technology matters.
Choosing to limit technological development when perceived adversaries pile forward is, in the logic of an arms race, choosing to lose.
there is no central authority controlling what technologies get developed, who does it, and for what purpose; technology is an orchestra with no conductor.
Raw curiosity, the quest for truth, the importance of openness, evidence-based peer review—these are core values for scientific and technological research.
The future is remarkably open-source, published on arXiv, documented on GitHub. It’s being built for citations, research kudos, and the promise of tenure.
no one can ever be sure of anything at the research frontier.
Science and technology research is inherently unpredictable, exceptionally open, and growing fast. Governing or controlling it is therefore immensely difficult.
At root, this is a story of systematically applying science and technology in the name of profit. This in turn drove huge leaps in output and living standards.
PwC forecasts AI will add $15.7 trillion to the global economy by 2030. McKinsey forecasts a $4 trillion boost from biotech over the same period. Boosting world robot installations 30 percent above a baseline forecast could unleash a $5 trillion dividend,
Little is ultimately more valuable than intelligence. Intelligence is the wellspring and the director, architect, and facilitator of the world economy.
In 1945, around 50 percent of the world’s population was seriously undernourished. Today, despite a population well over three times bigger, that’s down to 10 percent.
carbon capture and storage is an essential technology. And yet it’s largely not been invented or is still to be deployed at scale. To meet this global challenge, we’ll have to reengineer our agricultural, manufacturing, transport, and energy systems from the ground up with new technologies that are carbon neutral or probably even carbon negative.
A school of naive techno-solutionism sees technology as the answer to all of the world’s problems.
the idea that we can meet the century’s defining challenges without new technologies is completely fanciful.
Making history, doing something that matters, helping others, beating others, impressing a prospective partner, impressing a boss, peers, rivals: it’s all in there, all part of the ever-present drive to take risks, explore the edges, go further into the unknown. Build something new. Change the game. Climb the mountain.
It is the self-image technologists too often still aspire to, an archetype to emulate, a fantasy that still drives new technologies.
Everything leaks. Everything is copied, iterated, improved. And because everyone is watching and learning from everyone else, with so many people all scratching around in the same areas, someone is inevitably going to figure out the next big breakthrough.
On the one hand the most dystopian excesses of centralized power must be avoided, and on the other we must accept regular intervention to maintain order.
Global living conditions are objectively better today than at any time in the past. We take running water and plentiful food supplies for granted. Most people enjoy warmth and shelter all year round. Literacy rates, life expectancy, and gender equality sit at all-time highs.
Since 2010, more countries have slid backward on measures of democracy than have progressed, a process that appears to be accelerating. Rising nationalism and authoritarianism seem endemic, from Poland and China to Russia, Hungary, the Philippines, and Turkey. Populist movements range from the bizarre, like QAnon, to the directionless (the gilets jaunes in France), but from Bolsonaro in Brazil to Brexit in the U.K. their prominence on the world stage has been impossible to miss.
The last wave—the arrival of mainframes, desktop PCs and desktop software, the internet, and the smartphone—delivered immense benefits to society. It laid down the new tools for the modern economy, bolstering growth, transforming access to knowledge, to entertainment, and to one another.
