More on this book
Community
Kindle Notes & Highlights
You can’t walk someone through the decision-making process to explain precisely why an algorithm produced a specific prediction.
The AI researcher Stuart Russell calls it the “gorilla problem”: gorillas are physically stronger and tougher than any human being, but it is they who are endangered or living in zoos; they who are contained. We, with our puny muscles but big brains, do the containment.
By creating something smarter than us, we could put ourselves in the position of our primate cousins.
Ultimately, in its most dramatic forms, the coming wave could mean humanity will no longer be at the top of the food chain. Homo technologicus may end up being threatened by its own creation. The real question is not whether the wave is coming. It clearly is; just look and you can see it forming already. Given risks like these, the real question is why it’s so hard to see it as anything other than inevitable.
Technology is pushed on by all too rudimentary and fundamentally human drivers. From curiosity to crisis, fortune to fear, at its heart technology emerges to fill human needs. If people have powerful reasons to build and use it, it will get built and used.
China has a growing and impressive share of the most highly cited papers in AI. In terms of volume of AI research, Chinese institutions have published a whopping four and a half times more AI papers than U.S. counterparts since 2010,
Micius’s lead researcher and one of the world’s top quantum scientists, Pan Jianwei, made clear what this means. “I think we have started a worldwide
quantum space race,” he said. “With modern information science, China has been a learner and a follower. Now, with quantum technology, if we try our best we can be one of the main players.”
Xi Jinping made a speech with lasting consequences for China—and for the rest of the world. “Advanced technology is the sharp weapon of the modern state,” he declared. “Our technology still generally lags that of developed countries and we must adopt an asymmetric strategy of catching up and overtaking.”
Amazon’s R&D budget alone is $78 billion,
The future is remarkably open-source, published on arXiv, documented on GitHub.
The railway boom of the 1840s was “arguably the greatest bubble in history.” But in the annals of technology, it is more norm than exception. There was nothing inevitable
about the coming of the railways, but there was something inevitable about the chance to make money.
Science has to be converted into useful and desirable products for it to truly spread far and wide. Put simply: most technology is made to earn money.
The sheer breadth of human wants and needs, and the countless opportunities to profit from them, are integral to the story of technology
and will remain so in the future.
This engine has created a world economy worth $85 trillion—and counting.
Once an innovation delivers a competitive advantage like this, everyone must either adopt it, leapfrog it, switch focus, or lose market share and eventually go bust. The attitude around this dynamic in technology businesses in particular is simple and ruthless: build the next generation of technology or be destroyed.
Tech is by far the biggest single category in the S&P 500, constituting 26 percent of the index.
Within the next couple of years, whatever your job, you will be able to consult an on-demand expert, ask
it about your latest ad campaign or product design, quiz it on the specifics of a legal dilemma, isolate the most effective elements of a pitch, solve a thorny logistical question, get a second opinion on a diagnosis, keep probing and testing, getting ever more detailed answers grounded in the very cutting edge of knowledge, delivered with exceptional nuance. All of the world’s knowledge, best practices, precedent, and computational power
will be available, tailored to you, to your specific needs and circumstances, instantaneously and effortlessly. It is a leap in cognitive potential at least a...
This highlight has been truncated due to consecutive passage length restrictions.
The energy scholar Vaclav Smil calls ammonia, cement, plastics, and steel the four pillars of modern civilization:
the material base underwriting modern society, each hugely carbon-intensive to produce, with no obvious successors.
UN’s Intergovernmental Panel on Climate Change have been clear: carbon capture and storage is an essential technology.
rebuilding the entire infrastructure of modern society while hopefully also offering quality-of-life improvements to billions.
There’s a strong moral case for new technologies beyond profit or advantage.
A school of naive techno-solutionism sees technology as the answer to all of the world’s problems. Alone, it’s not. How
it is created, used, owned, and managed all make a difference.
Scientists and technologists are all too human. They crave status, success, and a legacy. They want to be the first and best and recognized as such. They’re competitive and clever with a carefully nurtured sense of their place in the world and in history.
“When you see something that is technically sweet, you go ahead and do it, and you argue about what to do about it only after you have had
your technical success.”
Hungarian American John von Neumann. “What we are creating now,” he said, “is a monster whose influence is going to change history, provided there is any history left, yet it would be impossible not to see it through, not only for military reasons, but it would als...
This highlight has been truncated due to consecutive passage length restrictions.
they know is feasible, no matter what terrible conseque...
This highlight has been truncated due to consecutive passage length restrictions.
Making history, doing something that matters, helping others, beating others, impressing a prospective partner, impressing a boss, peers, rivals: it’s all in there, all part of the ever-present drive to take risks, explore the edges, go further into the unknown. Build something new. Change the game. Climb the mountain.
Find a successful scientist or technologist and somewhere in there you will see someone driven by raw ego, spurred on by emotive impulses that might sound base or even unethical but are nonetheless an under-recognized part of why we get the technologies we do.
Altruism and curiosity, arrogance and competition, the desire to win the race, make your name, save your people, help the world, whatever it may be: these are what propel the wave on, and these cannot be expunged or circumvented.
A nested series of sub-races, in other words, adds up to a complex, mutually reinforcing dynamic.
Everything leaks. Everything is copied, iterated, improved. And because everyone is watching and learning from everyone else, with so many people all scratching around in the same areas, someone is inevitably going to figure out the next big breakthrough. And they will have no hope of containing it, for even if they do, someone else will come behind them and uncover the same insight or find an adjacent way of doing the same thing;
they will see the strategic potential or profit or prestige and go after it.
Our institutions for addressing massive
global problems were not fit for purpose.
As the historian of technology Langdon Winner puts it, “Technology in its various manifestations is a significant part of the human world. Its structures, processes, and alterations enter into and become part of the structures, processes, and alterations of human consciousness, society, and politics.” In other words, technology is political.
Supplanting more fluid oral traditions, the printed word fixed geography, knowledge, and history in place, promulgating set legal codes and ideologies.
Just as the cannon and the printing press upended society, so we should expect the same from technologies
like AI, robotics, and synthetic biology.
Pause for a moment and imagine a world where robots with the dexterity of human beings that can be “programmed” in plain English are ...
This highlight has been truncated due to consecutive passage length restrictions.
On another, unthinking adoption of some aspects of the coming wave opens pathways to domineering state control, creating supercharged Leviathans whose power goes beyond even history’s most extreme totalitarian governments. Authoritarian
Both failing states and authoritarian regimes are disastrous outcomes, not just on their own terms, but also for governing technology; neither flailing bureaucracies, populist opportunists, nor all-powerful dictators are people you’d want to be fundamentally responsible for controlling powerful new technologies.
Neither direction can or will contain the coming wave.
