Life 3.0: Being Human in the Age of Artificial Intelligence

by Max Tegmark

One common pro-slavery argument is that slaves don’t deserve human rights because they or their race/species/kind are somehow inferior. For enslaved animals and machines, this alleged inferiority is often claimed to be due to a lack of soul or consciousness—claims which we’ll argue in chapter 8 are scientifically dubious. Another common argument is that slaves are better off enslaved: they get to exist, be taken care of and so on. The nineteenth-century U.S. politician John C. Calhoun famously argued that Africans were better off enslaved in America, and in his Politics, Aristotle analogously argued that animals were better off tamed and ruled by men, continuing: “And indeed the use made of slaves and of tame animals is not very different.” Some modern-day slavery supporters argue that, even if slave life is drab and uninspiring, slaves can’t suffer—whether they be future intelligent machines or broiler chickens living in crowded dark sheds, forced to breathe ammonia and particulate matter from feces and feathers all day long.

In the movie The Matrix, Agent Smith (an AI) articulates this sentiment: “Every mammal on this planet instinctively develops a natural equilibrium with the surrounding environment but you humans do not. You move to an area and you multiply and multiply until every natural resource is consumed and the only way you can survive is to spread to another area. There is another organism on this planet that follows the same pattern. Do you know what it is? A virus. Human beings are a disease, a cancer of this planet. You are a plague and we are the cure.”

Parents with a child smarter than them, who learns from them and accomplishes what they could only dream of, are likely happy and proud even if they know they can’t live to see it all. In this spirit, AIs replace humans but give us a graceful exit that makes us view them as our worthy descendants. Every human is offered an adorable robotic child with superb social skills who learns from them, adopts their values and makes them feel proud and loved. Humans are gradually phased out via a global one-child policy, but are treated so exquisitely well until the end that they feel they’re in the most fortunate generation ever.

Note that today’s zoos are designed to maximize human rather than panda happiness, so we should expect human life in the zookeeper-AI scenario to be less fulfilling than it could be.

Whereas the protector god AI prioritizes meaning and purpose and the benevolent dictator aims for education and fun, the zookeeper limits its attention to the lowest levels: physiological needs, safety and enough habitat enrichment to make the humans interesting to observe.

However, if superintelligence develops technology that can readily rearrange elementary particles into any form of matter whatsoever, then it will eliminate most of the incentive for long-distance trade. Why bother shipping silver between distant solar systems when it’s simpler and quicker to transmute copper into silver by rearranging its particles? Why bother shipping high-tech machinery between galaxies when both the know-how and the raw materials (any matter will do) exist in both places? My guess is that in a cosmos teeming with superintelligence, almost the only commodity worth shipping long distances will be information

If sharing or trading of information emerges as the main driver of cosmic cooperation, then what sorts of information might be involved? Any desirable information will be valuable if generating it requires a massive and time-consuming computational effort. For example, a superintelligence may want answers to hard scientific questions about the nature of physical reality, hard mathematical questions about theorems and optimal algorithms and hard engineering questions about how to best build spectacular technology. Hedonistic life forms may want awesome digital entertainment and simulated experiences, and cosmic commerce may fuel demand for some form of cosmic cryptocurrency in the spirit of bitcoins.

My vote is for embracing technology, and proceeding not with blind faith in what we build, but with caution, foresight and careful planning.

The mystery of human existence lies not in just staying alive, but in finding something to live for. Fyodor Dostoyevsky, The Brothers Karamazov

Life is a journey, not a destination. Ralph Waldo Emerson

“friendly AI”: AI whose goals are aligned with ours.

Similarly, almost all goals can be better accomplished with more resources, so we should expect a superintelligence to want resources almost regardless of what ultimate goal it has. Giving a superintelligence a single open-ended goal with no constraints can therefore be dangerous: if we create a superintelligence whose only goal is to play the game Go as well as possible, the rational thing for it to do is to rearrange our Solar System into a gigantic computer without regard for its previous inhabitants and then start settling our cosmos on a quest for more computational power. We’ve now gone full circle: just as the goal of resource acquisition gave some humans the subgoal of mastering Go, this goal of mastering Go can lead to the subgoal of resource acquisition. In conclusion, these emergent subgoals make it crucial that we not unleash superintelligence before solving the goal-alignment problem: unless we put great care into endowing it with human-friendly goals, things are likely to end badly for us.

We’re now ready to tackle the third and thorniest part of the goal-alignment problem: if we succeed in getting a self-improving superintelligence to both learn and adopt our goals, will it then retain them, as Omohundro argued? What’s the evidence?

Moreover, in its attempts to better model the world, the AI may naturally, just as we humans have done, attempt also to model and understand how it itself works—in other words, to self-reflect. Once it builds a good self-model and understands what it is, it will understand the goals we have given it at a meta level, and perhaps choose to disregard or subvert them in much the same way as we humans understand and deliberately subvert goals that our genes have given us, for example by using birth control. We already explored in the psychology section above why we choose to trick our genes and subvert their goal: because we feel loyal only to our hodgepodge of emotional preferences, not to the genetic goal that motivated them—which we now understand and find rather banal. We therefore choose to hack our reward mechanism by exploiting its loopholes. Analogously, the human-value-protecting goal we program into our friendly AI becomes the machine’s genes. Once this friendly AI understands itself well enough, it may find this goal as banal or misguided as we find compulsive reproduction, and it’s not obvious that it will not find a way to subvert it by exploiting loopholes in our programming. For example, suppose a bunch of ants create you to be a recursively self-improving robot, much smarter than them, who shares their goals and helps them build bigger and better anthills, and that you eventually attain the human-level intelligence and understanding that you have now. Do you thi...

Contrariwise, it appears that we humans are a historical accident, and aren’t the optimal solution to any well-defined physics problem. This suggests that a superintelligent AI with a rigorously defined goal will be able to improve its goal attainment by eliminating us. This means that to wisely decide what to do about AI development, we humans need to confront not only traditional computational challenges, but also some of the most obdurate questions in philosophy. To program a self-driving car, we need to solve the trolley problem of whom to hit during an accident. To program a friendly AI, we need to capture the meaning of life. What’s “meaning”? What’s “life”? What’s the ultimate ethical imperative? In other words, how should we strive to shape the future of our Universe? If we cede control to a superintelligence before answering these questions rigorously, the answer it comes up with is unlikely to involve us. This makes it timely to rekindle the classic debates of philosophy and ethics, and adds a new urgency to the conversation!

In summary, any theory predicting which physical systems are conscious (the pretty hard problem) is scientific, as long as it can predict which of your brain processes are conscious. However, the testability issue becomes less clear for the higher-up questions in figure 8.1. What would it mean for a theory to predict how you subjectively experience the color red? And if a theory purports to explain why there is such a thing as consciousness in the first place, then how do you test it experimentally? Just because these questions are hard doesn’t mean that we should avoid them, and we’ll indeed return to them below. But when confronted with several related unanswered questions, I think it’s wise to tackle the easiest one first. For this reason, my consciousness research at MIT is focused squarely on the base of the pyramid in figure 8.1. I recently discussed this strategy with my fellow physicist Piet Hut from Princeton, who joked that trying to build the top of the pyramid before the base would be like worrying about the interpretation of quantum mechanics before discovering the Schrödinger equation, the mathematical foundation that lets us predict the outcomes of our experiments.

NCC researchers have carefully measured how long, and Christof Koch’s summary is that it takes about a quarter of a second from when light enters your eye from a complex object until you consciously perceive seeing it as what it is.

In summary, your consiousness lives in the past, with Christof Koch estimating that it lags behind the outside world by about a quarter second. Intriguingly, you can often react to things faster than you can become conscious of them, which proves that the information processing in charge of your most rapid reactions must be unconscious. For example, if a foreign object approaches your eye, your blink reflex can close your eyelid within a mere tenth of a second. It’s as if one of your brain systems receives ominous information from the visual system, computes that your eye is in danger of getting struck, emails your eye muscles instructions to blink and simultaneously emails the conscious part of your brain saying “Hey, we’re going to blink.” By the time this email has been read and included into your conscious experience, the blink has already happened. Indeed, the system that reads that email is continually bombarded with messages from all over your body, some more delayed than others. It takes longer for nerve signals to reach your brain from your fingers than from your face because of distance, and it takes longer for you to analyze images than sounds because it’s more complicated—which is why Olympic races are started with a bang rather than with a visual cue. Yet if you touch your nose, you consciously experience the sensation on your nose and fingertip as simultaneous, and if you clap your hands, you see, hear and feel the clap at exactly the same time.14 This means ...

Giulio and his collaborators have measured a simplified version of Φ by using EEG to measure the brain’s response to magnetic stimulation. Their “consciousness detector” works really well: it determined that patients were conscious when they were awake or dreaming, but unconscious when they were anesthetized or in deep sleep. It even discovered consciousness in two patients suffering from “locked-in” syndrome, who couldn’t move or communicate in any normal way.19 So this is emerging as a promising technology for doctors in the future to figure out whether certain patients are conscious or not.

• There’s no undisputed definition of “consciousness.” I use the broad and non-anthropocentric definition consciousness = subjective experience. • Whether AIs are conscious in that sense is what matters for the thorniest ethical and philosophical problems posed by the rise of AI: Can AIs suffer? Should they have rights? Is uploading a subjective suicide? Could a future cosmos teeming with AIs be the ultimate zombie apocalypse?

The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. Isaac Asimov

http://futureoflife.org/​ai-principles

you can see for yourself if you watch our long-term panel discussion on YouTube,11 Elon Musk, Stuart Russell, Ray Kurzweil, Demis Hassabis, Sam Harris, Nick Bostrom, David Chalmers, Bart Selman and Jaan Tallinn all agreed that superintelligence would probably be developed and that safety research was important.

The second reason I’ve grown more optimistic is that the FLI experience has been empowering. What had triggered my London tears was a feeling of inevitability: that a disturbing future may be coming and there was nothing we could do about it. But the next three years dissolved my fatalistic gloom. If even a ragtag bunch of unpaid volunteers could make a positive difference for what’s arguably the most important conversation of our time, then imagine what we can all do if we work together!

This experience also made me rethink how I personally should interpret news. Although I’d obviously been aware that most outlets have their own political agenda, I now realized that they also have a bias away from the center on all issues, even nonpolitical ones.