Life 3.0: Being Human in the Age of Artificial Intelligence

by Max Tegmark

For example, suppose a bunch of ants create you to be a recursively self-improving robot, much smarter than them, who shares their goals and helps them build bigger and better anthills, and that you eventually attain the human-level intelligence and understanding that you have now. Do you think you’ll spend the rest of your days just optimizing anthills, or do you think you might develop a taste for more sophisticated questions and pursuits that the ants have no ability to comprehend? If so, do you think you’ll find a way to override the ant-protection urge that your formicine creators endowed you with in much the same way that the real you overrides some of the urges your genes have given you? And in that case, might a ...

Perhaps there’s a way of designing a ...

improving AI that’s guaranteed to retain human-friendly goals forever, but I think it’s fair to say that we don’t yet know how to build one—or even whether it’s possible. In conclusion, the AI goal-alignment problem has three parts, none of which is solved and all of which are now the subject of active research. Since they’re so hard, it’s safest to start devoting our best efforts to them now, ...

For example, while Aristotle emphasized virtues, Immanuel Kant emphasized duties and utilitarians emphasized the greatest happiness for the greatest number.

For example, emphasis on beauty, goodness and truth traces back to both the Bhagavad Gita and Plato.

In his book A Beautiful Question, my colleague Frank Wilczek argues that truth is linked to beauty and that we can view our Universe as a work of art.

For example, our standards of male and female beauty may partly reflect our subconscious assessment of suitability for replicating our genes.

many ethical principles have

commonalities with social emotions such as empathy and compassion: they evolved to engender collaboration, and they affect our behavior through rewards and punishments.

If we do something mean and feel bad about it afterward, our emotional punishment is meted out directly by our brain chemistry. If we violate ethical principles, on the other hand, society may punish us in more indirect ways such as through info...

In other words, although humanity today is nowhere near an ethical consensus, there are many basic principles around which there’s broad agreement. This agreement isn’t surprising, because human societies that have survived until the present tend to have ethical principles that w...

four principles: Utilitarianism: Positive conscious experiences should be maximized and suffering should be minimized. Diversity: A diverse set of positive experiences is better than many repetitions of the same experience, even if the latter has been identified as the most positive experience possible. Autonomy: Conscious entities/societies should have the freedom to pursue their own goals unless this conflicts with an overriding principle.

Legacy: Compatibility with scenarios that most humans today would view as happy, incompatibility with scenarios that essentially all humans today would view as terrible.

beauty, joy, pleasure and suffering are subjective experiences.

The autonomy principle underlies many of the freedoms and rights spelled out in the Universal Declaration of Human Rights adopted by the United Nations in 1948 in an attempt to learn lessons from two world wars. This includes freedom of thought, speech and movement, freedom from slavery and torture, the right to life, liberty, security and education and the right to marry, work and own property. If we wish to be less anthropocentric, we can generalize this to the freedom to think, learn, communicate, own property and not be harmed, and the right to do whatever

doesn’t infringe on the freedoms of others.

A conscious entity has the freedom to think, learn, communicate, own property and not be harmed or destroyed. A conscious entity has the right to do whatever doesn’t conflict with the first law. Sounds good, no? But please ponder this for a moment. If animals are conscious, then what are predators supposed to eat? Must all your friends become vegetarians? If some sophisticated future computer programs turn out to be conscious, should it be illegal to terminate them? If there are rules against terminating digital life forms, then need there also be restrictions on creating them to avoid a digital population explosion?

Ultimate Goals? This chapter has been a brief history of goals. If we could watch a fast-forward replay of our 13.8-billion-year cosmic history, we’d witness several distinct stages of goal-oriented behavior:

Matter seemingly intent on maximizing its dissipation Primitive life seemingly trying to maximize its replication Humans pursuing not replication but goals related to pleasure, curiosity, compassion and other feelings that they’d evolved to help them replicate Machines built to help humans pursue their human goals

If these machines eventually trigger an intelligence explosion, then how will this history of goals ultimately end? Might there be a goal system or ethical framework that almost all entities converge to as they get ever more intell...

Some may even dismiss everything we call “human values” as nothing but a cooperation

protocol, helping us with the subgoal of collaborating more efficiently.

The orthogonality thesis is empowering by

telling us that the ultimate goals of life in our cosmos aren’t predestined, but that we have the freedom and power to shape them. It suggests that guaranteed convergence to a unique goal is to be found not in the future but in the past, when all life emerged with the single goal of replication.

As cosmic time passes, ever more intelligent minds get the opportunity to rebel and break free from this banal replication ...

We humans aren’t fully free in this sense, since many goals remain genetically hardwired into us, but AIs can enjoy this ultimate freedom of ...

Perhaps this freedom from

evolutionary biases can make AIs more ethical than humans in some deep sense:

moral philosophers such as Peter Singer have argued that most humans behave unethically for evolutionary reasons, for example by disc...

truly well-defined goal would specify how all particles in our Universe should be arranged at the end of time.

But it’s not clear that there exists a well-defined end of time in physics. If the particles are arranged in that way at an earlier time, that arrangement will typically not last. And what particle arrangement is preferable, anyway?

As we’ve explored above, the only reason that we humans have any preferences at all may be that we’re the solution to an evolutionary optimization problem. Thus all normative words in our human language, such as “delicious,” “fragrant,”

“beautiful,” “comfortable,” “interesting,” “sexy,” “meaningful,” “happy” and “good,” trace their origin to this evolutionary optimization: there is therefore no guarantee that a superintelligent AI would find them rigorously definable.

Even if the AI learned to accurately predict the preferences of some representative human, it wouldn’t be able to compute the goodness function for most particle arrangements: the vast majority of possible particle arrangements correspond to strange cosmic scenarios with no stars, planets or people whatso...

Here are a few examples of other quantities that one could strive to maximize and that may be rigorously definable in terms of particle arrangements: The fraction of all the matter in our Universe that’s in the form of a particular organism, say humans or E. coli (inspired by evolutionary inclusive-fitness maximization) The ability of an AI to predict the future, which AI researcher Marcus Hutter argues is a good measure of its intelligence What AI researchers Alex Wissner-Gross and Cameron Freer term causal entropy (a proxy for future opportunities), which they argue is the hallmark of intelligence

The computational capacity of our Universe The algorithmic complexity of our Universe (how many bits are needed to describe it) The amount of consciousness in our Universe (see next chapter)

THE BOTTOM LINE: The ultimate origin of goal-oriented behavior lies in the laws of physics, which involve optimization. Thermodynamics has the built-in goal of dissipation: to increase a measure of messiness that’s called entropy. Life is a phenomenon that can help dissipate (increase overall messiness) even faster by retaining or growing its complexity and replicating while increasing the messiness of its environment. Darwinian evolution shifts the goal-oriented behavior from dissipation to replication. Intelligence is the ability to accomplish

complex goals. Since we humans don’t always have the resources to figure out the truly optimal replication strategy, we’ve evolved useful rules of thumb that guide our decisions: feelings such as hunger, thirst, pain, lust and compassion. We therefore no longer have a simple goal such as replication; when our feelings conflict with the goal of our genes, we obey our feelings, as by using birth control. We’re building increasingly intelligent machines to help us accomplish our goals. Insofar as we build such machines to exhibit goal-oriented behavior, we strive to align the machine goals with ours. Aligning machine goals with our own involves three unsolved problems: making machines learn them, adopt them and retain

them. AI can be created to have virtually any goal, but almost any sufficiently ambitious goal can lead to subgoals of self-preservation, resource acquisition and curiosity to understand the world better—the former two may potentially lead a superintelligent AI to cause problems for humans, and the latter may prevent it from retaining the goals we give it. Although many broad ethical principles are agreed upon by most humans, it’s unclear how to apply them to other entities, such as non-human animals and future AIs. It’s unclear how to imbue a superintelligent AI with an ulti...

philo...

cannot imagine a consistent theory of everything that ignores consciousness. Andrei Linde, 2002 We should strive to grow consciousness itself—to generate bigger, brighter lights in an otherwise dark universe. Giulio Tononi, 2012

We face, in Nick Bostrom’s words, philosophy with a deadline.

Who Cares? Consciousness is controversial. If you mention the “C-word” to an AI researcher, neuroscientist or psychologist, they may roll their eyes. If they’re your mentor, they might instead take pity on you and try to talk you out of wasting your time on what they consider a hopeless and unscientific problem.

If you look up “consciousness” in the 1989 Macmillan Dictionary of Psychology, you’re informed that “Nothing worth reading has been written on it.”

If not, then would it be, in the words of the famous physicist Erwin Schrödinger, “a play before empty benches, not existing for anybody, thus quite properly speaking not existing”?

2 In other words, if we enable high-tech descendants that

we mistakenly think are conscious, would this be the ultimate zombie apocalypse, transforming our grand cosmic endowment into nothin...

What Is Consci...

there are many competing ones, including sentience, wakefulness, self-awareness, access to sensory input and ability to fuse information into a narrative.

if it feels like something to be you right now, then you’re conscious.