Rationality: From AI to Zombies

by Eliezer Yudkowsky

I like having a Creator I can outwit. Beats being a pet.

“You just said that evolution can’t plan simultaneous changes, and that evolution is very inefficient because mutations are random. Isn’t that what the creationists say? That you couldn’t assemble a watch by randomly shaking the parts in a box?”

But the reply to creationists is not that you can assemble a watch by shaking the parts in a box. The reply is that this is not how evolution works. If you think that evolution does work by whirlwinds assembling 747s, then the creationists have successfully misrepresented biology to you; they’ve sold the strawman.

Thank you

The real answer is that complex machinery evolves either incrementally, or by adapting previous complex machinery used for a new purpose. Squirrels jump from treetop to treetop using just their muscles, but the length they can jump depends to some extent on the aerodynamics of their bodies. So now there are flying squirrels, so aerodynamic they can glide short distances. If birds were wiped out, the descendants of flying squirrels might reoccupy that ecological niche in ten million years, gliding membranes transformed into wings.

First comes some gene A which is simple, but at least a little useful on its own, so that A increases to universality in the gene pool. Now along comes gene B, which is only useful in the presence of A, but A is reliably present in the gene pool, so there’s a reliable selection pressure in favor of B. Now a modified version of A* arises, which depends on B, but doesn’t break B’s dependency on A∕A*. Then along comes C, which depends on A* and B, and B*, which depends on A* and C. Soon you’ve got “irreducibly complex” machinery that breaks if you take out any single piece.

Gene pool

And yet you can still visualize the trail backward to that single piece: you can, without breaking the whole machine, make one piece less dependent on another piece, and do this a few times, until you can take out one whole piece without breaking the machine, and so on until you’ve turned a ticking watch back into a crude sundial.

Interesting. But break stuff apart still doesn't solve problems

DNA stores information very nicely, in a durable format that allows for exact duplication. A ribosome turns that stored information into a sequence of amino acids, a protein, which folds up into a variety of chemically active shapes. The combined system, DNA and ribosome, can build all sorts of protein machinery.

What was the very first replicator ever to exist? It may well have been an RNA strand, because by some strange coincidence, the chemical ingredients of RNA are chemicals that would have arisen naturally on the prebiotic Earth of 4 billion years ago.

Please note: evolution does not explain the origin of life; evolutionary biology is not supposed to explain the first replicator, because the first replicator does not come from another replicator.

The first replicator probably copied itself like a drunken monkey on LSD. It would have exhibited none of the signs of careful fine-tuning embodied in modern replicators, because the first replicator was an accident.

Wow

In this essay I describe some well-known inefficiencies and limitations of evolutions. I say “evolutions,” plural, because fox evolution works at cross-purposes to rabbit evolution, and neither can talk to snake evolution to learn how to build venomous fangs.

Human intelligence is so complicated that no one has any good way to calculate how efficient it is. Natural selection, though not simple, is simpler than a human brain; and correspondingly slower and less efficient, as befits the first optimization process ever to exist. In fact, evolutions are simple enough that we can calculate exactly how stupid they are.

A mutation conveying a 3% advantage (which is pretty darned large, as mutations go) has a 6% chance of spreading, at least on that occasion.2 Mutations can happen more than once, but in a population of a million with a copying fidelity of 10-8 errors per base per generation, you may have to wait a hundred generations for another chance, and then it still has only a 6% chance of fixating.

Then other evolutions don’t imitate it. If snake evolution develops an amazing new venom, it doesn’t help fox evolution or lion evolution. Contrast all this to a human programmer, who can design a new complex mechanism with a hundred interdependent parts over the course of a single afternoon. How is this even possible? I don’t know all the answer, and my guess is that neither does science; human brains are much more complicated than evolutions.

I could wave my hands and say something like “goal-directed backward chaining using combinatorial modular representations,” but you would not thereby be enabled to design your own human. Still: Humans can foresightfully design new parts in anticipation of later designing other new parts; produce coordinated simultaneous changes in interdependent machinery; learn by observing single test cases; zero in on problem spots and think abstractly about how to solve them; and prioritize which tweaks are worth trying, rather than waiting for a cosmic ray strike to produce a good one. By the standards of natural selection, this is simply magic.

Yes, some evolutionary handiwork is impressive even by comparison to the best technology of Homo sapiens. But our Cambrian explosion only started, we only really began accumulating knowledge, around . . . what, four hundred years ago? In some ways, biology still excels over the best human technology: we can’t build a self-replicating system the size of a butterfly.

So, once again: for a human to look to natural selection as inspiration on the art of design is like a sophisticated modern bacterium trying to imitate the first awkward replicator’s biochemistry. The first replicator would be eaten instantly if it popped up in today’s competitive ecology. The same fate would accrue to any human planner who tried making random point mutations to their strategies and waiting 768 iterations of testing to adopt a 3% improvement.

Does a beneficial mutation “sometimes spread, but not always”? Well, a psychic power would be a beneficial mutation, so you’d expect it to spread, right? Yet this is qualitative reasoning, not quantitative—if X is true, then Y is true; if psychic powers are beneficial, they may spread.

Qualitative quantitative

mutations which are only rarely useful are extremely unlikely to spread, and that it is practically impossible for complex adaptations to arise without constant use. If psychic powers really existed, we should expect to see everyone using them all the time—not just because they would be so amazingly useful, but because otherwise they couldn’t have evolved in the first place.

Psychic power

To sum up, if you have all of the following properties: Entities that replicate; Substantial variation in their characteristics; Substantial variation in their reproduction; Persistent correlation between the characteristics and reproduction; High-fidelity long-range heritability in characteristics; Frequent birth of a significant fraction of the breeding population; And all this remains true through many iterations . . . Then you will have significant cumulative selection pressures, enough to produce complex adaptations by the force of evolution.

Evolution doesn’t operate on particular individuals; individuals keep whatever genes they’re born with. Evolution operates on a reproducing population, a species, over time. There’s a natural tendency to think that if an Evolution Fairy is operating on the species, she must be optimizing for the species. But what really changes are the gene frequencies, and frequencies don’t increase or decrease according to how much the gene helps the species as a whole. As we shall later see, it’s quite possible for a species to evolve to extinction.

Main. Evolve to extinction

To see why this is surprising, consider that 1 male can impregnate 2, 10, or 100 females; it wouldn’t seem that you need the same number of males as females to ensure the survival of the species. This is even more surprising in the vast majority of animal species where the male contributes very little to raising the children—humans are extraordinary, even among primates, for their level of paternal investment. Balanced gender ratios are found even in species where the male impregnates the female and vanishes into the mist.

So while group selection ought to favor more girls, individual selection favors equal investment in male and female offspring. Looking at the statistics of a maternity ward, you can see at a glance that the quantitative balance between group selection forces and individual selection forces is overwhelmingly tilted in favor of individual selection in Homo sapiens.

Natural selection is really about gene frequencies. To get a complex adaptation, a machine with multiple dependent parts, each new gene as it evolves depends on the other genes being reliably present in its genetic environment. They must have high frequencies. The more complex the machine, the higher the frequencies must be.

In viruses, there’s the tension between individual viruses replicating as fast as possible, versus the benefit of leaving the host alive long enough to transmit the illness.

Since most mutations are detrimental, this gene complex is an advantage to its holders. Now you might wonder about beneficial mutations—they do happen occasionally, so wouldn’t the unmutable be at a disadvantage? But in a sexual species, a beneficial mutation that began in a mutable can spread to the descendants of unmutables as well. The mutables suffer from degenerate mutations in each generation; and the unmutables can sexually acquire, and thereby benefit from, any beneficial mutations that occur in the mutables. Thus the mutables have a pure disadvantage.

Multicellular organisms can only exist because they’ve evolved powerful internal mechanisms to outlaw evolution. If the cells start evolving, they rapidly evolve to extinction: the organism dies.

So praise not evolution for the solicitous concern it shows for the individual; nearly all of your ancestors are dead. Praise not evolution for the solicitous concern it shows for a species; no one has ever found a complex adaptation which can only be interpreted as operating to preserve a species, and the mathematics would seem to indicate that this is virtually impossible.

You can’t even praise evolution for the solicitous concern it shows for genes; the battle between two alternative alleles at the same location is a zero-sum game for frequency.

The fundamental problem is that it’s not only restrained breeders who reap the benefits of restrained breeding. If some foxes refrain from spawning cubs who eat rabbits, then the uneaten rabbits don’t go to only cubs who carry the restrained-breeding adaptation. The unrestrained foxes, and their many more cubs, will happily eat any rabbits left unhunted.

The mathematical conditions for group selection overcoming individual selection were too extreme to be found in Nature. Why not create them artificially, in the laboratory? Michael J. Wade proceeded to do just that, repeatedly selecting populations of insects for low numbers of adults per subpopulation.2 And what was the result? Did the insects restrain their breeding and live in quiet peace with enough food for all? No; the adults adapted to cannibalize eggs and larvae, especially female larvae.

Interesting

Once you have that experimental result in hand—and it’s massively obvious in retrospect—then it suddenly becomes clear how the original group selectionists allowed romanticism, a human sense of aesthetics, to cloud their predictions of Nature.

Nature is not rational but it works just fine. Like it works weirdly great. #groupselection

They started out with the beautiful idea of fox populations voluntarily restraining their reproduction to what the rabbit population would bear, Nature in perfect harmony; then they searched for a reason why this would happen, and came up with the idea of group selection; then, since they knew what they wanted the outcome of group selection to be, they didn’t look for any less beautiful and aesthetic adaptations that group selection would be more likely to promote instead.

If they’d really been trying to calmly and neutrally predict the result of selecting for small subpopulation sizes resistant to famine, they would have thought of cannibalizing other organisms’ children or some similarly “ugly” outcome—long before they imagined anything so evolutionarily outré as individual restraint in breeding!

Interesting

J. R. Molloy said: “Nature is the ultimate bigot, because it is obstinately and intolerantly devoted to its own prejudices and absolutely refuses to yield to the most persuasive rationalizations of humans.”

Wow

Natural selection creates purposefulnesses which are alien to humans, and students of evolutionary theory are warned accordingly. It’s good training for any thinker, but it is especially important if you want to think clearly about other weird mindish processes that do not work like you do.

why God’s infinite love for all His children requires burning some of them at the stake.

Lol

The future cannot cause the past. But the designer’s brain, as an actually existent thing within the past, can indeed be the cause of the screwdriver.

The designer may intend it to turn screws. A murderer may buy it to use as a weapon. And then accidentally drop it, to be picked up by a child, who uses it as a chisel. So the screwdriver’s cause, and its shape, and its consequence, and its various meanings, are all different things; and only one of these things is found within the screwdriver itself.

Cognitive causes are made of neurons. Evolutionary causes are made of ancestors.

They have sex, and shower, and go their separate ways. The main objective consequence is to keep the bar and the hotel and the condom-manufacturer in business; which was not the cognitive purpose in their minds, and has virtually nothing to do with the key statistical regularities of reproduction 50,000 years ago which explain how they got the genes that built their brains that executed all this behavior.

But much stronger—nearly perfect, in fact—was the correlation between the grief curves of these modern Canadians and the reproductive-potential curve of a hunter-gatherer people,

Robert Wright, The Moral Animal,

Book

A candy bar matches taste buds that evolved in a hunter-gatherer environment, but it matches those taste buds much more strongly than anything that actually existed in the hunter-gatherer environment.

If people have the right to be tempted—and that’s what free will is all about—the market is going to respond by supplying as much temptation as can be sold.

Interesting

Consider how our standards of product-selling feminine beauty have changed since the advertisements of the 1950s. And as candy bars demonstrate, the market incentive also continues well beyond the point where the superstimulus begins wreaking collateral damage on the consumer.

Development til damage

Our limited willpower evolved to deal with ancestral temptations; it may not operate well against enticements beyond anything known to hunter-gatherers.

DNA constructs protein brains with reward signals that have a long-distance correlation to reproductive fitness, but a short-distance correlation to organism behavior. You don’t have to figure out that eating sugary food in the fall will lead to digesting calories that can be stored fat to help you survive the winter so that you mate in spring to produce offspring in summer. An apple simply tastes good, and your brain just has to plot out how to get more apples off the tree.

And so organisms evolve rewards for eating, and building nests, and scaring off competitors, and helping siblings, and discovering important truths, and forming strong alliances, and arguing persuasively, and of course having sex . . .