Complexity: The Emerging Science at the Edge of Order and Chaos

by M. Mitchell Waldrop

This balance point—often called the edge of chaos—is were the components of a system never quite lock into place, and yet never quite dissolve into turbulence, either. The edge of chaos is where life has enough stability to sustain itself and enough creativity to deserve the name of life. The edge of chaos is where new ideas and innovative genotypes are forever nibbling away at the edges of the status quo, and where even the most entrenched old guard will eventually be overthrown. The edge of chaos is where centuries of slavery and segregation suddenly give way to the civil rights movement of the 1950s and 1960s; where seventy years of Soviet communism suddenly give way to political turmoil and ferment; where eons of evolutionary stability suddenly give way to wholesale species transformation. The edge of chaos is the constantly shifting battle zone between stagnation and anarchy, the one place where a complex system can be spontaneous, adaptive, and alive.

“Instead of solving incredibly complicated equations, he taught me to keep simplifying the problem until you found something you could deal with. Look for what made a problem tick. Look for the key factor, the key ingredient, the key solution.”

a lot of people will attack it head on, like a battering ram. They will storm the gates and try to smash through the defenses with sheer intellectual power and brilliance. But Arthur has never felt that the battering ram approach was his strength. “I like to take my time as I think,” he says. “So I just camp outside the city. I wait. And I think. Until one day—maybe after I’ve turned to a completely different problem—the drawbridge comes down and the defenders say, ‘We surrender.’ The answer to the problem comes all at once.”

Prigogine’s central point was that self-organization depends upon self-reinforcement: a tendency for small effects to become magnified when conditions are right, instead of dying away.

Positive feedback seemed to be the sine qua non of change, of surprise, of life itself.

Increasing returns, lock-in, unpredictability, tiny events that have immense historical

But in the United States, the ideal is maximum individual freedom—or, as Arthur puts it, “letting everybody be their own John Wayne and run around with guns.” However much that ideal is compromised in practice, it still holds mythic power.

But the fascination with complexity went still deeper than that, says Cowan. In part because of their computer simulations, and in part because of new mathematical insights, physicists had begun to realize by the early 1980s that a lot of messy, complicated systems could be described by a powerful theory known as “nonlinear dynamics.” And in the process, they had been forced to face up to a disconcerting fact: the whole really can be greater than the sum of its parts.

Light is also a linear system, which is why you can still see the Walk/Don’t Walk sign across the street even on a sunny day: the light rays bouncing from the sign to your eyes are not smashed to the ground by sunlight streaming down from above.

Weather is an emergent property: take your water vapor out over the Gulf of Mexico and let it interact with sunlight and wind, and it can organize itself into an emergent structure known as a hurricane. Life is an emergent property, the product of DNA molecules and protein molecules and myriad other kinds of molecules, all obeying the laws of chemistry. The mind is an emergent property, the product of several billion neurons obeying the biological laws of the living cell. In fact, as Anderson pointed out in the 1972 paper, you can think of the universe as forming a kind of hierarchy: “At each level of complexity, entirely new properties appear. [And] at each stage, entirely new laws, concepts, and generalizations are necessary, requiring

inspiration and creativity to just as great a degree as in the previous one. Psychology is not applied biology, nor is biology applied chemistry.”

In particular, the founding workshops made it clear that every topic of interest had at its heart a system composed of many, many “agents.” These agents might be molecules or neurons or species or consumers or even corporations. But whatever their nature, the agents were constantly organizing and reorganizing themselves into larger structures through the clash of mutual accommodation and mutual rivalry. Thus, molecules would form cells, neurons would form brains, species would form ecosystems, consumers and corporations would form economies, and so on. At each level, new emergent structures would form and engage in new emergent behaviors. Complexity, in other words, was really a science of emergence. And the challenge that Cowan had been trying to articulate was to find the fundamental laws of emergence.

“They say that time heals,” he adds. “But that’s not quite true. It’s simply that the grief erupts less often.” As they walked along the roads and hillsides around the convent, Arthur couldn’t help but be intrigued by Kauffman’s concept of order and self-organization. The irony of it was that when Kauffman used the word “order,” he was obviously referring to the same thing that Arthur meant by the word “messiness”—namely emergence, the incessant urge of complex systems to organize themselves into patterns. But then, maybe it wasn’t so surprising that Kauffman was using exactly the opposite word; he was coming at the concept from exactly the opposite direction.

Okay, this was admittedly piling up a lot of ifs on top of ifs. But to Kauffman, this autocatalytic set story was far and away the most plausible explanation for the origin of life that he had ever heard. If it were true, it meant the origin of life didn’t have to wait for some ridiculously improbable event to produce a set of enormously complicated molecules; it meant that life could indeed have bootstrapped its way into existence from very simple molecules. And it meant that life had not been just a random accident, but was part of nature’s incessant compulsion for self-organization.

Kauffman felt that it might ultimately be possible to apply these ideas far beyond the economy. “I think these kinds of models are the place for contingency and law at the same time,” he says. “The point is that the phase transitions may be lawful, but the specific details are not. So maybe we have the starts of models of historical, unfolding processes for such things as the Industrial Revolution, for example, or the Renaissance as a cultural transformation, and why it is that an isolated society, or ethos, can’t stay isolated when you start plugging some new ideas into it.” You can ask the same thing about the Cambrian explosion: the period some 570 million years ago when a world full of algae and pond scum suddenly burst forth with complex, multicellular creatures in immense profusion. “Why all of a sudden do you get all this diversity?” Kauffman asks. “Maybe you had to get to a critical diversity to then explode. Maybe it’s because you’ve gone from algal mats to something that’s a little more trophic and complex, so that there’s an explosion of processes acting on processes to make new processes. It’s the same thing as in an economy.”

The only problem, of course, is that real human beings are neither perfectly rational nor perfectly predictable—as the physicists pointed out at great length. Furthermore, as several of them also pointed out, there are real theoretical pitfalls in assuming perfect predictions, even if you do assume that people are perfectly rational. In nonlinear systems—and the economy is most certainly nonlinear—chaos theory tells you that the slightest uncertainty in your knowledge of the initial conditions will often grow inexorably. After a while, your predictions are nonsense.

Furthermore, said Holland, the control of a complex adaptive system tends to be highly dispersed. There is no master neuron in the brain, for example, nor is there any master cell within a developing embryo. If there is to be any coherent behavior in the system, it has to arise from competition and cooperation among the agents themselves. This is true even in an economy. Ask any president trying to cope with a stubborn recession: no matter what Washington does to fiddle with interest rates and tax policy and the money supply, the overall behavior of the economy is still the result of myriad economic decisions made every day by millions of individual people.

the very act of filling one niche opens up more niches—for new parasites, for new predators and prey, for new symbiotic partners. So new opportunities are always being created by the system. And that, in turn, means that it’s essentially meaningless to talk about a complex adaptive system being in equilibrium: the system can never get there. It is always unfolding, always in transition. In fact, if the system ever does reach equilibrium, it isn’t just stable. It’s dead. And by the same token, said Holland, there’s no point in imagining that the agents in the system can ever “optimize” their fitness, or their utility, or whatever. The space of possibilities is too vast; they have no practical way of finding the optimum. The most they can ever do is to change and improve themselves relative to what the other agents are doing. In short, complex adaptive systems are characterized by perpetual novelty.

And suddenly Gell-Mann and the others realized that they’d left a gaping hole in their agenda: What do these emergent structures actually do? How do they respond and adapt to their environment? Within months they were talking about the institute’s program being not just complex systems, but complex adaptive systems.

Burks’ Communication Sciences program was the kind of environment where such questions could thrive. What is emergence? And what is thinking? How does it work? What are its laws? What does it really mean for a system to adapt? Holland jotted down reams of ideas on these questions, then systematically filed them away in manila folders labeled Glasperlenspiel 1, Glasperlenspiel 2, et cetera.

This game analogy seemed to be true of any adaptive system. In economics the payoff is in money, in politics the payoff is in votes, and on and on. At some level, all these adaptive systems are fundamentally the same. And that meant, in turn, that all of them are fundamentally like checkers or chess: the space of possibilities is vast beyond imagining. An agent can learn to play the game better—that’s what adaptation is, after all. But it has just about as much chance of finding the optimum, stable equilibrium point of the game as you or I have of solving chess.

Newell and Simon had concluded that problem-solving always involves a step-by-step mental search through a vast “problem space” of possibilities, with each step guided by a heuristic rule of thumb: “If this is the situation, then that step is worth taking.” By building their theory into a program known as General Problem Solver, and by putting that program to work on those same puzzles and games, Newell and Simon had shown that the problem-space approach could reproduce human-style reasoning remarkably well. Indeed, their concept of heuristic search was already well on its way to becoming the dominant conventional wisdom in artificial intelligence. And General Problem Solver stood—as it still stands—as one of the most influential programs in the young field’s history.

“All complex, adaptive systems—economies, minds, organisms—build models that allow them to anticipate the world,”

The theory says that these default-hierarchy models ought to emerge whether the principles are implemented as a classifier system or in some other way, says Holland. (In fact, many of the computer simulations quoted in Induction were done with PI, a somewhat more conventional rule-based program devised by Thagard and Holyoak.) Nonetheless, he says, it was thrilling to see the hierarchies actually emerge in Goldberg’s pipeline simulation. The classifier system had started with nothing. Its initial set of rules had been totally random, the computer equivalent of primordial chaos. And yet, here was this marvelous structure emerging out of the chaos to astonish and surprise them. “We were elated,” says Holland. “It was the first case of what someone could really call an emergent model.”

Here was this elusive “Santa Fe approach”: Instead of emphasizing decreasing returns, static equilibrium, and perfect rationality, as in the neoclassical view, the Santa Fe team would emphasize increasing returns, bounded rationality, and the dynamics of evolution and learning. Instead of basing their theory on assumptions that were mathematically convenient, they would try to make models that were psychologically realistic. Instead of viewing the economy as some kind of Newtonian machine, they would see it as something organic, adaptive, surprising, and alive. Instead of talking about the world as if it were a static thing buried deep in the frozen regime, as Chris Langton might have put it, they would learn how to think about the world as a dynamic, ever-changing system poised at the edge of chaos.

“Economics, as it is usually practiced, operates in a purely deductive mode,” he says. “Every economic situation is first translated into a mathematical exercise, which the economic agents are supposed to solve by rigorous, analytical reasoning. But then here were Holland, the neural net people, and the other machine-learning theorists. And they were all talking about agents that operate in an inductive mode, in which they try to reason from fragmentary data to a useful internal model.” Induction is what allows us to infer the existence of a cat from the glimpse of a tail vanishing around a corner. Induction is what allows us to walk through the zoo and classify some exotic feathered creature as a bird, even though we’ve never seen a scarlet-crested cockatoo before. Induction is what allows us to survive in a messy, unpredictable, and often incomprehensible world.

‘If things are not repeating, if things are not in equilibrium, what can we, as economists, say? How could you predict anything? How could you have a science?’” Holland took the question very seriously; he’d thought a lot about it. Look at meteorology, he told them. The weather never settles down. It never repeats itself exactly. It’s essentially unpredictable more than a week or so in advance. And yet we can comprehend and explain almost everything that we see up there. We can identify important features such as weather fronts, jet streams, and high-pressure systems. We can understand their dynamics. We can understand how they interact to produce weather on a local and regional scale. In short, we have a real science of weather—without full prediction. And we can do it because prediction isn’t the essence of science. The essence is comprehension and explanation. And that’s precisely what Santa Fe could hope to do with economics and other social sciences, he said: they could look for the analog of weather fronts—dynamical social phenomena they could understand and explain.

Use local control instead of global control. Let the behavior emerge from the bottom up, instead of being specified from the top down. And while you’re at it, focus on ongoing behavior instead of the final result. As Holland loved to point out, living systems never really settle down. Indeed, said Langton, by taking this bottom-up idea to its logical conclusion, you could see it as a new and thoroughly scientific version of vitalism: the ancient idea that life involves some kind of energy, or force, or spirit that transcends mere matter. The fact is that life does transcend mere matter, he said—not because living systems are animated by some vital essence operating outside the laws of physics and chemistry, but because a population of simple things following simple rules of interaction can behave in eternally surprising ways. Life may indeed be a kind of biochemical machine, he said. But to animate such a machine “is not to bring life to a machine; rather, it is to organize a population of machines in such a way that their interacting dynamics are ‘alive.’” Finally, said Langton, there was a third great idea to be distilled from the workshop presentations: the possibility that life isn’t just like a computation, in the sense of being a property of the organization rather than the molecules. Life literally is a computation.

Nonetheless, since the whole point of artificial life research was to grapple with the most fundamental principles of life, there was no avoiding the question: Could human beings ultimately create artificial life for real? Langton found that question to be a tough one, not least because neither he nor anyone else had a clear idea of what “real” artificial life would be like. Some kind of genetically engineered superorganism, perhaps? A self-reproducing robot? An overeducated computer virus? What is life, precisely? How do you know for sure when you’ve got it and when you haven’t?

“With the advent of artificial life,” they wrote, “we may be the first creatures to create our own successors. … If we fail in our task as creators, they may indeed be cold and malevolent. However, if we succeed, they may be glorious, enlightened creatures that far surpass us in their intelligence and wisdom. It is quite possible that, when the conscious beings of the future look back on this era, we will be most noteworthy not in and of ourselves but rather for what we gave rise to. Artificial life is potentially the most beautiful creation of humanity.”

Artificial life, he says, was where you could get right down into the deep questions of emergence and self-organization, questions that had haunted him all his life.

That’s one of the reasons I jumped into chaos.”

Emergence First, says Farmer, this putative law would have to give a rigorous account of emergence: What does it really mean to say that the whole is greater than the sum of its parts? “It’s not magic,” he says. “But to us humans, with our crude little human brains, it feels like magic.” Flying boids (and real birds) adapt to the actions of their neighbors, thereby becoming a flock. Organisms cooperate and compete in a dance of coevolution, thereby becoming an exquisitely tuned ecosystem. Atoms search for a minimum energy state by forming chemical bonds with each other, thereby becoming the emergent structures known as molecules. Human beings try to satisfy their material needs by buying, selling, and trading with each other, thereby creating an emergent structure known as a market. Humans likewise interact with each other to satisfy less quantifiable goals, thereby forming families, religions, and cultures. Somehow, by constantly seeking mutual accommodation and self-consistency, groups of agents manage to transcend themselves and become something more. The trick is to figure out how, without falling back into sterile philosophizing or New Age mysticism.

there is a wide variety of models to choose from. One that Farmer has given particular attention to is connectionism: the idea of representing a population of interacting agents as a network of “nodes” linked by “connections.” And in that, he has plenty of company. Connectionist models have been popping up everywhere in the past decade or so. Exhibit A has to be the neural network movement, in which researchers use webs of artificial neurons to model such things as perception and memory retrieval—and, not incidentally, to mount a radical attack on the symbol-processing methods of mainstream artificial intelligence.

self-organizing criticality only tells you about the overall statistics of avalanches; it tells you nothing about any particular avalanche. This is yet another case where understanding is not the same thing as prediction. The scientists who try to predict earthquakes may ultimately succeed, but not because of self-organized criticality. They’re in the same position as an imaginary group of tiny scientists living on a critical sand pile. These microscopic researchers can certainly perform a lot of detailed measurements on the sand grains in their immediate neighborhood, and—with a tremendous effort—predict when those particular sand grains are going to collapse. But knowing the global power-law behavior of the avalanches doesn’t help them a bit, because the global behavior doesn’t depend on the local details. In fact, it doesn’t even make any difference if the sand pile scientists try to prevent the collapse they’ve predicted. They can certainly do so by putting up braces and support structures and such. But they just end up shifting the avalanche somewhere else. The global power law stays the same.

“So maybe the lesson to be learned is that evolution hasn’t stopped,” says Langton. “It’s still going on, exhibiting many of the same phenomena it did in biological history—except that now it’s taking place on the social-cultural plane. And we may be seeing a lot of the same kinds of extinctions and upheaval.”

“For example, suppose that these models about the origin of life are correct. Then life doesn’t hang in the balance. It doesn’t depend on whether some warm little pond just happens to produce template-replicating molecules like DNA or RNA. Life is the natural expression of complex matter. It’s a very deep property of chemistry and catalysis and being far from equilibrium. And that means that we’re at home in the universe. We’re to be expected. How welcoming that is! How far that is from the image of organisms as tinkered-together contraptions, where everything is bits of widgetry piled on top of bits of ad hocery, and it’s all blind chance. In that world there are no deep principles in biology, other than random variation and natural selection; we’re not at home in the universe in the same way.

“And now suppose it’s really true that coevolving, complex systems get themselves to the edge of chaos,” he says. “Well, that’s very Gaia-like. It says that there’s an attractor, a state that we collectively maintain ourselves in, an ever-changing state where species are always going extinct and new ones are coming into existence. Or if we imagine that this really carries over into economic systems, then it’s a state where technologies come into existence and replace others, et cetera. But if this is true, it means that the edge of chaos is, on average, the best that we can do. The ever-open and ever-changing world that we must make for ourselves is in some sense as good as it possibly can be. “Well, that’s a story about ourselves,” says Kauffman. “Matter has managed to evolve as best it can. And we’re at home in the universe. It’s not Panglossian, because there’s a lot of pain. You can go extinct, or broke. But here we are on the edge of chaos because that’s where, on average, we all do the best.”

Kurt Gödel showed that even some very simple mathematical systems—arithmetic, for example—are inherently incomplete. They always contain statements that cannot be proved true or false within the system, even in principle. At about the same time (and by using essentially the same argument), the logician Alan Turing showed that even very simple computer programs can be undecidable: you can’t tell in advance whether the computer will reach an answer or not. In the 1960s and 1970s, physicists got much the same message from chaos theory: even very simple equations can produce results that are surprising and essentially unpredictable. Indeed, says Arthur, that message has been repeated in field after field. “People realized that logic and philosophy are messy, that language is messy, that chemical kinetics is messy, that physics is messy, and finally that the economy is naturally messy. And it’s not that this is a mess created by the dirt that’s on the microscope glass. It’s that this mess is inherent in the systems themselves. You can’t capture any of them and confine them to a neat box of logic.”

That’s certainly the kind of metaphor that Chris Langton has in mind with artificial life: his whole point is that complex, lifelike behavior is the result of simple rules unfolding from the bottom up. And it’s likewise the kind of metaphor that influenced Arthur in the Santa Fe economics program: “If I had a purpose, or a vision, it was to show that the messiness and the liveliness in the economy can grow out of an incredibly simple, even elegant theory. That’s why we created these simple models of the stock market where the market appears moody, shows crashes, takes off in unexpected directions, and acquires something that you could describe as a personality.” While he was actually at the institute, ironically, Arthur had almost no time at all for Chris Langton’s artificial life, or the edge of chaos, or the hypothetical new second law. The economics program was taking up 110 percent of his workday as it was. But what he did hear he found fascinating. It seemed to him that artificial life and the rest captured something essential about the spirit of the institute. “Martin Heidegger once said that the fundamental philosophical question is being,” notes Arthur. “What are we doing here as conscious entities? Why isn’t the universe just a turbulent mess of particles tumbling around each other? Why are there structure, form, and pattern? Why is consciousness possible at all?”

“So from this point of view, the purpose of having a Santa Fe Institute is that it, and places like it, are where the metaphors and a vocabulary are being created in complex systems. So if somebody comes along with a beautiful study on the computer, then you can say ‘Here’s a new metaphor. Let’s call this one the edge of chaos,’ or whatever. So what the SFI will do, if it studies enough complex systems, is to show us the kinds of patterns we might observe, and the kinds of metaphor that might be appropriate for systems that are moving and in process and complicated, rather than the metaphor of clockwork. “So I would argue that a wise use of the SFI is to let it do science,” he says. “To make it into a policy shop would be a great mistake. It would cheapen the whole affair. And in the end it would be counterproductive, because what we’re missing at the moment is any precise understanding of how complex systems operate. This is the next major task in science for the next 50 to 100 years.” “I think there’s a personality that goes with this kind of thing,” Arthur says. “It’s people who like process and pattern, as opposed to people who are comfortable with stasis and order. I know that every time in my life that I’ve run across simple rules giving rise to emergent, complex messiness, I’ve just said, ‘Ah, isn’t that lovely!’ And I think that sometimes, when other people run across it, they recoil.”

untidy forces sometimes push a system slightly out of equilibrium, then they feel the whole trick is to push it back again. Lewontin called these scientists “Platonists,” after the renowned Athenian philosopher who declared that the messy, imperfect objects we see around us are merely the reflections of perfect “archetypes.” Scientists of the second type, however, see the world as a process of flow and change, with the same material constantly going around and around in endless combinations. Lewontin called these scientists “Heraclitians,” after the Ionian philosopher who passionately and poetically argued that the world is in a constant state of flux. Heraclitus, who lived nearly a century before Plato, is famous for observing that “Upon those who step into the same rivers flow other and yet other waters,” a statement that Plato himself paraphrased as “You can never step into the same river twice.”

“How do we survive the next hundred years without a ‘Class A’ catastrophe? That is, something that can’t be set straight in a generation.” In edge-of-chaos terms, avoiding such cataclysms would mean finding some way to damp out the very largest, most destructive avalanches of change. “Originally, number one on my list of Class A catastrophes was nuclear war,” says Cowan, “with a Class B catastrophe being something like World War II. But by the time of the first meeting, rapprochement between Russia and the United States was such that the nuclear war problem was down around number five on the list. And what emerged very quickly instead was the population explosion, the [Paul] Ehrlich-type catastrophe. Then came possible environmental catastrophe, such as the greenhouse warming, which I myself didn’t think of as a Class A catastrophe, but which others have seized upon.”

global sustainability is possible only if human society undergoes at least six fundamental transitions within a very few decades: 1. A demographic transition to a roughly stable world population. 2. A technological transition to a minimal environmental impact per person. 3. An economic transition to a world in which serious attempts are made to charge the real costs of goods and services—including environmental costs—so that there are incentives for the world economy to live off nature’s “income” rather than depleting its “capital.” 4. A social transition to a broader sharing of that income, along with increased opportunities for nondestructive employment for the poor families of the world. 5. An institutional transition to a set of supranational alliances that facilitate a global attack on global problems and allow various aspects of policy to be integrated with one another. 6. An informational transition to a world in which scientific research, education, and global monitoring allow large numbers of people to understand the nature of the challenges they face.

The trick, of course, is to get from here to there without one of Cowan’s Class A global catastrophes. And if we’re to have any hope of doing that, said Gell-Mann, the study of complex adaptive systems is clearly critical. Understanding these six fundamental transitions means understanding economic, social, and political forces that are deeply intertwined and mutually dependent upon one another. You can’t just look at each piece of the problem individually, as has been done in the past, and hope to describe the behavior of the system as a whole. The only way to do it is to look at the world as a strongly interconnected system—even if the models are crude.

the trick in getting from here to there is to make sure that “there” is a world worth living in. A sustainable human society could easily be some Orwellian dystopia

On the one hand, humanity is gravely threatened by superstition and myth, the stubborn refusal to recognize the urgent planetary problems, and generalized tribalism in all its forms. To achieve those six fundamental transitions will require some kind of broad-scale agreement on principles and a more rational way of thinking about the future of the planet, not to mention a more rational way of governing ourselves on a global scale.

But on the other hand, he said, “How do you reconcile the identification and labeling of error with the tolerance—not only tolerance, but celebration and preservation—of cultural diversity?” This isn’t a matter of political correctness, but of hard-edged practicality. Cultures can’t be eradicated by fiat; witness the violent reaction to the Shah’s efforts to westernize Iran. The world will have to be governed pluralistically or not at all. Moreover, cultural diversity will be just as important in a sustainable world as genetic diversity is in biology. We need cross-cultural ferment, said Gell-Mann. “Of particular importance may be discoveries about how [our own culture can] restrain the appetite for material goods and substitute more spiritual appetites.” In the long run, he said, solving this dilemma may require much more than sensitivity. It may require profound new developments in the behavioral sciences. After all, the cure of individual neuroses is not easy; the same is true of social neuroses.