How to Create a Mind: The Secret of Human Thought Revealed

by Ray Kurzweil

The mammalian brain has a distinct aptitude not found in any other class of animal. We are capable of hierarchical thinking, of understanding a structure composed of diverse elements arranged in a pattern, representing that arrangement with a symbol, and then using that symbol as an element in a yet more elaborate configuration. This capability takes place in a brain structure called the neocortex, which in humans has achieved a threshold of sophistication and capacity such that we are able to call these patterns ideas. Through an unending recursive process we are capable of building ideas that are ever more complex. We call this vast array of recursively linked ideas knowledge. Only Homo sapiens have a knowledge base that itself evolves, grows exponentially, and is passed down from one generation to another.

These tools represented a new form of evolution, as neurology gave rise to technology. It is only because of our tools that our knowledge base has been able to grow without limit.

Libraries of written language vastly extended the ability of our unaided brains to retain and extend our knowledge base of recursively structured ideas.

The Singularity Is Near, written in the early 2000s and published in 2005) is that an evolutionary process inherently accelerates (as a result of its increasing levels of abstraction) and that its products grow exponentially in complexity and capability. I call this phenomenon the law of accelerating returns (LOAR), and it pertains to both biological and technological evolution.

The Web is itself a powerful and apt example of the ability of a hierarchical system to encompass a vast array of knowledge while preserving its inherent structure. The world itself is inherently hierarchical—trees contain branches; branches contain leaves; leaves contain veins. Buildings contain floors; floors contain rooms; rooms contain doorways, windows, walls, and floors.

In this book I present a thesis I call the pattern recognition theory of mind (PRTM), which, I argue, describes the basic algorithm of the neocortex (the region of the brain responsible for perception, memory, and critical thinking). In the chapters ahead I describe how recent neuroscience research, as well as our own thought experiments, leads to the inescapable conclusion that this method is used consistently across the neocortex. The implication of the PRTM combined with the LOAR is that we will be able to engineer these principles to vastly extend the powers of our own intelligence.

The cutting edge of the project to understand, model, and simulate the human brain is to reverse-engineer the cerebral neocortex, where we do our recursive hierarchical thinking. The cerebral cortex, which accounts for 80 percent of the human brain, is composed of a highly repetitive structure, allowing humans to create arbitrarily complex structures of ideas.

The operating principle of the neocortex is arguably the most important idea in the world, as it is capable of representing all knowledge and skills as well as creating new knowledge.

It would be fair to say that the concept of a forest is simpler than the concept of a tree.

Thus it is with the brain, which has a similar enormous redundancy, especially in the neocortex. As I will describe in this book, it would be fair to say that there is more complexity in a single neuron than in the overall structure of the neocortex.

My goal in this book is definitely not to add another quotation to the millions that already exist attesting to how complex the brain is, but rather to im...

“Identity lies not in our genes, but in the connections between our brain cells.”6

Nothing is at last sacred but the integrity of your own mind. —Ralph Waldo Emerson

CHAPTER 2 THOUGHT EXPERIMENTS ON THINKING

It turns out that the distinction between being determined and being predictable is an important one, to which I will return.

Do you remember your social security number? If you do, can you recite it backward without first writing it down? How about the nursery rhyme “Mary Had a Little Lamb”? Computers can do this trivially. Yet we fail at it unless we specifically learn the backward sequence as a new series. This tells us something important about how human memory is organized.

Of course, we are able to perform this task easily if we write down the sequence and then read it backward. In doing so we are using a technology—written language—to compensate for one of the limitations of our unaided thinking, albeit a very early tool. (It was our second invention, with spoken language as the first.) This is why we invent tools—to compensate for our shortcomings.

This suggests that our memories are sequential and in order. They can be accessed in the order that they are remembered. We are unable to dir...

We also have some difficulty starting a memory in the middle of a sequence. If I learn to play a piece of music on the piano, I generally can’t just begin it at an arbitrary point in its middle. There are a few points at which I can jump in, because my sequential memory of the piece is organized in segments. If I try to start in the middle of...

If you take walks regularly, think back to the first walk you took last month (or to the first trip to the office last month, if you commute). You probably cannot recall the specific walk or commute at all, and if you do, you doubtless recall even fewer details about it than about your walk today. I will later discuss the issue of consciousness and make the point that we tend to equate consciousness with our memory of events. The primary reason we believe that we are not conscious when under anesthesia is that we don’t remember anything from that period (albeit there are intriguing—and disturbing—exceptions to this). So with regard to the walk I took this morning, was I not conscious during most of it? It’s a reasonable question, given that I remember almost nothing about what I saw or even what I was thinking about.

Next, think about people whom you have encountered only once or twice. Can you visualize them clearly? If you are a visual artist, then you may have learned this observational skill, but typically we are unable to visualize people we’ve only casually come across to draw or describe them sufficiently but would have little difficulty in recognizing a picture of them.

This suggests that there are no images, videos, or sound recordings stored in the brain. Our memories are stored as sequences of patterns. Memories that are not accessed dim over time. When police sketch artists interview a crime victim, they do not ask, “What did the perpetrator’s eyebrows look like?” Rather, they will show a series of images of eyebrows and ask the victim to select one. The correct set of eyebrows will trigger the recognition of the same pattern that is stored in the victim’s memory.

You are undoubtedly able to recognize these familiar personalities, even though they are partially covered or distorted. This represents a key strength of human perception: We can recognize a pattern even if only part of it is perceived (seen, heard, felt) and even if it contains alterations. Our recognition ability is apparently able to detect invariant features of a pattern—characteristics that survive real-world variations. The apparent distortions in a caricature or in certain forms of art such as impressionism emphasize the patterns of an image (person, object) that we recognize while changing other details. The world of art is actually ahead of the world of science in appreciating the power of the human perceptual system. We use the same approach when we recognize a melody from only a few notes.

Thus our conscious experience of our perceptions is actually changed by our interpretations.

Consider that we see what we expect to ___ I’m confident that you were able to complete the above sentence. Had I written out the last word, you would have needed only to glance at it momentarily to confirm that it was what you had expected.

This implies that we are constantly predicting the future and hypothesizing what we will experience. This expectation influences what we actually perceive. Predicting the future is ...

Consider an experience that we all have on a regular basis: A memory from years ago inexplicably pops into your head. Often this will be a memory of a person or an event that you haven’t thought about for a long time. It is evident that something has triggered the memory. The train of thought that did so may be apparent and something you are able to articulate. At other times you may be aware of the sequence of thoughts that led to the memory but would have a hard time expressing it. Often the trigger is quickly lost, so the me...

A related phenomenon that everyone experiences frequently is trying to think of a name or a word. The procedure we use in this circumstance is to try to remind ourselves of triggers that may unlock the memory. (For example: Who played Queen Padmé in Revenge of the Sith? Let’s see, it’s that same actress who was the star in a recent dark movie about dancing, that was Black Swan, oh yes, Natalie Portman.) Sometimes we adopt idiosyncratic mnemonics to help us remember. (For example: She’s always slim, not portly, oh yes, Portman, Natalie Portman.) Some of our memories are sufficiently robust that we can go directly from a question (such as who played Queen Padmé) to the answer; often we need to go through a series of triggers until we find one that works. It’s very much like having the right Web link. Memories can indeed become lost like a Web page to which no other page links to (at least no page that we can find).

While executing routine procedures—such as putting on a shirt—watch yourself performing them, and consider the extent to which you follow the same sequence of steps each time. From my own observation (and as I mentioned, I am constantly trying to observe myself), it is likely that you follow very much the same steps each time you perform a particular routine task, though there may be additional modules added. For example, most of my shirts do not require cuff links, but when one does, that involves a further series of tasks.

The lists of steps in my mind are organized in hierarchies. I follow a routine procedure before going to sleep. The first step is to brush my teeth. But this action is in turn broken into a smaller series of steps, the first of which is to put toothpaste on the toothbrush. That step in turn is made up of...

This nesting actually continues down to a very fine grain of movements, so that there are literally thousands of little actions constituting my nighttime routine. Although I may have difficulty remembering details of a walk I took just a few hours ago, I have no difficulty recalling all of these many steps in preparing for bed—so much so that I am able to think about other things while I go through these procedures. It is important to point out that this list is not stored as one long list of thousands of steps—rather, each of our routine procedures is remembered as an elaborate hierarchy of nested activities.

The same type of hierarchy is involved in our ability to recognize objects and situations. We recognize the faces of people we know well and also recognize that these faces contain eyes, a nose, a mouth, and so on—a hierarchy of patterns that we use in both our perceptions and our actions. The use of hierarchies allows us to reuse patterns. For example, we do not need to relearn the concept of a nose and a mouth each time we are introduced to a new face.

CHAPTER 3 A MODEL OF THE NEOCORTEX: THE PATTERN RECOGNITION THEORY OF MIND

First, let me explain why this section specifically discusses the neocortex (from the Latin meaning “new rind”). We do know the neocortex is responsible for our ability to deal with patterns of information and to do so in a hierarchical fashion. Animals without a neocortex (basically nonmammals) are largely incapable of understanding hierarchies.1 Understanding and leveraging the innately hierarchical nature of reality is a uniquely mammalian trait and results from mammals’ unique possession of this evolutionarily recent brain structure. The neocortex is responsible for sensory perception, recognition of everything from visual objects to abstract concepts, controlling movement, reasoning from spatial orientation to rational thought, and language—basically, what we regard as “thinking.”

Due to its elaborate folding, the neocortex constitutes the bulk of the human brain, accounting for 80 percent of its weight. Homo sapiens developed a large forehead to allow for an even larger neocortex; in particular we have a frontal lobe where we deal with the more abstract patterns associated with high-level concepts.

This thin structure is basically made up of six layers, numbered I (the outermost layer) to VI. The axons emerging from the neurons in layers II and III project to other parts of the neocortex. The axons (output connections) from layers V and VI are connected primarily outside of the neocortex to the thalamus, brain stem, and spinal cord. The neurons in layer IV receive synaptic (input) connections from neurons that are outside the neocortex, especially in the thalamus. The number of layers varies slightly from region to region. Layer IV is very thin in the motor cortex, because in that area it largely does not receive input from the thalamus, brain stem, or spinal cord. Conversely, in the occipital lobe (the part of the neocortex usually responsible for visual processing), there are three additional sublayers that can be seen in layer IV, due to the considerable input flowing into this region, including from the thalamus. A critically important observation about the neocortex is the extraordinary uniformity of its fundamental structure. This was first noticed by American neuroscientist Vernon Mountcastle (born in 1918). In 1957 Mountcastle discovered the columnar organization of the neocortex. In 1978 he made an observation that is as significant to neuroscience as the Michelson-Morley ether-disproving experiment of 1887 were to physics. That year he described the remarkably unvarying organization of the neocortex, hypothesizing that it was composed of a single mechanism ...

Mountcastle hypothesized the existence of mini-columns within columns, but this theory became controversial because there were no visible demarcations of such smaller structures. However, extensive experimentation has revealed that there are in fact repeating units within the neuron fabric of each column. It is my contention that the basic unit is a pattern recognizer and that this constitutes the fundamental component of the neocortex. In contrast to Mountcastle’s notion of a mini-column, there is no specific physical boundary to these recognizers, as they are placed closely one to the next in an interwoven fashion, so the cortical column is simply an aggregate of a large number of them. These recognizers are capable of wiring themselves to one another throughout the course of a ...

The same principle applies to the levels of modeling and understanding in the brain. It is certainly a useful and necessary part of reverse-engineering the brain to model its interactions at the molecular level, but the goal of the effort here is essentially to refine our model to account for how the brain processes information to produce cognitive meaning.

There are about a half million cortical columns in a human neocortex, each occupying a space about two millimeters high and a half millimeter wide and containing about 60,000 neurons (resulting in a total of about 30 billion neurons in the neocortex). A rough estimate is that each pattern recognizer within a cortical column contains about 100 neurons, so there are on the order of 300 million pattern recognizers in total in the neocortex.

Human beings have only a weak ability to process logic, but a very deep core capability of recognizing patterns. To do logical thinking, we need to use the neocortex, which is basically a large pattern recognizer. It is not an ideal mechanism for performing logical transformations, but it is the only facility we have for the job. Compare, for example, how a human plays chess to how a typical computer chess program works. Deep Blue, the computer that defeated Garry Kasparov, the human world chess champion, in 1997 was capable of analyzing the logical implications of 200 million board positions (representing different move-countermove sequences) every second. (That can now be done, by the way, on a few personal computers.) Kasparov was asked how many positions he could analyze each second, and he said it was less than one. How is it, then, that he was able to hold up to Deep Blue at all? The answer is the very strong ability humans have to recognize patterns. However, we need to train this facility, which is why not everyone can play master chess.

Kasparov had learned about 100,000 board positions. That’s a real number—we have established that a human master in a particular field has mastered about 100,000 chunks of knowledge. Shakespeare composed his plays with 100,000 word senses (employing about 29,000 distinct words, but using most of them in multiple ways). Medical expert systems that have been built to represent the knowledge of a human medical physician have shown that a typical human medical specialist has mastered about 100,000 concepts in his or her domain. Recognizing a chunk of knowledge from this store is not straightforward, as a particular item will present itself a little bit differently each time it is experienced.

Armed with his knowledge, Kasparov looks at the chessboard and compares the patterns that he sees to all 100,000 board situations that he has mastered, and he does all 100,000 comparisons simultaneously. There is consensus on this point: All of our neurons are processing—considering the patterns—at the same time. That does not mean that they are all firing simultaneously (we would probably fall to ...

How many patterns can the neocortex store? We need to factor in the phenomenon of redundancy. The face of a loved one, for example, is not stored once but on the order of thousands of times. Some of these repetitions are largely the same image of the face, whereas most show different perspectives of it, different lighting, different expressions, and so on. None of these repeated patterns are stored as images per se (that is, as two-dimensional arrays of pixels). Rather, they are stored as lists of features where the constituent elements of a ...

If we take the core knowledge of an expert as consisting of about 100,000 “chunks” of knowledge (that is, patterns) with a redundancy estimate of about 100 to 1, that gives us a requirement of 10 million patterns. This core expert knowledge is built on more general and extensive professional knowledge, so we can increase the order of magnitude of patterns to about 30 to 50 million. Our everyday “commonsense” knowledge as a human being is even greater; “street smarts” actually require substantially more of our neocortex than “book smarts.” Including this brings our estimate to well over 100 million patterns, taking into account the redundancy factor of about 100. Note that the redundancy...

As I will discuss below, our procedures and actions also comprise patterns and are likewise stored in regions of the cortex, so my estimate of the total capacity of the human neocortex is on the order of low hundreds of millions of patterns. This rough tally correlates well with the number of pattern recognizers that I estimated above at about 300 million, so it is a reasonable conclusion that the function of each neocortical pattern recognizer is to process one iteration (that is, one copy among the multiple redundant copies of most patterns in the neocortex) of a pattern. Our estimates of the number of patterns that a human brain is capable of dealing with (including necessary redundancy) and the number of physical pattern recognizers happen to be the same order of magnitude. It should be noted here that when I refer to “processing” a pattern, I am re...

Three hundred million pattern processors may sound like a large number, and indeed it was sufficient to enable Homo sapiens to develop verbal and written language, all of our tools, and other diverse creations. These inventions have built upon themselves, giving rise to the exponential growth of the inf...

The pattern recognition theory of mind that I present here is based on the recognition of patterns by pattern recognition modules in the neocortex. These patterns (and the modules) are organized in hierarchies. I discuss below the intellectual roots of this idea, including my own work with hierarchical pattern recognition in the 1980s and 1990s and Jeff Hawkins (born in 1957) and Dileep George’s (born in 1977) model of the neocortex in the early 2000s.

Each pattern (which is recognized by one of the estimated 300 million pattern recognizers in the neocortex) is composed of three parts. Part one is the input, which consists of the lower-level patterns that compose the main pattern. The descriptions for each of these lower-level patterns do not need to be repeated for each higher-level pattern that references them. For example, many of the patterns for words will include the letter “A.”

Think of it as being like a Web pointer. There is one Web page (that is, one pattern) for the letter “A,” and all of the Web pages (patterns) for words that include “A” will have a link to the “A” page (to the “A” pattern). Instead of Web links, the neocortex uses actual neural connections. There is an axon from the “A” pattern recognizer that connects to multiple dendrites, one for each word that uses “A.” Keep in mind also the redundancy factor: There is more than one pattern recognizer for the letter “A.” Any of these multiple “A” pattern recognizers can send a signal up to the pattern recognizers that incorporate “A.”

The second part of each pattern is the pattern’s name. In the world of language, this higher-level pattern is simply the word “apple.” Although we directly use our neocortex to understand and process every level of language, most of the patterns it contains are not language patterns per se. In the neocortex the “name” of a pattern is simply the axon that emerges from each pattern processor; when that axon fires, its corresponding pattern has been recognized. The firing of the axon is that pattern recognizer shouting the name of the pattern: “Hey guys, I just saw the written word ‘apple.’”