How to Create a Mind: The Secret of Human Thought Revealed

by Ray Kurzweil

Kasparov had learned about 100,000 board positions.

That’s a real number—we have established that a human master in a particular field has mastered about 100,000 chunks of knowledge.

Shakespeare composed his plays with 100,0...

Kasparov looks at the chessboard and compares the patterns that he sees to all 100,000 board situations that he has mastered, and he does all 100,000 comparisons simultaneously.

“street smarts” actually require substantially more of our neocortex than “book smarts.”

Three hundred million pattern processors may sound like a large number, and indeed it was sufficient to enable Homo sapiens to develop verbal and written language, all of our tools, and other diverse creations. These inventions have built upon themselves, giving rise to the exponential growth of the information content of technologies as described in my law of accelerating returns. No other species has achieved this.

every level constitutes a pattern. In this case, the shapes are patterns, the letters are patterns, and the words are also patterns. Each of these patterns has a set of inputs, a process of pattern recognition (based on the inputs that take place in the module), and an output (which feeds to the next higher level of pattern recognizer).

In a different part of the cortex is a comparable hierarchy of pattern recognizers processing actual images of objects (as opposed to printed letters). If you are looking at an actual apple, low-level recognizers will detect curved edges and surface color patterns leading up to a pattern recognizer firing its axon and saying in effect, “Hey guys, I just saw an actual apple.” Yet other pattern recognizers will detect combinations of frequencies of sound leading up to a pattern recognizer in the auditory cortex that might fire its axon indicating, “I just heard the spoken word ‘apple.’” Keep in mind the redundancy factor—we don’t just have a single pattern recognizer for “apple” in each of its forms (written, spoken, visual).

The redundancy not only increases the likelihood that you will successfully recognize each instance of an apple but also deals with the variations in real-world apples.

Also keep in mind that the hierarchy shown above is a hierarchy of concepts. These recognizers are not physically placed above each other; because of the thin construction of the neocortex, it is physically only one pattern recognizer high. The conceptual hierarchy is created by the connections between the individual pattern recognizers.

Not every input pattern has to be present for a recognizer to fire. The recognizer may still fire if an input with a low weight is missing, but it is less likely to fire if a high-importance input is missing. When it fires, a pattern recognizer is basically saying, “The pattern I am responsible for is probably present.”

In a biological brain, the source of these parameters comes from the brain’s own experience. We are not born with an innate knowledge of phonemes; indeed different languages have very different sets of them. This implies that multiple examples of a pattern are encoded in the learned parameters of each pattern recognizer (as it requires multiple instances of a pattern to ascertain the expected distribution of magnitudes of the inputs to the pattern). In some AI systems, these types of parameters are hand-coded by experts (for example, linguists who can tell us the expected durations of different phonemes, as I articulated above). In my own work, we found that having an AI system discover these parameters on its own from training data (similar to the way the brain does it) was a superior approach. Sometimes we used a hybrid approach; that is, we primed the system with the intuition of human experts (for the initial settings of the parameters) and then had the AI system automatically refine these estimates using a learning process from real examples of speech.

If we go up several dozen more levels, we get to higher-level concepts like irony and envy. Even though every pattern recognizer is working simultaneously, it does take time for recognitions to move upward in this conceptual hierarchy.

A very important point to note here is that information flows down the conceptual hierarchy as well as up. If anything, this downward flow is even more significant. If, for example, we are reading from left to right and have already seen and recognized the letters “A,” “P,” “P,” and “L,” the “APPLE” recognizer will predict that it is likely to see an “E” in the next position. It will send a signal down to the “E” recognizer saying, in effect, “Please be aware that there is a high likelihood that you will see your ‘E’ pattern very soon, so be on the lookout for it.”

The neocortex is, therefore, predicting what it expects to encounter. Envisaging the future is one of the primary reasons we have a neocortex. At the highest conceptual level, we are continually making predictions—who is going to walk through the door next, what someone is likely to say next, what we expect to see when we turn the corner, the likely results of our own actions, and so on. These predictions are constantly occurring at every level of the neocortex hierarchy.

In addition to positive signals, there are also negative or inhibitory signals which indicate that a certain pattern is less likely to exist. These can come from lower conceptual levels (for example, the recognition of a mustache will inhibit the likelihood that a person I see in the checkout line is my wife), or from a higher level (for example, I know that my wife is on a trip, so the person in the checkout line can’t be she).

Our memories are in fact patterns organized as lists

memories exist in the neocortex in order to be recognized.

Even if we do have some level of awareness of the memories (that is, the patterns) that triggered the old memory, memories (patterns) do not have language or image labels. This is the reason why old memories may seem to suddenly jump into our awareness. Having been buried and not activated for perhaps years, they need a trigger in the same way that a Web page needs a Web link to be activated. And just as a Web page can become “orphaned” because no other page links to it, the same thing can happen to our memories.

In the undirected mode, we let the links play themselves out without attempting to move them in any particular direction. Some forms of meditation (such as Transcendental Meditation, which I practice) are based on letting the mind do exactly this. Dreams have this quality as well.

In directed thinking we attempt to step through a more orderly process of recalling a memory (a story, for example) or solving a problem. This also involves stepping through lists in our neocortex,

Because these patterns are not labeled with words or sounds or pictures or videos, when you try to recall a significant event, you will essentially be reconstructing the images in your mind, because the actual images do not exist.

As we experience our own thoughts and memories, we “know” what they mean, but they do not exist as readily explainable thoughts and recollections. If we want to share them with others, we need to translate them into language. This task is also accomplished by the neocortex, using pattern recognizers trained with patterns that we have learned for the purpose of using language. Language is itself highly hierarchical and evolved to take advantage of the hierarchical nature of the neocortex, which in turn reflects the hierarchical nature of reality.

I could pick out a picture of the woman with the baby carriage whom I saw earlier today from among a group of pictures of other women, despite the fact that I am unable to actually visualize her and cannot describe much specific about her.

Even though I saw this woman only once on my walk, there are probably already multiple copies of her pattern in my neocortex. However, if I don’t think about her for a given period of time, then these pattern recognizers will become reassigned to other patterns.

The ability to recognize patterns even when aspects of them are transformed is called feature invariance,

Learning and recognition take place simultaneously. We start learning immediately, and as soon as we’ve learned a pattern, we immediately start recognizing it. The neocortex is continually trying to make sense of the input presented to it. If a particular level is unable to fully process and recognize a pattern, it gets sent to the next higher level. If none of the levels succeeds in recognizing a pattern, it is deemed to be a new pattern.

What, then, is the overall method for determining what patterns get stored? In mathematical terms, the problem can be stated as follows: Using the available limits of pattern storage, how do we optimally represent the input patterns that have thus far been presented? While it makes sense to allow for a certain amount of redundancy, it would not be practical to fill up the entire available storage area (that is, the entire neocortex) with repeated patterns, as that would not allow for a sufficient diversity of patterns.

There is a mathematical solution to this optimization problem called linear programming, which solves for the best possible allocation of limited resources (in this case, a limited number of pattern recognizers) that would represent all of the cases on which the system has trained.

experiences that are routine are recognized but do not result in a permanent memory’s being made.

Brain: an apparatus with which we think we think.

Patterns triggered in the neocortex trigger other patterns. Partially complete patterns send signals down the conceptual hierarchy; completed patterns send signals up the conceptual hierarchy. These neocortical patterns are the language of thought. Just like language, they are hierarchical, but they are not language per se. Our thoughts are not conceived primarily in the elements of language, although since language also exists as hierarchies of patterns in our neocortex, we can have language-based thoughts. But for the most part, thoughts are represented in these neocortical patterns.

It is hard enough for us to understand the content of our own thoughts, but understanding another person’s requires mastering a neocortex different from our own. Of course we don’t yet have access to someone else’s neocortex; we need instead to rely on her attempts to express her thoughts into language (as well as other means such as gestures). People’s incomplete ability to accomplish these communication tasks adds another layer of complexity—it is no wonder that we misunderstand one another as much as we do.

Dreams are examples of undirected thoughts. They make a certain amount of sense because the phenomenon of one thought’s triggering another is based on the actual linkages of patterns in our neocortex. To the extent that a dream does not make sense, we attempt to fix it through our ability to confabulate.

We confabulate all the time in explaining the outcome of events.

The actual content of a dream, to the extent that we remember it, is again a sequence of patterns. These patterns represent constraints in a story; we then confabulate a story that fits these constraints. The version of the dream that we retell (even if only to ourselves silently) is this confabulation. As we recount a dream we trigger cascades of patterns that fill in the actual dream as we originally experienced it.

As we learn professional skills, we learn the ways of thinking that are recognized and rewarded in our professions, and thereby avoid patterns of thought that might betray the methods and norms of that profession. Many of these taboos are worthwhile, as they enforce social order and consolidate progress. However, they can also prevent progress by enforcing an unproductive orthodoxy.

Cultural rules are enforced in the neocortex with help from the old brain, especially the amygdala.

In dreams, however, these taboos are relaxed, and we will often dream about matters that are culturally, sexually, or professionally forbidden. It is as if our brain realizes that we are not an actual actor in the world while dreaming.

Relaxing professional taboos turns out to be useful for creative problem solving. I use a mental technique each night in which I think about a particular problem before I go to sleep. This triggers sequences of thoughts that will continue into my dreams. Once I am dreaming, I can think—dream—about solutions to the problem without the burden of the professional restraints I carry during the day. I can then access these dream thoughts in the morning while in an in-between state of dreaming and being awake, sometimes referred to as “lucid dreaming.”

For example, in speech recognition the levels included basic patterns of sound frequency at the lowest level, then phonemes, then words and phrases (which were often recognized as if they were words). Some of our speech recognition systems could understand the meaning of natural-language commands, so yet higher levels included such structures as noun and verb phrases. Each pattern recognition module could recognize a linear sequence of patterns from a lower conceptual level. Each input had parameters for importance, size, and variability of size. There were “downward” signals indicating that a lower-level pattern was expected.

Once biological evolution stumbled on a neural mechanism capable of hierarchical learning, it found it to be immensely useful for evolution’s one objective, which is survival.

It can take a great many generations—thousands of years—for a species without a neocortex to learn significant new behaviors

haphazardly

“It is well established that neurons along the visual cortical pathways have increasingly larger spatial receptive fields. This is a basic organizing principle of the visual system…. Real-world events occur not only over extended regions of space, but also over extended periods of time.

a hierarchy analogous to that found for spatial receptive field sizes should also exist for the temporal response characteristics of different brain regions.”

“similar to the known cortical hierarchy of spatial receptive fields, there is a hierarchy of progressively longer temporal rece...

pervasive evidence of plasticity (not just learning but interchangeability): In other words, one region is able to do the work of other regions, implying a common algorithm across the entire neocortex.

The classical technique for determining this has been to take advantage of brain damage from injury or stroke and to correlate lost functionality with specific damaged regions.

The underlying assumption has been that each of these regions is designed to recognize and process a particular type of pattern. Particular physical regions have become associated with particular types of patterns, because under normal circumstances that is how the information happens to flow. But when that normal flow of information is disrupted for any reason, another region of the neocortex is able to step in and take over. Plasticity has been widely noted by neurologists, who observed that patients with brain damage from an injury or a stroke can relearn the same skills in another area of the neocortex.