How to Create a Mind: The Secret of Human Thought Revealed

by Ray Kurzweil

Memories that are successfully recognized may also result in the creation of a new pattern to achieve greater redundancy. If patterns are not perfectly recognized, they are likely to be stored as reflecting a different perspective of the item that was recognized.

What, then, is the overall method for determining what patterns get stored? In mathematical terms, the problem can be stated as follows: Using the available limits of pattern storage, how do we optimally represent the input patterns that have thus far been presented? While it makes sense to allow for a certain amount of redundancy, it would not be practical to fill up the entire available storage area (that is, the entire neocortex) with repeated patterns, as that would not allow for a sufficient diversity of patterns.

There is a mathematical solution to this optimization problem called linear programming, which solves for the best possible allocation of limited resources (in this case, a limited number of pattern recognizers) that would represent all of the cases on which the system has trained. Linear programming is designed for systems with one-dimensional inputs, which is another reason why it is optimal to represent the input to each pattern recognition module as a linear string of inputs. We can use this mathematical approach in a software system, and though an actual brain is further constrained by the physical connections it has available that it can adapt between pattern recognizers, the method is nonetheless similar.

The output of a pattern can feed back to a pattern at a lower level or even to the pattern itself, giving the human brain its powerful recursive ability. An element of a pattern can be a decision point based on another pattern. This is especially useful for lists that compose actions—for example, getting another tube of toothpaste if the current one is empty. These conditionals exist at every level. As anyone who has attempted to program a procedure on a computer knows, conditionals are vital to describing a course of action.

b) When this pattern recognizer recognizes its pattern (based on all or most of the input dendrite signals being activated), the axon (output) of this pattern recognizer will activate. In turn, this axon can connect to an entire network of dendrites connecting to many higher-level pattern recognizers that this pattern is input to. This signal will transmit magnitude information so that the pattern recognizers at the next higher conceptual level can consider it.

e) Inhibitory signals from above would also make it less likely that this pattern recognizer will recognize its pattern. This can result from a higher-level context that is inconsistent with the pattern associated with this recognizer.

hierarchical hidden Markov models.

Patterns triggered in the neocortex trigger other patterns. Partially complete patterns send signals down the conceptual hierarchy; completed patterns send signals up the conceptual hierarchy. These neocortical patterns are the language of thought.

In doing directed thinking, we are stepping through lists in our neocortex, each of which expands into extensive hierarchies of sublists, each with its own considerations. Keep in mind that elements in a list in a neocortical pattern can include conditionals, so our subsequent thoughts and actions will depend on assessments made as we go through the process.

Our actual mental experience is complex and messy, made up of these lightning storms of triggered patterns, which change about a hundred times a second.

In 2003 and 2004, PalmPilot inventor Jeff Hawkins and Dileep George developed a hierarchical cortical model called hierarchical temporal memory. With science writer Sandra Blakeslee, Hawkins described this model eloquently in their book On Intelligence. Hawkins provides a strong case for the uniformity of the cortical algorithm and its hierarchical and list-based organization. There are some important differences between the model presented in On Intelligence and what I present in this book. As the name implies, Hawkins is emphasizing the temporal (time-based) nature of the constituent lists. In other words, the direction of the lists is always forward in time. His explanation for how the features in a two-dimensional pattern such as the printed letter “A” have a direction in time is predicated on eye movement. He explains that we visualize images using saccades, which are very rapid movements of the eye of which we are unaware. The information reaching the neocortex is therefore not a two-dimensional set of features but rather a time-ordered list. While it is true that our eyes do make very rapid movements, the sequence in which they view the features of a pattern such as the letter “A” does not always occur in a consistent temporal order. (For example, eye saccades will not always register the top vertex in “A” before its bottom concavity.) Moreover, we can recognize a visual pattern that is presented for only a few tens of milliseconds, which is too short a period of ti...

CHAPTER 4 THE BIOLOGICAL NEOCORTEX

In this way, biological evolution found that the hierarchical learning of the neocortex was so valuable that this region of the brain continued to grow in size until it virtually took over the brain of Homo sapiens.

The central assumption in Hebb’s theory is that the basic unit of learning in the neocortex is the neuron. The pattern recognition theory of mind that I articulate in this book is based on a different fundamental unit: not the neuron itself, but rather an assembly of neurons, which I estimate to number around a hundred. The wiring and synaptic strengths within each unit are relatively stable and determined genetically—that is, the organization within each pattern recognition module is determined by genetic design. Learning takes place in the creation of connections between these units, not within them, and probably in the synaptic strengths of those interunit connections.

Recent support for the basic module of learning’s being a module of dozens of neurons comes from Swiss neuroscientist Henry Markram (born in 1962), whose ambitious Blue Brain Project to simulate the entire human brain I describe in chapter 7. In a 2011 paper he describes how while scanning and analyzing actual mammalian neocortex neurons, he was “search[ing] for evidence of Hebbian assemblies at the most elementary level of the cortex.” What he found instead, he writes, were “elusive assemblies [whose] connectivity and synaptic weights are highly predictable and constrained.” He concludes that “these findings imply that experience cannot easily mold the synaptic connections of these assemblies” and speculates that “they serve as innate, Lego-like building blocks of knowledge for perception and that the acquisition of memories involves the combination of these building blocks into complex constructs.”

Since these assemblies will all be similar in topology and synaptic weights, not molded by any specific experience, we consider these to be innate assemblies…. Experience plays only a minor role in determining synaptic connections and weights within these assemblies….

Connections between assemblies may combine them into super-assemblies within a neocortical layer, then in higher-order assemblies in a cortical column, even higher-order assemblies in a brain region, and finally in the highest possible order assembly represented by the whole brain…. Acquiring memories is very similar to building with Lego. Each assembly is equivalent to a Lego block holding some piece of elementary innate knowledge about how to process, perceive and respond to the world…. When different blocks come together, they therefore form a unique combination of these innate percepts that represents an individual’s specific knowledge and experience.3

Another recent study, this one from Massachusetts General Hospital, funded by the National Institutes of Health and the National Science Foundation and published in a March 2012 issue of the journal Science, also shows a regular structure of connections across the neocortex.5 The article describes the wiring of the neocortex as following a grid pattern, like orderly city streets: “Basically, the overall structure of the brain ends up resembling Manhattan, where you have a 2-D plan of streets and a third axis, an elevator going in the third dimension,” wrote Van J. Wedeen, a Harvard neuroscientist and physicist and the head of the study.

Whereas the Markram study shows a module of neurons that repeats itself across the neocortex, the Wedeen study demonstrates a remarkably orderly pattern of connections between modules. The brain starts out with a very large number of “connections-in-waiting” to which the pattern recognition modules can hook up. Thus if a given module wishes to connect to another, it does not need to grow an axon from one and a dendrite from the other to span the entire physical distance between them. It can simply harness one of these axonal connections-in-waiting and just hook up to the ends of the fiber. As Wedeen and his colleagues write, “The pathways of the brain follow a base-plan established by…early embryogenesis. Thus, the pathways of the mature brain present an image of these three primordial gradients, physically deformed by development.” In other words, as we learn and have experiences, the pattern recognition modules of the neocortex are connecting to these preestablished connections that were created when we were embryos.

Our understanding of the lower hierarchical levels of the visual neocortex is consistent with the PRTM I described in the previous chapter, and observation of the hierarchical nature of neocortical processing has recently extended far beyond these levels. University of Texas neurobiology professor Daniel J. Felleman and his colleagues traced the “hierarchical organization of the cerebral cortex…[in] 25 neocortical areas,” which included both visual areas and higher-level areas that combine patterns from multiple senses. What they found as they went up the neocortical hierarchy was that the processing of patterns became more abstract, comprised larger spatial areas, and involved longer time periods. With every connection they found communication both up and down the hierarchy.8

This is a basic organizing principle of the visual system…. Real-world events occur not only over extended regions of space, but also over extended periods of time. We therefore hypothesized that a hierarchy analogous to that found for spatial receptive field sizes should also exist for the temporal response characteristics of different brain regions.” This is exactly what they found, which enabled them to conclude that “similar to the known cortical hierarchy of spatial receptive fields, there is a hierarchy of progressively longer temporal receptive windows in the human brain.”9

The most powerful argument for the universality of processing in the neocortex is the pervasive evidence of plasticity (not just learning but interchangeability): In other words, one region is able to do the work of other regions, implying a common algorithm across the entire neocortex.

However, congenitally blind individuals also activate the visual cortex in some verbal tasks. We provide evidence that this visual cortex activity in fact reflects language processing. We find that in congenitally blind individuals, the left visual cortex behaves similarly to classic language regions…. We conclude that brain regions that are thought to have evolved for vision can take on language processing as a result of early experience.”10

Consider the implications of this study: It means that neocortical regions that are physically relatively far apart, and that have also been considered conceptually very different (primitive visual cues versus abstract language concepts), use essentially the same algorithm. The regions that process these disparate types of patterns can substitute for one another.

Many other studies document these attributes of the neocortex, but let’s summarize what we can observe from the neuroscience literature and from our own thought experiments. The basic unit of the neocortex is a module of neurons, which I estimate at around a hundred. These are woven together into each neocortical column so that each module is not visibly distinct. The pattern of connections and synaptic strengths within each module is relatively stable. It is the connections and synaptic strengths between modules that represent learning.

There are on the order of a quadrillion (1015) connections in the neocortex, yet only about 25 million bytes of design information in the genome (after lossless compression),16 so the connections themselves cannot possibly be predetermined genetically. It is possible that some of this learning is the product of the neocortex’s interrogating the old brain, but that still would necessarily represent only a relatively small amount of information. The connections between modules are created on the whole from experience (nurture rather than nature).

Signals go up and down the conceptual hierarchy. A signal going up means, “I’ve detected a pattern.” A signal going down means, “I’m expecting your pattern to occur,” and is essentially a prediction. Both upward and downward signals can be either excitatory or inhibitory.

CHAPTER 5 THE OLD BRAIN

Each of us lives within the universe—the prison—of his own brain. Projecting from it are millions of fragile sensory nerve fibers, in groups uniquely adapted to sample the energetic states of the world around us: heat, light, force, and chemical composition. That is all we ever know of it directly; all else is logical inference. —Vernon Mountcastle1

In a study published in Nature, Frank S. Werblin, professor of molecular and cell biology at the University of California at Berkeley, and doctoral student Boton Roska, MD, showed that the optic nerve carries ten to twelve output channels, each of which carries only a small amount of information about a given scene.2 One group of what are called ganglion cells sends information only about edges (changes in contrast). Another group detects only large areas of uniform color, whereas a third group is sensitive only to the backgrounds behind figures of interest.

This data reduction is what in the AI field we call “sparse coding.” We have found in creating artificial systems that throwing most of the input information away and retaining only the most salient details provides superior results. Otherwise the limited ability to process information in a neocortex (biological or otherwise) gets overwhelmed.

Everyone knows what attention is. It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness, are of its essence. It implies withdrawal from some things in order to deal effectively with others. —William James

The most significant role of the thalamus, however, is its continual communication with the neocortex. The pattern recognizers in the neocortex send tentative results to the thalamus and receive responses principally using both excitatory and inhibitory reciprocal signals from layer VI of each recognizer. Keep in mind that these are not wireless messages, so that there needs to be an extraordinary amount of actual wiring (in the form of axons) running between all regions of the neocortex and the thalamus. Consider the vast amount of real estate (in terms of the physical mass of connections required) for the hundreds of millions of pattern recognizers in the neocortex to be constantly checking in with the thalamus.5

So what are the hundreds of millions of neocortical pattern recognizers talking to the thalamus about? It is apparently an important conversation, because profound damage to the main region of the thalamus bilaterally can lead to prolonged unconsciousness. A person with a damaged thalamus may still have activity in his neocortex, in that the self-triggering thinking by association can still work. But directed thinking—the kind that will get us out of bed, into our car, and sitting at our desk at work—does not function without a thalamus. In a famous case, twenty-one-year-old Karen Ann Quinlan suffered a heart attack and respiratory failure and remained in an unresponsive, apparently vegetative state for ten years. When she died, her autopsy revealed that her neocortex was normal but her thalamus had been destroyed.

Dr. Wolfram’s primary thesis is that the world is one big class IV cellular automaton. The reason that his book is titled A New Kind of Science is because this theory contrasts with most other scientific laws. If there is a satellite orbiting Earth, we can predict where it will be five years from now without having to run through each moment of a simulated process by using the relevant laws of gravity and solve where it will be at points in time far in the future. But the future state of class IV cellular automata cannot be predicted without simulating every step along the way. If the universe is a giant cellular automaton, as Dr. Wolfram postulates, there would be no computer big enough—since every computer would be a subset of the universe—that could run such a simulation. Therefore the future state of the universe is completely unknowable even though it is deterministic.

Thus even though our decisions are determined (because our bodies and brains are part of a deterministic universe), they are nonetheless inherently unpredictable because we live in (and are part of) a class IV automaton. We cannot predict the future of a class IV automaton except to let the future unfold. For Dr. Wolfram, this is sufficient to allow for free will.