A Thousand Brains: A New Theory of Intelligence

by Jeff Hawkins

It is a book of many exciting ideas, rather than one huge idea like Darwin’s.

Hawkins proposes multiple models of the world, constructed microcosms, informed and adjusted by the rain of nerve impulses pouring in from the senses.

More specifically, do we want our future to be driven by the processes that got us here, namely, natural selection, competition, and the drive of selfish genes? Or, do we want our future to be driven by intelligence and its desire to understand the world?”

The cells in your head are reading these words. Think of how remarkable that is. Cells are simple. A single cell can’t read, or think, or do much of anything. Yet, if we put enough cells together to make a brain, they not only read books, they write them. They design buildings, invent technologies, and decipher the mysteries of the universe. How a brain made of simple cells creates intelligence is a profoundly interesting question, and it remains a mystery.

The goal of our research is to understand how the neocortex works in sufficient detail that we can explain the biology of the brain and build intelligent machines that work on the same principles.

Some of our basic skills are determined by our genes, such as how to eat or how to recoil from pain. But most of what we know about the world is learned.

I have divided the book into three parts. In the first part, I describe our theory of reference frames, which we call the Thousand Brains Theory. The theory is partly based on logical deduction, so I will take you through the steps we took to reach our conclusions. I will also give you a bit of historical background to help you see how the theory relates to the history of thinking about the brain.

The second part of the book is about machine intelligence. The twenty-first century will be transformed by intelligent machines in the same way that the twentieth century was transformed by computers.

In the third part of the book, I look at the human condition from the perspective of the brain and intelligence. The brain’s model of the world includes a model of our self. This leads to the strange truth that what you and I perceive, moment to moment, is a simulation of the world, not the real world. One consequence of the Thousand Brains Theory is that our beliefs about the world can be false.

We are the first species on Earth to know the size and age of the universe. We are the first species to know how the Earth evolved and how we came to be. We are the first species to develop tools that allow us to explore the universe and learn its secrets. From this point of view, humans are defined by our intelligence and our knowledge, not by our genes. The choice we face as we think about the future is, should we continue to be driven by our biological past or choose instead to embrace our newly emerged intelligence?

If we are willing to embrace intelligence and knowledge as what defines us, instead of our genes, then perhaps we can create a future that is longer lasting and has a more noble purpose.

I find it amazing that the only thing in the universe that knows the universe exists is the three-pound mass of cells floating in our heads.

Unlike species which often disappear as new ones appear, the brain evolved by adding new parts on top of the older parts.

The newest part of our brain is the neocortex, which means “new outer layer.” All mammals, and only mammals, have a neocortex. The human neocortex is particularly large, occupying about 70 percent of the volume of our brain.

The neocortex is the organ of intelligence. Almost all the capabilities we think of as intelligence—such as vision, language, music, math, science, and engineering—are created by the neocortex. When we think about something, it is mostly the neocortex doing the thinking.

The neocortex and the older parts of the brain are connected via nerve fibers; therefore, we cannot think of them as completely separate organs. They are more like roommates, with separate agendas and personalities, but who need to cooperate to get anything done. The neocortex is in a decidedly unfair position, as it doesn’t control behavior directly. Unlike other parts of the brain, none of the cells in the neocortex connect directly to muscles, so it can’t, on its own, make any muscles move.

The old brain contains dozens of separate organs, each with a specific function. They are visually distinct, and their shapes, sizes, and connections reflect what they do. For example, there are several pea-size organs in the amygdala, an older part of the brain, that are responsible for different types of aggression, such as premeditated and impulsive aggression.

The neocortex is surprisingly different. Although it occupies almost three-quarters of the brain’s volume and is responsible for a myriad of cognitive functions, it has no visually obvious divisions. The folds and creases are needed to fit the neocortex into the skull, similar to what you would see if you forced a napkin into a large wine glass. If you ignore the folds and creases, then the neocortex looks like one large sheet of cells, with no obvious divisions.

Nonetheless, the neocortex is still divided into several dozen areas, or regions, that perform different functions. Some of the regions are responsible for vision, some for hearing, and some for touch. There are regions responsible for language and planning. When the neocortex is damaged, the deficits that arise depend on what part of the neocortex is affected. Damage to the...

The regions of the neocortex connect to each other via bundles of nerve fibers that travel under the neocortex, the so-called white matter of the brain. By carefully following these nerve fibers, scientists can determ...

For example, the first region that gets input from the eyes might detect simple patterns such as lines or edges. This information is sent to the next region, which might detect more complex features such as corners or shapes. This stepwise process continues until some regions detect complete objects.

Today we know that there are dozens of different types of neurons in the neocortex, not six.

It is important to keep in mind that the layers are only a rough guide to where a particular type of neuron might be found. It matters more what a neuron connects to and how it behaves.

Neurons in some layers make long-distance horizontal connections, but most of the connections are vertical. This means that information arriving in a region of the neocortex moves mostly up and down between the layers before being sent elsewhere.

Under one square millimeter of neocortex (about 2.5 cubic millimeters), there are roughly one hundred thousand neurons, five hundred million connections between neurons (called synapses), and several kilometers of axons and dendrites.

There are dozens of different types of neurons under each square millimeter. Each type of neuron makes prototypical connections to other types of neurons. Scientists often describe regions of the neocortex as performing a simple function, such as detecting features. However, it only takes a handful of neurons to detect features. The precise and extremely complex neural circuits seen everywhere in the neocortex tell us that every region is doing something far more complex than feature detection.

The complex circuitry of the neocortex looks remarkably alike in visual regions, language regions, and touch regions. It even looks similar across species such as rats, cats, and humans.

For example, the visual regions that get input from the eyes send a signal down to the part of the old brain responsible for moving the eyes. Similarly, the auditory regions that get input from the ears project to the part of the old brain that moves the head. Moving your head changes what you hear, similar to how moving your eyes changes what you see. The evidence we have indicates that the complex circuitry seen everywhere in the neocortex performs a sensory-motor task. There are no pure motor regions and no pure sensory regions.

Some of the connections between the regions are hierarchical, suggesting that information flows from region to region in an orderly fashion like a flowchart. But there are other connections between the regions that seem to have little order, suggesting that information goes all over at once. All regions, no matter what function they perform, look similar in detail to all other regions.

We are not aware of the vast majority of these predictions unless the input to the brain does not match. As I casually reach out to grab my coffee cup, I am not aware that my brain is predicting what each finger will feel, how heavy the cup should be, the temperature of the cup, and the sound the cup will make when I place it back on my desk. But if the cup was suddenly heavier, or cold, or squeaked, I would notice the change. We can be certain that these predictions are occurring because even a small change in any of these inputs will be noticed. But when a prediction is correct, as most will be, we won’t be aware that it ever occurred.

The overall structure of the neocortex is not random. Its size, the number of regions it has, and how they are connected together is largely determined by our genes. For example, genes determine what parts of the neocortex are connected to the eyes, what other parts are connected to the ears, and how those parts connect to each other. Therefore, we can say that the neocortex is structured at birth to see, hear, and even learn language. But it is also true that the neocortex doesn’t know what it will see, what it will hear, and what specific languages it might learn. We can think of the neocortex as starting life having some built-in assumptions about the world but knowing nothing in particular. Through experience, it learns a rich and complicated model of the world.

We learn many high-level concepts too. It is estimated that each of us knows about forty thousand words. We have the ability to learn spoken language, written language, sign language, the language of mathematics, and the language of music. We learn how electronic forms work, what thermostats do, and even what empathy or democracy mean, although our understanding of these may differ. Independent of what other things the neocortex might do, we can say for certain that it learns an incredibly complex model of the world. This model is the basis of our predictions, perceptions, and actions.

the world can change. For example, when listening to music, the inputs from the ears change rapidly, reflecting the movement of the music. Similarly, a tree swaying in the breeze will lead to visual and perhaps auditory changes. In these two examples, the inputs to the brain are changing from moment to moment, not because you are moving but because things in the world are moving and changing on their own.

The second reason is because we move. Every time we take a step, move a limb, move our eyes, tilt our head, or utter a sound, the input from our sensors change. For example, our eyes make rapid movements, called saccades, about three times a second. With each saccade, our eyes fixate on a new point in the world and the information from the eyes to the brain changes completely. This change would not occur if we hadn’t moved our eyes.

The brain learns its model of the world by observing how its inputs change over time. There isn’t another way to learn. Unlike with a computer, we cannot upload a file into our brain. The only way for a brain to learn anything is via changes in its input...

Some things, like a melody, can be learned without moving the body. We can sit perfectly still, with eyes closed, and learn a new melody by just listening to how the sounds change over time. But most learning requires that we actively move and explore. Imagine you enter a new house, one you have not been in before. If you don’t move, there will be no changes in your sensory input, and you can’t possibly learn anything about the house. To learn a model of the house, you have to look in different directions and walk from room to room. You need to open doors, peek in draw...

How does the neocortex, which is composed of thousands of nearly identical cortical columns, learn a predictive model of the world through movement? This is the question my team and I set out to answer. Our belief was that if we could answer it, we could reverse engineer the neocortex. We would understand both what the neocortex did and how it did it. And ultimately, we would be able to build machines that worked the same way.

Thoughts and experiences are always the result of a set of neurons that are active at the same time. Individual neurons can participate in many different thoughts or experiences. Every thought you have is the activity of neurons. Everything you see, hear, or feel is also the activity of neurons. Our mental states and the activity of neurons are one and the same.

However, over the past few decades, scientists have discovered that in many parts of the brain, including the neocortex, new synapses form and old ones disappear. Every day, many of the synapses on an individual neuron will disappear and new ones will replace them. Thus, much of learning occurs by forming new connections between neurons that were previously not connected. Forgetting happens when old or unused connections are removed entirely.

Why are reference frames so important? What does the brain gain from having them? First, a reference frame allows the brain to learn the structure of something. A coffee cup is a thing because it is composed of a set of features and surfaces arranged relative to each other in space. Similarly, a face is a nose, eyes, and mouth arranged in relative positions. You need a reference frame to specify the relative positions and structure of objects.

Second, by defining an object using a reference frame, the brain can manipulate the entire object at once. For example, a car has many features arranged relative to each other. Once we learn a car, we can imagine what it looks like from different points of view or if it were stretched in one dimension. To accomplish these feats, the brain only has to rotate or stretch the reference frame and all the features of the car rotate and stretch with

Third, a reference frame is needed to plan and create movements. Say my finger is touching the front of my phone and I want to press the power button at the top. If my brain knows the current location of my finger and the location of the power button, then it can calculate the movement needed to get my finger from its current location to the d...

Humans have grid cells and place cells too. Unless you are completely disoriented, you always have a sense of where you are. I am now standing in my office. Even if I close my eyes, my sense of location persists, and I continue to know where I am. Keeping my eyes closed, I take two steps to my right and my sense of location in the room changes. The grid cells and place cells in my brain have created a map of my office, and they keep track of where I am in my office, even when my eyes are closed. As I walk, which cells are active changes to reflect my new location. Humans, rats, indeed all mammals use the same mechanism for knowing our location. We all have grid cells and place cells that create models of the places we have been.

The most important component of how brains learn continuously is the neuron. When a neuron learns a new pattern, it forms new synapses on one dendrite branch. The new synapses don’t affect previously learned ones on other branches. Thus, learning something new doesn’t force the neuron to forget or modify something it learned earlier. The artificial neurons used in today’s AI systems don’t have this ability. This is one reason they can’t learn continuously.

The unit of processing in the neocortex is the cortical column. Each column is a complete sensory-motor system—that is, it gets inputs and it can generate behaviors. With every movement, a column predicts what its next input will be. Prediction is how a column tests and updates its model.

The key to making the many-models design work is voting. Each column operates somewhat independently, but long-range connections in the neocortex allow columns to vote on what object they are sensing.

The extreme flexibility of human intelligence requires the attributes I described in this chapter: continuous learning, learning through movement, learning many models, and using general-purpose reference frames for storing knowledge and generating goal-oriented behaviors.

the sense of presence, the feeling that I am an acting agent in the world—is the core of what it means to be conscious.

Where do our perceived sensations come from? The origin of qualia is considered one of the mysteries of consciousness.

Sometimes we actually know that the same input is perceived differently by different people. A recent famous example of this is a photograph of a dress that some people see as white and gold and other people see as black and blue. The exact same picture can result in different perceptions of color. This tells us that the qualia of color is not purely a property of the physical world. If it were, we would all say the dress has the same color. The color of the dress is a property of our brain’s model of the world. If two people perceive the same input differently, that tells us their model is different.