Being You: A New Science of Consciousness

by Anil Seth

I will make the case that the experiences of being you, or of being me, emerge from the way the brain predicts and controls the internal state of the body. The essence of selfhood is neither a rational mind nor an immaterial soul. It is a deeply embodied biological process, a process that underpins the simple feeling of being alive that is the basis for all our experiences of self, indeed for any conscious experience at all. Being you is literally about your body.

By the end of the book, you’ll understand that our conscious experiences of the world and the self are forms of brain-based prediction—“controlled hallucinations”—that arise with, through, and because of our living bodies.

Consciousness is first and foremost about subjective experience—it is about phenomenology.

My preferred philosophical position, and the default assumption of many neuroscientists, is physicalism. This is the idea that the universe is made of physical stuff, and that conscious states are either identical to, or somehow emerge from, particular arrangements of this physical stuff. Some philosophers use the term materialism instead of physicalism, but for our purposes they can be treated synonymously.

Functionalism is the idea that consciousness does not depend on what a system is made of (its physical constitution), but only on what the system does, on the functions it performs, on how it transforms inputs into outputs. The intuition driving functionalism is that mind and consciousness are forms of information processing which can be implemented by brains, but for which biological brains are not strictly necessary.

Panpsychism is the idea that consciousness is a fundamental property of the universe, alongside other fundamental properties such as mass/energy and charge; that it is present to some degree everywhere and in everything.

Mysterianism is the idea that there may exist a complete physical explanation of consciousness—a full solution to Chalmers’s hard problem—but that we humans just aren’t clever enough, and never will be clever enough, to discover this solution, or even to recognize a solution if it were presented to us by super-smart aliens.

According to the real problem, the primary goals of consciousness science are to explain, predict, and control the phenomenological properties of conscious experience.

The deeper problem is that correlations are not explanations. We all know that mere correlation does not establish causation, but it is also true that correlation falls short of explanation.

IIT says that subjective experience is a property of patterns of cause and effect, that information is as real as mass or energy, and that even atoms may be a little bit conscious.

IIT, integrated information theory

In IIT, Φ measures the amount of information a system generates “as a whole,” over and above the amount of information generated by its parts independently. This underpins the main claim of the theory, which is that a system is conscious to the extent that its whole generates more information than its parts.

High Φ, lots of consciousness. Zero Φ, no consciousness.

Immanuel Kant realized that the chaos of unrestricted sensory data would always remain meaningless without being given structure by preexisting conceptions, which for him included a priori frameworks like space and time. Kant’s term “noumenon” refers to a “thing in itself”—Ding an sich—a mind-independent reality that will always be inaccessible to human perception, hidden behind a sensory veil.

By adjusting top-down predictions so as to suppress bottom-up prediction errors, the brain’s perceptual best guesses maintain their grip on their causes in the world. In this view, perception happens through a continual process of prediction error minimization.

We never experience sensory signals themselves, we only ever experience interpretations of them.

Both “normal” perception and “abnormal” hallucination involve internally generated predictions about the causes of sensory inputs, and both share a core set of mechanisms in the brain. The difference is that in “normal” perception, what we perceive is tied to—controlled by—causes in the world, whereas in the case of hallucination, our perceptions have, to some extent, lost their grip on these causes. When we hallucinate, our perceptual predictions are not properly updated in light of prediction errors.

You could even say that we’re all hallucinating all the time. It’s just that when we agree about our hallucinations, that’s what we call reality.

The function of perception, at least to a first approximation, is to figure out the most likely causes of the sensory signals, not to deliver awareness of the sensory signals themselves—whatever that might mean.

The controlled hallucination of our perceptual world has been designed by evolution to enhance our survival prospects, not to be a transparent window onto an external reality, a window that anyway makes no conceptual sense.

Perception is a rolling process, not a static snapshot.

And it is prediction error minimization that provides the connection between controlled hallucinations and Bayesian inference. It takes a Bayesian claim about what the brain should do and turns it into a proposal about what it actually does do. By minimizing prediction errors everywhere and all the time, it turns out that the brain is actually implementing Bayes’ rule. More precisely, it is approximating Bayes’ rule. It is this connection that licenses the idea that perceptual content is a top-down controlled hallucination, rather than a bottom-up “readout” of sensory data.

three core components of prediction error minimization in the brain: generative models, perceptual hierarchies, and the “precision weighting” of sensory signals.

We perceive the world around us in order to act effectively within it, to achieve our goals, and—in the long run—to promote our prospects of survival. We don’t perceive the world as it is, we perceive it as it is useful for us to do so.

The beholder’s share is that part of perceptual experience that is contributed by the perceiver and which is not to be found in the artwork—or the world—itself. The concept of the beholder’s share cries out to be connected with predictive theories of perception—like the controlled hallucination theory. As Kandel put it: “The insight that the beholder’s perception involves a top-down inference convinced Gombrich that there is no ‘innocent eye’: that is, all visual perception is based on classifying concepts and interpreting visual information. One cannot perceive that which one cannot classify.”

Impressionist landscapes attempt to remove the artist from the act of painting, to recover Gombrich’s “innocent eye” by imparting to the canvas the variations in brightness that are the raw materials for perceptual inference, rather than the output of this process. To do this, the artist must develop and deploy a sophisticated understanding of how the subjective, phenomenological aspects of vision come about. Each work can be understood as an exercise in reverse engineering the human visual system, from sensory input all the way to a coherent subjective experience. The paintings become experiments into predictive perception and into the nature of the conscious experiences that these processes give rise to.

When we experience the world as being “really out there,” this is not a passive revealing of an objective reality, but a vivid and present projection—a reaching out to the world from the brain.

What we call “hallucination” is what happens when perceptual priors are unusually strong, overwhelming the sensory data so that the brain’s grip on their causes in the world starts to slide.

Objecthood is a property of how visual conscious contents generally appear, rather than being a property of any single conscious experience.

objecthood. Another situation where objecthood is typically absent is in “grapheme-color synesthesia.” The term “synesthesia” refers to a kind of “mixing of the senses.” People with the grapheme-color variety have experiences of color when seeing letters: for example, the letter “A” may elicit a luminous redness, regardless of its actual color on the page. Although these color experiences happen consistently and automatically—meaning that the same color is experienced whenever a particular

In the controlled hallucination view, the purpose of perception is to guide action and behavior—to promote the organism’s prospects of survival. We perceive the world not as it is, but as it is useful for us.

To put it another way, even though perceptual properties depend on top-down generative models, we do not experience the models as models. Rather, we perceive with and through our generative models, and in doing so, out of mere mechanism a structured world is brought forth.

(We humans can also suffer “anticipatory regret”—the feeling of certainty that what I’m about to do will turn out badly, that despite knowing this, I will do it anyway, and that I and others will suffer as a consequence.)

Our perceptions may change, but this doesn’t mean that we perceive them as changing. This distinction is exemplified by the phenomenon of “change blindness,” in which slowly changing things (in the world) do not evoke any corresponding experience of change.

We do not perceive ourselves in order to know ourselves, we perceive ourselves in order to control ourselves.

We do not see things as they are, we see them as we are. ANAÏS NIN

As Conant and Ashby said, every good regulator of a system must be a model of that system.*

The control-oriented perceptions that underpin emotions and moods are all about predicting the consequences of actions for keeping the body’s essential variables where they belong.

Evolutionarily speaking, brains are not “for” rational thinking, linguistic communication, or even for perceiving the world. The most fundamental reason any organism has a brain—or any kind of nervous system—is to help it stay alive, through making sure that its physiological essential variables remain within the tight ranges compatible with its continued survival.

Allostasis means the process of achieving stability through change, as compared to the more familiar term “homeostasis,” which simply means a tendency toward a state of equilibrium.

Putting these ideas together, we perceive ourselves as stable over time in part because of a self-fulfilling prior expectation that our physiological condition is restricted to a particular range, and in part because of a self-fulfilling prior expectation that this condition does not change. In other words, effective physiological regulation may depend on systematically misperceiving the body’s internal state as being more stable than it really is, and as changing less than it really does.

Across every aspect of being a self, we perceive ourselves as stable over time because we perceive ourselves in order to control ourselves, not in order to know ourselves.

Being alive means being in a condition of low entropy.

Following the FEP, we can now say that organisms maintain themselves in the low-entropy states that ensure their continued existence by actively minimizing this measurable quantity called “free energy.” But what is free energy from the perspective of the organism? It turns out, after some mathematical juggling, that free energy is basically the same thing as sensory prediction error. When an organism is minimizing sensory prediction error, as in schemes like predictive processing and active inference, it is also minimizing this theoretically more profound quantity of free energy.

Paraphrasing Friston, the view from the FEP is of organisms gathering and modeling sensory information so as to maximize the sensory evidence for their own existence. Or, as I like to say, “I predict myself, therefore I am.”

Gerald Edelman summed it up well with a provocative one-liner: Free will—whatever you think about it, we’re determined to have it.

Voluntary actions are voluntary not because they descend from an immaterial soul, nor because they ascend from a quantum soup. They are voluntary because they express what I, as a person, want to do, even though I cannot choose these wants.

As nineteenth-century philosopher Arthur Schopenhauer put it, “Man can do what he wills, but he cannot will what he wills.”

There’s one further reason why we experience voluntary actions the way we do, a reason that puts even more clear air between volition as perceptual inference and volition as dualistic magic. Experiences of volition are useful for guiding future behavior, just as much as for guiding current behavior.

Experiences of volition are not only real, they are indispensable to our survival. They are self-fulfilling perceptual inferences that bring about voluntary actions.

Absence of language is not evidence for absence of consciousness. Neither is absence of so-called “high-level” cognitive abilities like metacognition—which is the ability, broadly speaking, to reflect on one’s thoughts and perceptions.