Consciousness: Confessions of a Romantic Reductionist (The MIT Press)

by Christof Koch

As a rule, the farther removed a region of the visual brain is from the retina, the more strongly felt is consciousness’ influence. As expectations, biases, and memory come to play a larger role in higher regions of the brain, the impact of the external world weakens correspondingly. The subjective mind manifests itself most strongly in the upper echelon of the cortex. That is its habitat. This implies that not just any cortical activity is sufficient for conscious sensation. Even though a million neurons in the primary visual cortex are busily firing away, their exuberant spiking may not give rise to an experience, if higher-up neurons are not reflecting this activity.

Nonetheless, sensitive tests indicated that invisible pictures of naked women attracted the attention of straight men, whereas images of naked men repulsed them. Yet this occurred sub rosa, under the radar of consciousness. The volunteers didn’t see the nude yet still paid attention. Conversely, the attention of straight women—and of gay men—was attracted by invisible pictures of naked men. Functionally, this makes sense, as your brain needs to know about potential mates. It also affirms a widespread cliché about the unconscious nature of desire.

Although the responses of cells in the retina and primary visual cortex do share certain attributes with visual consciousness some of the time, during other conditions their responses can differ greatly. Let me give you three examples of why you don’t see with your eyes—something that painters have known for centuries.

However, retinal nerve cells, like those in the primary visual cortex, are incapable of distinguishing between object motion and eye motion. They react to both. Unlike smart phones, they do not have an accelerometer or a GPS sensor attached to them that distinguishes external, image motion from internal, ocular motion. It is neurons in the higher reaches of the visual cortex that produce your perception that the world is stationary.

These are just three of numerous dissociations between the state of retinal neurons and what you consciously see. The myriad action potentials streaming up your optic nerve carry data that are heavily edited before they become part of the neural correlates of consciousness. And sometimes retinal information is dispensed with altogether, such as when you close your eyes and conjure up Winnie the Pooh, your inseparable childhood companion, or when you dream about him.

Given its strategic location, it is ironic that the primary visual cortex is not even necessary for all forms of visual perception. Brain imaging in dreaming volunteers (no easy feat, given the tight quarters and the noisy banging inside a scanner) suggests that activity in the primary visual cortex is curtailed in the rapid eye movement (REM) phase of sleep, when most dreams occur, compared with non-REM sleep, when little dreaming takes place. Furthermore, patients with damage to their primary visual cortex dream without any concomitant loss of visual content.

Binocular rivalry is another one. In binocular rivalry, a small picture, say of a face, is shown to your left eye, and another photo, say the old imperial flag of Japan (with its rays of light emanating from a central disk) to your right eye. You might think that you see the face superimposed onto the flag. If the illusion is set up correctly, though, you will perceive the face alternating with the flag. Your brain won’t let you see two things at the same time in the same place. At first, you vividly see the face without a hint of the sunburst pattern; after a couple of seconds, a patch of the flag appears somewhere in your field of view, erasing the underlying facial pattern at that location. From this initial seed, the image spreads until the face is entirely gone and only the flag remains. Then the eyes start to shine through. The patchwork of flag–face image resolves itself a few seconds later into a whole face. Then the percept of the flag establishes dominance again. And on and on. The two images move in and out of consciousness in a never-ending dance. You can abort this dance by closing one eye—you’ll instantly resolve all ambiguity and perceive the image present to the open eye.

These are early days. We cannot yet pinpoint which regions of the brain underlie consciousness. But this is barking up the wrong tree—we must resist the hypnotic appeal of hot spots in brain scans with their naïve phrenological interpretation: the perception of faces is computed over here, pain over there, and consciousness just yonder. Consciousness does not arise from regions but from highly networked neurons within and across regions.

One singular feature of the brain that has only become apparent over the past two decades is the astounding heterogeneity of neurons. The approximately 100,000 neurons packed below each square millimeter of cortex, an area about the size of the letter “o” on this page, are highly heterogeneous. They can be distinguished based on their location, the shape and morphology of their dendrites, the architecture of their synapses, their genetic makeup, their electrophysiologic character, and the places to which they send their axons.

What is of great interest to the neuroscientist is when the damage is limited and circumscribed. The fact that losing a particular chunk of nervous tissue turns the world into gray tones and well-known faces into unfamiliar ones indicates that this region must be, at least partially, responsible for generating the sense of color or face identity.

A pure loss of color perception is called achromatopsia. This is quite different from everyday, hereditary color blindness that mainly afflicts men. Because they miss the gene for one of the color pigments in the eye, these dichromats don’t perceive as rich a color palette as normal sighted people (trichromats) with three retinal photopigments. Achromatopsia, in contrast, follows destruction of the color center in the visual cortex. In its wake, all hues are leached from the world. No more glorious violet- and purple-tinted Alpenglow in the setting sun. Instead, the world is experienced in chiaroscuro, like a color television that has reverted to shades of black and white. Notably, colors words and associations with colors (e.g., “red” with “fire truck”) remain.

Face-blindness leads to social isolation and shyness because the afflicted have difficulty recognizing, let alone naming, the people with whom they are conversing. They adopt coping strategies, focusing on some singular mark, a mole or a prominent nose, a brightly colored shirt, or on the voice. Makeup and changing hairstyles impede recognition, as do groups of people in uniforms.

The linkage between cortical location and function is a hallmark of nervous systems. Contrast this with another vital organ, the liver. Like the brain, it weighs about three pounds and has a left and a right lobe. But liver tissue is much less differentiated and more homogeneous than nervous tissue. Liver impairment is proportional to the amount of damage, with little regard to where the destruction is located.

How can the content of consciousness be so replete with particular and evocative details? There are no pictures inside my skull of me climbing, only a brown-grayish organ with the consistency, size, and shape of overcooked cauliflower. This tofu-like tissue, buffered by blood and cerebrospinal fluid, consists of nerve and glial cells. Neurons and their interconnecting synapses are the atoms of perception, memory, thought, and action. If science is ever to comprehend these processes, it must be able to explain them in terms of the interactions of large coalitions of neurons embedded within a network of unfathomable complexity.

But a few neurons are much more discerning. I was thrilled when Gabriel showed me the first such cells. One fired only when the patient was looking at photos of then-President Bill Clinton, but not other famous people, and the other responded exclusively to cartoons of Bart and Homer Simpson.

Itzhak refers to these cells as concept neurons. We try not to anthropomorphize them, to avoid the temptation to call them “Jennifer Aniston cells” (the cells don’t like it when you do that!). Each cell, together with its sisters—for there are likely thousands of such cells in the medial temporal lobe for any one idea—encodes a concept, such as Jennifer Aniston, no matter whether the patient sees or hears her name, looks at her picture, or imagines her. Think of them as the cellular substrate of the Platonic Ideal of Jennifer Aniston. Whether the actress is sitting or running, whether her hair is up or down, as long as the patient recognizes Jennifer Aniston, those neurons are active.

Many neurons in visual cortex react to a line with a particular orientation, to a patch of gray, or to a generic face with promiscuous exuberance, whereas concept cells in the medial temporal lobe are considerably more restrained. Any one individual or thing evokes activity in only a very small fraction of neurons. This is known as a sparse representation.

Concept cells demonstrate compellingly that the specificity of conscious experience has a direct counterpart at the cellular level.

Most cells are silent most of the time, the essence of a sparse representation.

Moran arranged the feedback such that the more this cell fired relative to the other one, the more visible Brolin became and the more the image of Monroe faded and vice versa. The image on the screen kept changing until only Brolin or only Monroe remained visible and the trial was over. The patient loved it, as she controlled the movie purely with her thoughts.

The way I tell the story here, it sounds like there are two principals, the way the puppeteer Craig occupied the head of actor John Malkovich in the movie Being John Malkovich. One is the patient’s mind, concentrating on Monroe. The other is the patient’s brain—namely, the nerve cells in the medial temporal lobe that up- and down-regulate their activity according to the mind’s desire. But both are part of the same person. So who is in control of whom? Who is the puppeteer and who the puppet?

The patient can quite deliberately and very selectively turn the volume on her medial temporal lobe neurons up and down. But many regions of the brain will be immune from this influence. For instance, you can’t will yourself to see the world in shades of gray. This most likely means that you can’t consciously downregulate color neurons in your visual cortex. And much as you may sometimes want to, you can’t turn off the pain centers in your brain.

All the weirdness of the mind–body nexus is apparent here. The patient doesn’t feel an itch every time the Monroe neuron fires; she doesn’t think, “Suppression, suppression, suppression,” to banish Brolin from the screen. She has absolutely no idea whatsoever what goes on inside her head. Yet the thought of Monroe translates into a particular pattern of neuronal activity. Events in her phenomenal mind find their parallel in her material brain. A mind-quake occurs simultaneously with a brain-quake.

The brain, like the rest of the body, has a remarkable degree of bilateral symmetry. It is helpful to think of it as an enlarged walnut. One side is not the exact mirror image of the other, but approximately so. Almost every brain structure has two copies, one on the left and one on the right. The left side of the visual field is represented by the visual cortex in the right hemisphere, whereas the right side is mapped onto the left visual cortex. When you look out at the world, you don’t see a fine vertical line running down your field of view; the two hemifields are integrated seamlessly. Philosophers emphasize that experience is unitary. You don’t experience two streams of consciousness, one for each side, but only one. And what is true for vision applies with equal force to touch, to hearing, and so on.

The corpus callosum, the largest white matter structure in the brain, is primarily responsible for this integration. It is a thick bundle of about 200 million axons, each extending from a pyramidal cell on one side of the brain to the other side. These axons, together with some minor wire bundles, tightly coordinate the activities of the two cerebral hemispheres so that they work together effortlessly, giving rise to a single view of the world.

In certain cases of intractable epileptic seizures, part or all of the corpus callosum is cut to prevent a seizure that originates in one hemisphere from spreading into the other and causing generalized convulsions. This operation, first performed in the early 1940s, alleviates seizures. Remarkably, split-brain patients, once they’ve recovered from the surgery, are inconspicuous in everyday life. They see, hear, and smell as before, they move about, talk, and interact appropriately with other people, and their IQ is unchanged. They have their usual sense of self and report no obvious alteration in their perception of the world—no shrinkage of their visual field, for example. The surgeons who pioneered this operation, such as Joseph Bogen at Loma Linda University in Southern California, were puzzled by this lack of clear symptoms.

If a key is placed into his right hand, which is under the table, out of sight, the patient will quickly name it. The touch information from his right hand is transmitted to his left hemisphere, where the object is identified and its label relayed to the language center. If the key is placed into the person’s left hand, however, he is at a loss to say what it is and rambles on. The right hemisphere might very well know that the object is a key, but it cannot convey this knowledge to the language centers on the left, because the communication links have been cut.

When asked how many seizures she had following her operation, her right hand held up two fingers. Her left hand then reached over and forced the fingers on her right hand down. After trying several times to tally her seizures, she paused and then simultaneously displayed three fingers with her right hand and one with her left. When Mark pointed out this discrepancy, the patient commented that her left hand frequently did things on its own.

Studies with split-brain patients, work for which Sperry was awarded the Nobel Prize in 1981, teach us that cutting the corpus callosum cleaves the cortico-thalamic complex in two but leaves consciousness intact. Both hemispheres are independently capable of conscious experience, one being much more verbal than the other. Whatever the neural correlates of consciousness, they must exist independently in both hemispheres of the cerebral cortex. Two conscious minds in one skull.

Even when your body is asleep, you can have vivid experiences in your dreams. In contrast, during deep sleep, anesthesia, fainting, concussion, and coma, there is no experience at all. Not a black screen but nada.

In the United States alone, as many as 25,000 patients hover for years in a vegetative state termed persistent vegetative state (PVS), with bleak prospects for recovery. What makes the situation almost unbearable is that unlike comatose patients, who exhibit almost no reflexes, patients in this limbo state have daily sleep–wake cycles.

She had brief episodes of automatisms: head turning, eye movements, and the like, but no reproducible or consistent, purposeful behavior. Her EEG was flat, indicating that her cerebral cortex had shut down. Her condition failed to improve over many years. The autopsy showed that her cortex had shrunk by half, with her visual centers atrophied; so contrary to public reports circulating at the time, she couldn’t have seen anything.

The patient showed no signs of comprehending, let alone responding. Yet the pattern of hemodynamic brain activity was similar to that of healthy volunteers who closed their eyes and imagined similar actions. Such fantasizing is a complex and purposeful mental activity that takes place over minutes: It is unlikely to occur unconsciously. The injured woman, despite her inability to signal with her hands, eyes, or voice, was at least sporadically conscious and able to follow an external command. Most other vegetative state patients who were tested in this manner had no such brain signatures; they appear to be truly not conscious. The MRI scanner could be a lifeline for grievously brain-injured patients because it opens up a means of communication: “If you are in pain, think of playing tennis. If not, imagine walking through your house.”

If both the left and the right copies of a subcortical region are destroyed, the patient may lose consciousness permanently. (In general, the brain tolerates injury to a structure on one side but is much less resistant to damage to both sides.)

A plethora of nuclei in the thalamus and the brain stem keep the forebrain sufficiently aroused for experience to occur. None of these structures, with their distinct chemical signatures, is responsible for generating the content of that experience, but they make experience possible. The endpoint of their efforts is the sixteen billion neurons in the cerebral cortex and their close associates in the thalamus, the amygdala, the claustrum, and the basal ganglia. By controlling the release of a cocktail of neurotransmitters, the intralaminar nuclei and other nuclei in the catacombs of the brain tune synaptic and neuronal activity up or down, enabling the cortico-thalamic complex to form and shape the tightly synchronized coalition of neurons that is at the heart of any one conscious experience.

What does man actually know about himself? Is he, indeed, ever able to perceive himself completely, as if laid out in a lighted display case? Does nature not conceal most things from him—even concerning his own body—in order to confine and lock him within a proud, deceptive consciousness, aloof from the coils of the bowels, the rapid flow of the bloodstream, and the intricate quivering of the fibers! She threw away the key. —Friedrich Nietzsche, On Truth and Lying in an Extramoral Sense (1873)

This singular, yet universal experience vividly brought home the insight that much of what goes on in my head is not accessible to me. Somewhere in the brain, my body is monitored; love, joy, and fear are born; thoughts arise, are mulled over, and discarded; plans are made; and memories are laid down. The conscious me, Christof, is oblivious to all of this furious activity.

Suppressing knowledge of the certain doom that awaits us all must have been a major factor in the evolution of what Freud calls defense mechanisms (are we the only animals with them? can a chimp suppress or repress?). These are processes by which the brain removes negative feelings, anxiety, guilt, unbidden thoughts, and so on from consciousness. Without such cleansing mechanisms, early humans might have become too transfixed by their ultimate fate to successfully dominate their niche. Perhaps clinical depression amounts to a loss of such defense mechanisms.

But with the right trigger, the unconscious can manifest itself dramatically.

What I learned that day is that a symbol, in the right context, can abruptly release long-dormant memories and emotions.

now understand that the actions of the sovereign “I” are determined by habits, instincts, and impulses that largely bypass conscious inspection.

Spelunking the caverns of your own subterranean desires, dreams, and motivations, rendering them conscious, and thereby, maybe, making them comprehensible is very difficult. Psychoanalysis and other inference methods are imperfect; they create a new fiction, a different narrative based on intuitive, folk-psychological notions about why people do the things they do. The talking cure may never unearth the actual reasons why a relationship broke asunder: These remain consigned to the dark cellars of the brain, where consciousness does not cast its prying light.

Friedrich Nietzsche was the first major Western thinker to explore the darker recesses of humanity’s unconscious desire to dominate others and acquire power over them, frequently disguised as compassion.

These Freudian concepts have slipped into everyday language and are only slowly being replaced by more brain-based ones.

I won’t be discussing case studies of neurotic, upper-class patients lying on a couch and talking incessantly about themselves at a rate of $200 an hour.

Your actions are profoundly shaped by unconscious processes to which you are not privy.

Collectively, this zombie army manages the fluid and rapid interplay of muscles and nerves that is at the heart of all skills and that makes up a lived life.

Zombie agents carry out routine missions below the radar screen of consciousness. You can become conscious of the action of a zombie agent, but only after the fact.

action can indeed be faster than thought, with the onset of corrective motor action preceding conscious perception by about a quarter of a second. To put this into perspective, consider a world-class sprinter running one hundred meters in ten seconds. By the time he consciously hears the pistol, the runner is already several strides out of the starting block.

Unconscious agents come into being by dint of training. Repeating the same sequence over and over reinforces the individual components until they smoothly and automatically interlink. The more you train, the more effortless and synchronized the whole becomes.