This Is Your Brain on Music: The Science of a Human Obsession

by Daniel J. Levitin

The auditory cortex also has a tonotopic map, with low to high tones stretched out across the cortical surface. In this sense, the brain also contains a “map” of different pitches, and different areas of the brain respond to different pitches. Pitch is so important that the brain represents it directly; unlike almost any other musical attribute, we could place electrodes in the brain and be able to determine what pitches were being played to a person just by looking at the brain activity. And although music is based on pitch relations rather than absolute pitch values, it is, paradoxically, these absolute pitch values that the brain is paying attention to throughout its different stages of processing. This direct mapping of pitch is so important, it bears repeating. If I put electrodes in your visual cortex (the part of the brain at the back of the head, concerned with seeing), and I then showed you a red tomato, there is no group of neurons that will cause my electrodes to turn red. But if I put electrodes in your auditory cortex and play a pure tone in your ears at 440 Hz, there are neurons in your auditory cortex that will fire at precisely that frequency, causing the electrode to emit electrical activity at 440 Hz—for pitch, what goes into the ear comes out of the brain!

Here is a fundamental quality of music. Note names repeat because of a perceptual phenomenon that corresponds to the doubling and halving of frequencies. When we double or halve a frequency, we end up with a note that sounds remarkably similar to the one we started out with. This relationship, a frequency ratio of 2:1 or 1:2, is called the octave. It is so important that, in spite of the large differences that exist between musical cultures—between Indian, Balinese, European, Middle Eastern, Chinese, and so on—every culture we know of has the octave as the basis for its music, even if it has little else in common with other musical traditions.

The particular placement of the two half steps in the sequence of the major is crucial; it is not only what defines the major scale and distinguishes it from other scales, but it is an important ingredient in musical expectations. Experiments have shown that young children, as well as adults, are better able to learn and memorize melodies that are drawn from scales that contain unequal distances such as this. The presence of the two half steps, and their particular positions, orient the experienced, acculturated listener to where we are in the scale.

When a sound is generated on a piano, flute, or any other instrument—including percussion instruments like drums and cowbells—it produces many modes of vibration occurring simultaneously. When you listen to a single note played on an instrument, you’re actually hearing many, many pitches at once, not a single pitch. Most of us are not aware of this consciously, although some people can train themselves to hear this. The one with the slowest vibration rate—the one lowest in pitch—is referred to as the fundamental frequency, and the others are collectively called overtones.

When an instrument creates energy at frequencies that are integer multiples such as this, we say that the sound is harmonic, and we refer to the pattern of energy at different frequencies as the overtone series. There is evidence that the brain responds to such harmonic sounds with synchronous neural firings—the neurons in auditory cortex responding to each of the components of the sound synchronize their firing rates with one another, creating a neural basis for the coherence of these sounds. The brain is so attuned to the overtone series that if we encounter a sound that has all of the components except the fundamental, the brain fills it in for us in a phenomenon called restoration of the missing fundamental. A sound composed of energy at 100 Hz, 200 Hz, 300 Hz, 400 Hz, and 500 Hz is perceived as having a pitch of 100 Hz, its fundamental frequency.

But if we artificially create a sound with energy at 200 Hz, 300 Hz, 400 Hz, and 500 Hz (leaving off the fundamental), we still perceive it as having a pitch of 100 Hz. We don’t perceive it as having a pitch of 200 Hz, because our brain “knows” that a normal, harmonic sound with a pitch of 200 Hz would have an overtone series of 200 Hz, 400 Hz, 600 Hz, 800 Hz, etc. We can also fool the brain by playing sequences that deviate from the overtone series such as this: 100 Hz, 210 Hz, 302 Hz, 405 Hz, etc. In cases like these, the perceived pitch shifts away from 100 Hz in a compromise between what is presented and what a normal harmonic series would imply.

One could imagine an alien species that does not have ears, or that doesn’t have the same internal experience of hearing that we do. But it would be difficult to imagine an advanced species that had no ability whatsoever to sense vibrating objects. Where there is atmosphere there are molecules that vibrate in response to movement. And knowing whether something is generating noise or moving toward us or away from us, even when we can’t see it (because it is dark, our eyes aren’t attending to it, or we’re asleep) has a great survival value. Because most physical objects cause molecules to vibrate in several modes at once, and because for many, many objects the modes bear simple integer relations to one another, the overtone series is a fact-of-the-world that we expect to find everywhere we look: in North America, in Fiji, on Mars, and on the planets orbiting Antares. Any organism that evolved in a world with vibrating objects is likely—given enough evolutionary time—to have evolved a processing unit in the brain that incorporated these regularities of its world. Because pitch is a fundamental cue to an object’s identity, we would expect to find tonotopic mappings as we do in human auditory cortex, and synchronous neural firings for tones that bear octave and other harmonic relations to one another; this would help the brain (alien or terrestrial) to figure out that all these tones probably originated from the same object.

When you hear a saxophone playing a tone with a fundamental frequency of 220 Hz, you are actually hearing many tones, not just one. The other tones you hear are integer multiples of the fundamental: 440, 660, 880, 1100, 1320, 1540, etc. These different tones—the overtones—have different intensities, and so we hear them as having different loudnesses. The particular pattern of loudnesses for these tones is distinctive of the saxophone, and they are what give rise to its unique tonal color, its unique sound—its timbre. A violin playing the same written note (220 Hz) will have overtones at the same frequencies, but the pattern of how loud each one is with respect to the others will be different. Indeed, for each instrument, there exists a unique pattern of overtones. For one instrument, the second overtone might be louder than in another, while the fifth overtone might be softer. Virtually all of the tonal variation we hear—the quality that gives a trumpet its trumpetiness and that gives a piano its pianoness—comes from the unique way in which the loudnesses of the overtones are distributed.

The neural basis for this striking accuracy is probably in the cerebellum, which is believed to contain a system of timekeepers for our daily lives and to synchronize to the music we are hearing. This means that somehow, the cerebellum is able to remember the “settings” it uses for synchronizing to music as we hear it, and it can recall those settings when we want to sing a song from memory.

One counterintuitive point is that loudness is, like pitch, an entirely psychological phenomenon, that is, loudness doesn’t exist in the world, it only exists in the mind. And this is true for the same reason that pitch only exists in the mind. When you’re adjusting the output of your stereo system, you’re technically increasing the amplitude of the vibration of molecules, which in turn is interpreted as loudness by our brains. The point here is that it takes a brain to experience what we call “loudness.” This may seem largely like a semantic distinction, but it is important to keep our terms straight. Several odd anomalies exist in the mental representation of amplitude, such as loudnesses not being additive the way that amplitudes are (loudness, like pitch, is logarithmic), or the phenomenon that the pitch of a sinusoidal tone varies as a function of its amplitude, or the finding that sounds can appear to be louder than they are when they have been electronically processed in certain ways—such as dynamic range compression—that are often done in heavy metal music.

The ratio between the loudest sound we can hear without causing permanent damage and the softest sound we can detect is a million to one, when measured as sound-pressure levels in the air; on the dB scale this is 120 dB. The range of loudnesses we can perceive is called the dynamic range. Sometimes critics talk about the dynamic range that is achieved on a high-quality music recording; if a record has a dynamic range of 90 dB, it means that the difference between the softest parts on the record and the loudest parts is 90 dB—considered high fidelity by most experts, and beyond the capability of most home audio systems.

Our ears compress sounds that are very loud in order to protect the delicate components of the middle and inner ear. Normally, as sounds get louder in the world, our perception of the loudness increases proportionately to them. But when sounds are really loud, a proportional increase in the signal transmitted by the eardrum would cause irreversible damage. The compression of the sound levels—of the dynamic range—means that large increases in sound level in the world create much smaller changes of level in our ears. The inner hair cells have a dynamic range of 50 decibels (dB) and yet we can hear over a 120 dB dynamic range. For every 4 dB increase in sound level, a 1 dB increase is transmitted to the inner hair cells. Most of us can detect when this compression is taking place; compressed sounds have a different quality.

Here are some landmarks for sound levels, expressed in dB (SPL): 0 dB Mosquito flying in a quiet room, ten feet away from your ears 20 dB A recording studio or a very quiet executive office 35 dB A typical quiet office with the door closed and computers off 50 dB Typical conversation in a room 75 dB Typical, comfortable music listening level in headphones 100–105 dB Classical music or opera concert during loud passages; some portable music players go to 105 dB 110 dB A jackhammer three feet away 120 dB A jet engine heard on the runway from three hundred feet away; a typical rock concert 126–130 dB Threshold of pain and damage; a rock concert by the Who (note that 126 dB is four times as loud as 120 dB) 180 dB Space shuttle launch

250–275 dB Center of a tornado; volcanic eruption

So far, we’ve been able to figure out that the brain stem and the dorsal cochlear nucleus—structures that are so primitive that all vertebrates have them—can distinguish between consonance and dissonance; this distinction happens before the higher level, human brain region—the cortex—gets involved.

The filling-in phenomenon I’ve described is not just a laboratory curiosity; composers exploit this principle as well, knowing that our perception of a melodic line will continue, even if part of it is obscured by other instruments. Whenever we hear the lowest notes on the piano or double bass, we are not actually hearing 27.5 or 35 Hz, because those instruments are typically incapable of producing much energy at these ultralow frequencies: Our ears are filling in the information and giving us the illusion that the tone is that low.

Musical training appears to have the effect of shifting some music processing from the right (imagistic) hemisphere to the left (logical) hemisphere, as musicians learn to talk about—and perhaps think about—music using linguistic terms. And the normal course of development seems to cause greater hemispheric specialization: Children show less lateralization of musical operations than do adults, regardless of whether they are musicians or not.

How does a multiple-trace memory model account for the fact that we extract invariant properties of melodies as we are listening to them? As we attend to a melody, we must be performing calculations on it; in addition to registering the absolute values, the details of its presentation—details such as pitch, rhythms, tempo, and timbre—we must also be calculating melodic intervals and tempo-free rhythmic information. Neuroimaging studies from Robert Zatorre and his colleagues at McGill have suggested this is the case. Melodic “calculation centers” in the dorsal (upper) temporal lobes—just above your ears—appear to be paying attention to interval size and distances between pitches as we listen to music, creating a pitch-free template of the very melodic values we will need in order to recognize songs in transposition. My own neuroimaging studies have shown that familiar music activates both these regions and the hippocampus, a structure deep in the center of the brain that is known to be crucial to memory encoding and retrieval. Together, these findings suggest that we are storing both the abstract and the specific information contained in melodies. This may be the case for all kinds of sensory stimuli.

According to the multiple-trace memory models, every experience is potentially encoded in memory. Not in a particular place in the brain, because the brain is not like a warehouse; rather, memories are encoded in groups of neurons that, when set to proper values and configured in a particular way, will cause a memory to be retrieved and replayed in the theater of our minds. The barrier to being able to recall everything we might want to is not that it wasn’t “stored” in memory, then; rather, the problem is finding the right cue to access the memory and properly configure our neural circuits. The more we access a memory, the more active become the retrieval and recollection circuits, and the more facile we are with the cues necessary to get at the memory. In theory, if we only had the right cues, we could access any past experience.

A song playing comprises a very specific and vivid set of memory cues. Because the multiple-trace memory models assume that context is encoded along with memory traces, the music that you have listened to at various times in your life is cross-coded with the events of those times. That is, the music is linked to events of the time, and those events are linked to the music.

And the idea that emotions might be bound up with cerebellar neurons makes sense too. The most crucial survival activities often involve running—away from a predator or toward escaping prey—and our ancestors needed to react quickly, instantly, without analyzing the situation and studying the best course of action. In short, those of our ancestors who were endowed with an emotional system that was directly connected to their motor system could react more quickly, and thus live to reproduce and pass on those genes to another generation.

And although many people say that music lessons didn’t take, cognitive neuroscientists have found otherwise in their laboratories. Even just a small exposure to music lessons as a child creates neural circuits for music processing that are enhanced and more efficient than for those who lack training. Music lessons teach us to listen better, and they accelerate our ability to discern structure and form in music, making it easier for us to tell what music we like and what we don’t like.

The strongest evidence for the talent position is that some people simply acquire musical skills more rapidly than others. The evidence against the talent account—or rather, in favor of the view that practice makes perfect—comes from research on how much training the experts or high achievement people actually do. Like experts in mathematics, chess, or sports, experts in music require lengthy periods of instruction and practice in order to acquire the skills necessary to truly excel. In several studies, the very best conservatory students were found to have practiced the most, sometimes twice as much as those who weren’t judged as good. In another study, students were secretly divided into two groups (not revealed to the students so as not to bias them) based on teachers’ evaluations of their ability, or the perception of talent. Several years later, the students who achieved the highest performance ratings were those who had practiced the most, irrespective of which “talent” group they had been assigned to previously. This suggests that practice is the cause of achievement, not merely something correlated with it. It further suggests that talent is a label that we’re using in a circular fashion: When we say that someone is talented, we think we mean that they have some innate predisposition to excel, but in the end, we only apply the term retrospectively, after they have made significant achievements.

The ten-thousand-hours theory is consistent with what we know about how the brain learns. Learning requires the assimilation and consolidation of information in neural tissue. The more experiences we have with something, the stronger the memory/learning trace for that experience becomes. Although people differ in how long it takes them to consolidate information neurally, it remains true that increased practice leads to a greater number of neural traces, which can combine to create a stronger memory representation. This is true whether you subscribe to multiple-trace theory or any number of variants of theories in the neuroanatomy of memory: The strength of a memory is related to how many times the original stimulus has been experienced.

Memory strength is also a function of how much we care about the experience. Neurochemical tags associated with memories mark them for importance, and we tend to code as important things that carry with them a lot of emotion, either positive or negative. I tell my students if they want to do well on a test, they have to really care about the material as they study it. Caring may, in part, account for some of the early differences we see in how quickly people acquire new skills. If I really like a particular piece of music, I’m going to want to practice it more, and because I care about it, I’m going to attach neurochemical tags to each aspect of the memory that label it as important: The sounds of the piece, the way I move my fingers, if I’m playing a wind instrument the way that I breathe—all these become part of a memory trace that I’ve encoded as important. Similarly, if I’m playing an instrument I like, and whose sound pleases me in and of itself, I’m more likely to pay attention to subtle differences in tone, and the ways in which I can moderate and affect the tonal output of my instrument. It is impossible to overestimate the importance of these factors; caring leads to attention, and together they lead to measurable neurochemical changes. Dopamine, the neurotransmitter associated with emotional regulation, alertness, and mood, is released, and the dopaminergic system aids in the encoding of the memory trace.

We also know that, on average, successful people have had many more failures than unsuccessful people. This seems counterintuitive. How could successful people have failed more often than everyone else? Failure is unavoidable and sometimes happens randomly. It’s what you do after the failure that is important. Successful people have a stick-to-it-iveness. They don’t quit. From the president of FedEx to the novelist Jerzy Kosinsky, from Van Gogh to Bill Clinton to Fleetwood Mac, successful people have had many, many failures, but they learn from them and keep going. This quality might be partly innate, but environmental factors must also play a role.

The best guess that scientists currently have about the role of genes and the environment in complex cognitive behaviors is that each is responsible for about 50 percent of the story. Genes may transmit a propensity to be patient, to have good eye-hand coordination, or to be passionate, but certain life events—life events in the broadest sense, meaning not just your conscious experiences and memories, but the food you ate and the food your mother ate while you were in her womb—can influence whether a genetic propensity will be realized or not. Early life traumas, such as the loss of a parent, or physical or emotional abuse, are only the obvious examples of environmental influences causing a genetic predisposition to become either heightened or suppressed. Because of this interaction, we can only make predictions about human behavior at the level of a population, not an individual. In other words, if you know that someone has a genetic predisposition toward criminal behavior, you can’t make any predictions about whether he will end up in jail in the next five years. On the other hand, knowing that a hundred people have this predisposition, we can predict that some percentage of them will probably wind up in jail; we simply don’t know which ones. And some will never get into any trouble at all. The same applies to musical genes we may find someday. All we can say is that a group of people with those genes is more likely to produce expert musicians, but we cannot know which i...

What most of us turn to music for is an emotional experience. We aren’t studying the performance for wrong notes, and so long as they don’t jar us out of our reverie, most of us don’t notice them. So much of the research on musical expertise has looked for accomplishment in the wrong place, in the facility of fingers rather than the expressiveness of emotion. I recently asked the dean of one of the top music schools in North America about this paradox: At what point in the curriculum is emotion and expressivity taught? Her answer was that they aren’t taught. “There is so much to cover in the approved curriculum,” she explained, “repertoire, ensemble, and solo training, sight singing, sight reading, music theory—that there simply isn’t time to teach expressivity.” So how do we get expressive musicians? “Some of them come in already knowing how to move a listener. Usually they’ve figured it out themselves somewhere along the line.” The surprise and disappointment in my face must have been obvious. “Occasionally,” she added, almost in a whisper, “if there’s an exceptional student, there’s time during the last part of their last semester here to coach them on emotion . . . . Usually this is for people who are already performing as soloists in our orchestra, and we help them to coax out more expressivity from their performance.” So, at one of the best music schools we have, the raison d’ětre for music is taught to a select few, and then, only in the last few weeks of a four- or...

The pianist Alfred Brendel says he doesn’t think about notes when he’s onstage; he thinks about creating an experience. Stevie Wonder told me that when he’s performing, he tries to get himself into the same frame of mind and “frame of heart” that he was in when he wrote the song; he tries to capture the same feelings and sentiment, and that helps him to deliver the performance. What this means in terms of how he sings or plays differently is something no one knows. From a neuroscientific perspective, though, this makes perfect sense. As we’ve seen, remembering music involves setting the neurons that were originally active in the perception of a piece of music back to their original state—reactivating their particular pattern of connectivity, and getting the firing rates as close as possible to their original levels. This means recruiting neurons in the hippocampus, amygdala, and temporal lobes in a neural symphony orchestrated by attention and planning centers in the front lobe.

When musicians memorize songs, then, they are relying on a structure for their memory, and the details fit into that structure. This is an efficient and parsimonious way for the brain to function. Rather than memorizing every chord or every note, we build up a framework within which many different songs can fit, a mental template that can accommodate a large number of musical pieces.

The point is that musicians don’t typically learn new pieces one note at a time once they have reached a certain level of experience, knowledge, and proficiency. They can scaffold on the previous pieces they know, and just note any variations from the standard schema. Memory for playing a musical piece therefore involves a process very much like that for music listening as we saw in Chapter 4, through establishing standard schemas and expectation. In addition, musicians use chunking, a way of organizing information similar to the way chess players, athletes, and other experts organize information. Chunking refers to the process of tying together units of information into groups, and remembering the group as a whole rather than the individual pieces.

Musicians also use chunking in several ways. First, they tend to encode in memory an entire chord, rather than the individual notes of the chord; they remember “C major 7” rather than the individual tones C - E - G - B, and they remember the rule for constructing chords, so that they can create those four tones on the spot from just one memory entry. Second, musicians tend to encode sequences of chords, rather than isolated chords. “Plagal cadence,” “aeolian cadence,” “twelve-bar minor blues with a V-I turnaround,” or “rhythm changes” are shorthand labels that musicians use to describe sequences of varying lengths. Having stored the information about what these labels mean allows the musician to recall big chunks of information from a single memory entry. Third, we obtain knowledge as listeners about stylistic norms, and as players about how to produce these norms. Musicians know how to take a song and apply this knowledge—schemas again—to make the song sound like salsa, or grunge, or disco, or heavy metal; each genre and era has stylistic tics or characteristic rhythmic, timbral, or harmonic elements that define it. We can encode those in memory holistically, and then retrieve these features all at once.

Several studies have found microstructural changes in the cerebellum after the acquisition of motor skills, such as are acquired by musicians, including an increased number and density of synapses. Schlaug found that musicians tended to have larger cerebellums than nonmusicians, and an increased concentration of gray matter; gray matter is that part of the brain that contains the cell bodies, axons, and dendrites, and is understood to be responsible for information processing, as opposed to white matter, which is responsible for information transmission.

Young children start to show a preference for the music of their culture by age two, around the same time they begin to develop specialized speech processing. At first, children tend to like simple songs, where simple means music that has clearly defined themes (as opposed to, say, four-part counterpoint) and chord progressions that resolve in direct and easily predictable ways. As they mature, children start to tire of easily predictable music and search for music that holds more challenge. According to Mike Posner, the frontal lobes and the anterior cingulate—a structure just behind the frontal lobes that directs attention—are not fully formed in children, leading to an inability to pay attention to several things at once; children show difficulty attending to one stimulus when distracters are present. This accounts for why children under the age of eight or so have so much difficulty singing “rounds” like “Row, Row, Row Your Boat.” Their attentional system—specifically the network that connects the cingulate gyrus (the larger structure within which the anterior cingulate sits) and the orbitofrontal regions of the brain—cannot adequately filter out unwanted or distracting stimuli. Children who have not yet reached the developmental stage of being able to exclude irrelevant auditory information face a world of great sonic complexity with all sounds coming in as a sensory barrage. They may try to follow the part of the song that their group is supposed to be singing, only ...

Part of the reason we remember songs from our teenage years is because those years were times of self-discovery, and as a consequence, they were emotionally charged; in general, we tend to remember things that have an emotional component because our amygdala and neurotransmitters act in concert to “tag” the memories as something important. Part of the reason also has to do with neural maturation and pruning; it is around fourteen that the wiring of our musical brains is approaching adultlike levels of completion.

A third argument in favor of music’s primacy in human (and protohuman) evolution is that music evolved because it promoted cognitive development. Music may be the activity that prepared our pre-human ancestors for speech communication and for the very cognitive, representational flexibility necessary to become humans. Singing and instrumental activities might have helped our species to refine motor skills, paving the way for the development of the exquisitely fine muscle control required for vocal or signed speech. Because music is a complex activity, Trehub suggests that it may help prepare the developing infant for its mental life ahead. It shares many of the features of speech and it may form a way of “practicing” speech perception in a separate context. No human has ever learned language by memorization. Babies don’t simply memorize every word and sentence they’ve ever heard; rather, they learn rules and apply them in perceiving and generating new speech. One piece of evidence for this is empirical; the other is logical. The empirical evidence comes from what linguists call overextension: Children just learning the rules of language often apply them logically, but incorrectly. We see this most clearly in the case of irregular verb conjugations and irregular plurals in English. The developing brain is primed to make new neural connections and to prune away old ones that are not useful or accurate, and its mission is to instantiate rules insofar as possible. This is why ...

Music is also generative. For every musical phrase I hear, I can always add a note to the beginning, end, or middle to generate a new musical phrase.