This Is Your Brain on Music: The Science of a Human Obsession

by Daniel J. Levitin

that contained polyphony (more than one musical part playing at a time), fearing that it would cause people to doubt the unity of God. The church also banned the musical interval of an augmented fourth, the distance between C and F-sharp and also known as a tritone (the interval in Leonard Bernstein’s West Side Story when Tony sings the name “Maria”). This interval was considered so dissonant that it must have been the work of Lucifer, and so the church named it Diabolus in musica.

Almost all of us, even without musical training, can tell if a singer is offkey; we might not be able to say whether she is sharp or flat, or by how much, but after the age of five, most humans have as well a refined ability to detect tones that are out of tune as to discriminate a question from an accusation (in English, a rising pitch indicates a question, a straight or slightly falling pitch indicates an accusation).

The unit of measurement, cycles per second, is often called Hertz (abbreviated Hz) after Heinrich Hertz, the German theoretical physicist who was the first to transmit radio waves (a dyed-in-the-wool theoretician, when asked what practical use radio waves might have, he reportedly shrugged, “None”).

The word pitch refers to the mental representation an organism has of the fundamental frequency of a sound. That is, pitch is a purely psychological phenomenon related to the frequency of vibrating air molecules.

When you ask someone a question, your voice naturally rises in intonation at the end of the sentence, signaling that you are asking. But you don’t try to make the rise in your voice match a specific pitch. It is enough that you end the sentence somewhat higher in pitch than you began it. This is a convention in English (though not in all languages—we have to learn it), and is known in linguistics as a prosodic cue.

Pitch is so important that the brain represents it directly; unlike almost any other musical attribute, we could place electrodes in the brain and be able to determine what pitches were being played to a person just by looking at the brain activity.

This direct mapping of pitch is so important, it bears repeating. If I put electrodes in your visual cortex (the part of the brain at the back of the head, concerned with seeing), and I then showed you a red tomato, there is no group of neurons that will cause my electrodes to turn red. But if I put electrodes in your auditory cortex and play a pure tone in your ears at 440 Hz, there are neurons in your auditory cortex that will fire at precisely that frequency, causing the electrode to emit electrical activity at 440 Hz—for pitch, what goes into the ear comes out of the brain!

Here is a fundamental quality of music. Note names repeat because of a perceptual phenomenon that corresponds to the doubling and halving of frequencies. When we double or halve a frequency, we end up with a note that sounds remarkably similar to the one we started out with.

The white keys are named A, B, C, D, E, F, and G. The notes between—the black keys—are the ones with compound names. The note between A and B is called either A-sharp or B-flat, and in all but formal music theoretic discussions, the two terms are interchangeable. (In fact, this note could also be referred to as C double-flat, and similarly, A could be called G double-sharp, but this is an even more theoretical usage.) Sharp means high, and flat means low. B-flat is the note one semitone lower than B; A-sharp is the note one semitone higher than A.

The frequency of each note in our system is approximately 6 percent more than the one before it. Our auditory system is sensitive both to relative changes and to proportional changes in sound. Thus, each increase in frequency of 6 percent gives us the impression that we have increased pitch by the same amount as we did last time.

In Western music we rarely use all the notes of chromatic scale in composition; instead, we use a subset of seven (or less often, five) of those twelve tones. Each of these subsets is itself a scale, and the type of scale we use has a large impact on the overall sound of a melody, and its emotional qualities. The most common subset of seven tones used in Western music is called the major scale, or Ionian mode (reflecting its ancient Greek origins).

The particular placement of the two half steps in the sequence of the major is crucial; it is not only what defines the major scale and distinguishes it from other scales, but it is an important ingredient in musical expectations. Experiments have shown that young children, as well as adults, are better able to learn and memorize melodies that are drawn from scales that contain unequal distances such as this.

For reasons that are largely cultural, we tend to associate major scales with happy or triumphant emotions, and minor scales with sad or defeated emotions.

If we build a chord starting on C and use the tones from the C major scale, we use C, E, and G. If instead we use the C minor scale, the first, third, and fifth notes are C, E-flat, and G. This difference in the third degree, between E and E-flat, turns the chord itself from a major chord into a minor chord. All of us, even without musical training, can tell the difference between these two even if we don’t have the terminology to name them; we hear the major chord as sounding happy and the minor chord as sounding sad, or reflective, or even exotic.

The brain is so attuned to the overtone series that if we encounter a sound that has all of the components except the fundamental, the brain fills it in for us in a phenomenon called restoration of the missing fundamental. A sound composed of energy at 100 Hz, 200 Hz, 300 Hz, 400 Hz, and 500 Hz is perceived as having a pitch of 100 Hz, its fundamental frequency. But if we artificially create a sound with energy at 200 Hz, 300 Hz, 400 Hz, and 500 Hz (leaving off the fundamental), we still perceive it as having a pitch of 100 Hz.

Petr sent the output of these electrodes to a small amplifier, and played back the sound of the owl’s neurons through a loudspeaker. What he heard was astonishing; the melody of “The Blue Danube Waltz” sang clearly from the loudspeakers: ba da da da da, deet deet, deet deet. We were hearing the firing rates of the neurons and they were identical to the frequency of the missing fundamental. The overtone series had an instantiation not just in the early levels of auditory processing, but in a completely different species.

After Schaeffer edited out the attack of orchestral instrument recordings, he played back the tape and found that it was nearly impossible for most people to identify the instrument that was playing. Without the attack, pianos and bells sounded remarkably unlike pianos and bells, and remarkably similar to one another. If you splice the attack of one instrument onto the steady state, or body, from another, you get varied results: In some cases, you hear an ambiguous hybrid instrument that sounds more like the instrument that the attack came from than the one the steady state came from.

Rhythm, meter, and tempo are related concepts that are often confused with one another. Briefly, rhythm refers to the lengths of notes, tempo refers to the pace of a piece of music (the rate at which you would tap your foot to it), and meter refers to when you tap your foot hard versus light, and how these hard and light taps group together to form larger units.

The average person seems to have a remarkable memory for tempo. In an experiment that Perry Cook and I published in 1996, we asked people to simply sing their favorite rock and popular songs from memory and we were interested to know how close they came to the actual tempo of the recorded versions of those songs. As a baseline, we considered how much variation in tempo the average person can detect; that turns out to be 4 percent. In other words, for a song with a tempo of 100 bpm, if the tempo varies between 96–100, most people, even some professional musicians, won’t detect this small change (although most drummers would—their job requires that they be more sensitive to tempo than other musicians, because they are responsible for maintaining tempo when there is no conductor to do it for them). A majority of people in our study—nonmusicians—were able to sing songs within 4 percent of their nominal tempo.

The neural basis for this striking accuracy is probably in the cerebellum, which is believed to contain a system of timekeepers for our daily lives and to synchronize to the music we are hearing. This means that somehow, the cerebellum is able to remember the “settings” it uses for synchronizing to music as we hear it, and it can recall those settings when we want to sing a song from memory. It allows us to synchronize our singing with a memory of the last time we sang. The basal ganglia—what Gerald Edelman has called “the organs of succession”—are almost certainly involved, as well, in generating and shaping rhythm, tempo, and meter.

The scale is logarithmic, and doubling the intensity of a sound source results in a 3 dB increase in sound. The logarithmic scale is useful for discussing sound because of the ear’s extraordinary sensitivity: The ratio between the loudest sound we can hear without causing permanent damage and the softest sound we can detect is a million to one, when measured as sound-pressure levels in the air; on the dB scale this is 120 dB.

Interestingly, if we divide the octave precisely in half, the interval we end up with is called a tritone and most people find it the most disagreeable interval possible. Part of the reason for this may be related to the fact that the tritone cannot be reduced to a simple integer ratio, its ratio being close to 41:29 (it is actually , an irrational number).

Perhaps the most documented illusion—or parlor trick—in Western classical music is the deceptive cadence. A cadence is a chord sequence that sets up a clear expectation and then closes, typically with a satisfying resolution. In the deceptive cadence, the composer repeats the chord sequence again and again until he has finally convinced the listeners that we’re going to get what we expect, but then at the last minute, he gives us an unexpected chord—not outside the key, but a chord that tells us that it’s not over, a chord that doesn’t completely resolve. Haydn’s use of the deceptive cadence is so frequent, it borders on an obsession. Perry Cook has likened this to a magic trick: Magicians set up expectations and then defy them, all without you knowing exactly how or when they’re going to do it. Composers do the same thing. The Beatles’ “For No One” ends on the V chord (the fifth degree of the scale we’re in) and we wait for a resolution that never comes—at least not in that song. But the very next song on the album Revolver starts a whole step down from the very chord we were waiting to hear, a semi-resolution (to the flat seven) that straddles surprise and release.

In “Over the Rainbow,” the melody begins with one of the largest leaps we’ve ever experienced in a lifetime of music listening: an octave. This is a strong schematic violation, and so the composer rewards and soothes us by bringing the melody back toward home again, but not by too much—he does come down, but only by one scale degree—because he wants to continue to build tension. The third note of this melody fills the gap. Sting does the same thing in “Roxanne”: He leaps up an interval of roughly a half octave (a perfect fourth) to hit the first syllable of the word Roxanne, and then comes down again to fill the gap.

Seemingly minor interventions can powerfully affect the accuracy of memory retrieval. An important series of studies was carried out by Elizabeth Loftus of the University of Washington, who was interested in the accuracy of witnesses’ courtroom testimonies. Subjects were shown videotapes and asked leading questions about the content. If shown two cars that barely scraped each other, one group of subjects might be asked, “How fast were the cars going when they scraped each other?” and another group would be asked, “How fast were the cars going when they smashed each other?” Such one-word substitutions caused dramatic differences in the eyewitnesses’ estimates of the speeds of the two vehicles. Then Loftus brought the subjects back, sometimes up to a week later, and asked, “How much broken glass did you see?” (There really was no broken glass.) The subjects who were asked the question with the word smashed in it were more likely to report “remembering” broken glass in the video. Their memory of what they actually saw had been reconstructed on the basis of a simple question the experimenter had asked a week earlier.

The record-keeping account follows an old idea of my favorite researchers, the Gestalt psychologists, who said that every experience leaves a trace or residue in the brain. Experiences are stored as traces, they said, that are reactivated when we retrieve the episodes from memory. A great deal of experimental evidence supports this theory. Roger Shepard showed people hundreds of photographs for a few seconds each. A week later, he brought the subjects back into the laboratory and showed them pairs of photographs that they had seen before, along with some new ones that they hadn’t. In many cases, the “new” photos had only subtle differences from the old, such as the angle of the sail on a sailboat, or the size of a tree in the background. Subjects were able to remember which ones they had seen a week earlier with astonishing accuracy.

The common neural mechanisms that underlie perception of music and memory for music help to explain how it is that songs get stuck in our heads. Scientists call these ear worms, from the German Ohrwurm, or simply the stuck song syndrome.

Music communicates to us emotionally through systematic violations of expectations. These violations can occur in any domain—the domain of pitch, timbre, contour, rhythm, tempo, and so on—but occur they must. Music is organized sound, but the organization has to involve some element of the unexpected or it is emotionally flat and robotic. Too much organization may technically still be music, but it would be music that no one wants to listen to. Scales, for example, are organized, but most parents get sick of hearing their children play them after five minutes.

From phylogenetic studies—studies of the brains of different animals up and down the genetic ladder—we’ve learned that the cerebellum is one of the oldest parts of the brain, evolutionarily speaking. In popular language, it has sometimes been referred to as the reptilian brain. Although it weighs only 10 percent as much as the rest of the brain, it contains 50 to 80 percent of the total number of neurons. The function of this oldest part of the brain is something that is crucial to music: timing. The cerebellum has traditionally been thought of as that part of the brain that guides movement. Most movements made by most animals have a repetitive, oscillatory quality. When we walk or run, we tend to do so at a more or less constant pace; our body settles into a gait and we maintain it. When fish swim or birds fly, they tend to flip their fins or flap their wings at a more or less constant rate. The cerebellum is involved in maintaining this rate, or gait. One of the hallmarks of Parkinson’s disease is difficulty walking, and we now know that cerebellar degeneration accompanies this disease.

Scientists can’t even agree about what emotions are. We distinguish between emotions (temporary states that are usually the result of some external event, either present, remembered, or anticipated), moods (not-so-temporary, longer-lasting states that may or may not have an external cause), and traits (a proclivity or tendency to display certain states, such as “She is generally a happy person,” or “He never seems satisfied”). Some scientists use the word affect to refer to the valence (positive or negative) of our internal states, and reserve the word emotion to refer to particular states. Affect can thus take on only two values (or a third value if you count “no affective state”) and within each we have a range of emotions: Positive emotions would include happiness and satiety, negative would include fear and anger.

Related to the startle reflex, and to the auditory system’s exquisite sensitivity to change, is the habituation circuit. If your refrigerator has a hum, you get so used to it that you no longer notice it—that is habituation. A rat sleeping in his hole in the ground hears a loud noise above. This could be the footstep of a predator, and he should rightly startle. But it could also be the sound of a branch blowing in the wind, hitting the ground above him more or less rhythmically. If, after one or two dozen taps of the branch against the roof of his house, he finds he is in no danger, he should ignore these sounds, realizing that they are no threat. If the intensity or frequency should change, this indicates that environmental conditions have changed and that he should start to notice.

We found exactly what we had hoped. Listening to music caused a cascade of brain regions to become activated in a particular order: first, auditory cortex for initial processing of the components of the sound. Then the frontal regions, such as BA44 and BA47, that we had previously identified as being involved in processing musical structure and expectations. Finally, a network of regions—the mesolimbic system—involved in arousal, pleasure, and the transmission of opioids and the production of dopamine, culminating in activation in the nucleus accumbens. And the cerebellum and basal ganglia were active throughout, presumably supporting the processing of rhythm and meter. The rewarding and reinforcing aspects of listening to music seem, then, to be mediated by increasing dopamine levels in the nucleus accumbens, and by the cerebellum’s contribution to regulating emotion through its connections to the frontal lobe and the limbic system. Current neuropsychological theories associate positive mood and affect with increased dopamine levels, one of the reasons that many of the newer antidepressants act on the dopaminergic system. Music is clearly a means for improving people’s moods. Now we think we know why.

To the newbie, the whole thing may seem chaotic. Yet, simply knowing that the improvisation takes place over the original chords and form of the song can make a big difference in orienting the neophyte to where in the song the players are. I often advise new listeners to jazz to simply hum the main tune in their mind once the improvisation begins—this is what the improvisers themselves are often doing—and that enriches the experience considerably.

He explained that, once in a while, we find a behavior or attribute in an organism that lacks any clear evolutionary basis; this occurs when evolutionary forces propagate an adaptation for a particular reason, and something else comes along for the ride, what Stephen Jay Gould called a spandrel, borrowing the term from architecture.