The Spike: An Epic Journey Through the Brain in 2.1 Seconds

by Mark Humphries

Warren McCulloch made an untenable leap of faith in the early 1940s. The kind of creative daring that only his bizarre mélange of psychiatrist-neuroscientist-philosopher-theorist1 would attempt. The first fuzzy pictures of spikes appeared in the late 1920s and early 1930s. Tiny wobbles on an oscilloscope,2 showing electrical pulses so small they’d be vaporized by a cough in the next room. Yet McCulloch was struck by how each spike from the same neuron looked roughly the same shape, the same size, every time it appeared. With just a handful of neurons then recorded, he made a bold prediction: that every spike from every neuron in the entire brain is all-or-nothing; the spike either appears in its predetermined shape and size, or does not appear at all.

For you see, neurons sit in salty water—outside a neuron’s skin are lots of sodium (which has a positive charge: +) and lots of chlorine (which has a negative charge: −). But inside the neuron is a little sodium, a little chlorine, and lots of potassium (another +). Because the concentrations of each type of charged ion—particularly the potassium—are different either side of the neuron’s skin, this creates a voltage across the skin. By playing with the concentrations of ions outside the neuron, Hodgkin and Huxley were thus mucking about with the neuron’s voltage. And crucially they could find out which types of ions (sodium and friends) affected what part of sending a spike.

What they unpacked with their squid axon in a bath of saltwater was the remarkable birth of a spike (figure 2.2). When the neuron’s voltage increases beyond its tipping point, suddenly holes that only permit sodium open up in the neuron’s skin, and sodium ions rush in, rapidly increasing their concentration on the inside, and voltage rockets. But only briefly. For the onrush of sodium triggers the opening of a different set of holes in the skin, which pump potassium back to the outside, sending positive charge back out almost as quickly as it’s arriving via the sodium ions. In turn, this outrush of potassium shuts off the sodium holes, ions stop flowing, and just as quickly as it rose the voltage becomes negative again. This rapid leap then crash of voltage is the spike.

Together McCulloch and Pitts proved a deep theory that a group of neurons sending 1s or 0s to each other could produce all of logic. That, for example, a pair of neurons could compute AND: by both sending a spike—a 1—if both received an input, and neither sending a spike—a 0—for any other combination of their inputs. A different pair could compute OR: by each sending a spike (1) when that neuron received an input, but not sending a spike (0) if both neurons received no inputs or an input at the same time. Adding more and more neurons, McCulloch and Pitts showed, could compute all such statements of logic, no matter how convoluted.

John von Neumann laid out the architecture for modern electronic computer hardware in 1945.6 Von Neumann knew McCulloch well, and read McCulloch and Pitts’s paper; he then used the ideas of encoding 0s and 1s in elements of a circuit, and of how to combine these elements to do logic, in his architecture for a computer. Indeed, throughout his report laying out the EDVAC computer’s architecture, von Neumann talks of his computer as being modeled on how the brain works. Computer hardware has some foundations in brain science, not the other way around.

If so many neurons in the retina don’t need to use spikes, why does any neuron send spikes? Why convert the flexible, continuous, analogue signal of molecules and voltage into a rigid, discrete, binary one—why seemingly throw away useful information? The answer is that spikes let neurons send information accurately, fast, and far.

A spike is a time stamp that says “a thing happened just now.”

Some axons are custom built for speed. A spike can travel about 200 millimeters per second along an axon in the cortex, covering the distance from the back to the front of your cortex in less than a second.14 Sensory axons in the spine are a hundredfold faster still:15 the sciatic nerve in the shrew sends spikes at 42 meters per second; in the elephant, at 70 meters per second. Or 156 miles per hour. Elephants have Ferrari nerves.

Sending messages between distant neurons any other way is doomed to failure. Releasing molecules can send information over tiny gaps, as we saw in the retina, and we’ll see again in the next chapter. But the molecules newly released into the ocean of saltwater that surrounds neurons would get rapidly lost, as they diffuse away from where they were released; so releasing molecules is useless beyond a few micrometers. The neuron’s voltage alone decays rapidly with distance, so it would become indistinguishable from noise within 1 to 2 millimeters.

Our spike bursts into the first vision area of the cortex, V1, the first of the many areas dedicated to seeing that make up one-third of your entire cortex.1 Its message—about one small pixel of crumbly chocolate temptation—needs passing on upward through all these areas, combining with all other messages carried by the millions of other spikes to create the perception of “cookie.” First we have to make landfall. Your cortex is a delicately layered cake, six layers in all, five layers crammed with juicy neurons, the first, top layer bereft of them. We’re about to hit axon’s end in the fourth layer of this V1 area. Above us the packed neurons of layers two and three; above them layer one, a smattering of rare small neurons, but mostly filled with axons going elsewhere and brain cells that aren’t neurons, the glia cells, who are the scaffolding, the moppers-up, the below-stairs workforce. Below us, we can make out the larger, hulking neurons of layers five and six. Their bodies may be packed into layers, but the bulk of the neurons are not. Around us it’s a forest. A vast tree sprouts from the body of every neuron in sight. The tree’s branches are thin but divide, ramify, contort. They fill a volume that dwarfs the neuron’s body. This tree—the dendrite—is how a neuron collects its inputs from other neurons and funnels them all toward its body. Axon terminals from seemingly countless other neurons pepper the gnarled trees of neurons all about us.

The shape of the tree, and how many trees, tells us a lot about what that neuron is trying to do. Indeed, historically, it was often how we could tell neurons apart. Our trip from the retina is about to land us out in the compact, starburst tree of the first neuron in the cortex.2 Below us, the poster-child neuron of the cortex, the pyramidal cell of layer five with its two types of dendritic trees—one sticking out of the top, a single long slender stem stretching up almost to the cortical surface, the rest sticking out below the body, fat and squat (figure 3.1). Above us, in layers two and three, a more modest pyramidal cell, its tree compact and surrounding the body, less attention-seeking than its big brother in layer five. Whatever shape and size,3 all these dendrites are covered in inputs from other neurons.

Our cookie-pixel spike crashing into axon’s end rips open bags of glutamate molecules. Bags split, the glutamate molecules tumble out of the terminal, diffusing across the micrometer-sized gap and bumping up against glutamate-shaped receptors on the other side.

Nearby, farther down the tree, closer to the target neuron’s body, we can see axon ends that have not come from the retina. Rather, they have been sent from small, rare neurons nearby. And these send across the gap a different molecule, GABA. When GABA locks into the GABA-shaped receptors on the same tree, the voltage flickers downward, decreasing. Unsurprisingly, we call this “inhibition.”

The whole process seems a bit bonkers. Your brain went to all that effort to make a spike—a process that costs a lot of energy—to get around the fact that sending messages long distance can’t be done just by dumping molecules or spreading flickers of voltage. And then it turns the spike back into dumped molecules, which cause flickers of voltage. There are good reasons for doing this. For example, spreading voltage and chemistry are much cheaper in terms of energy—in tiny brains, everything is sent by spreading voltage and diffusing molecules, not spikes. But perhaps the key reason is flexibility. Transforming spikes back into chemistry then voltage gives the brain options for how to interpret the all-or-nothing spike.

Thinking about the type of inputs lets us narrow the numbers down a bit. Remember, the inputs at some gaps cause the neuron’s voltage to go down, not up. They inhibit the neuron, making it less likely to birth a spike. So we are really asking about how many of the inputs that excite the neuron we need to make a spike. Braitenberg and Schuz painstakingly counted those too—dedicated, admirable scientists, but ones who’d monologue at you for three hours on the best way to slice a mouse brain into wafer-thin bits and count its synapses with nary a pause to sip the steadily warming beer on the table leaving you trapped, beer-less, in an etiquette nightmare. They counted: about 90 percent of the inputs to a cortical neuron excite it; only about 10 percent inhibit it. We’ve brought our upper limit of the number of spikes down a bit to 6,750. Woohoo. I did say “a bit.”

Because in truth “how many spikes?” is a deep, hard question whose answer depends on myriad factors. And these myriad factors tell us a lot about how the brain uses spikes to make things happen. Three stand out: the balance of excitation and inhibition arriving at a neuron, the synchrony of the inputs, and where they land on the tree itself.

As theorists, they instantly realized something was amiss. Our best models for how neurons make spikes don’t have randomly different intervals between those spikes. No matter how irregularly spaced the spikes these models receive, the spikes they make are evenly spaced, the intervals between them far more regular than Softky and Koch saw in the cortex. To grasp why, think about the total number of spikes arriving at a neuron. Even though each of the individual inputs has highly irregular spikes, there are thousands of such inputs. So when we sum over them to get the total number of spikes arriving, then we find the sum is pretty constant in time. So if a model neuron needs, say, 175 input spikes to make one new spike, then a total of 175 input spikes turns up regular as clockwork, thus making a new spike regular as clockwork (figure 3.3).

According to our best models, spikes coming in irregularly would be made into an output of regular, well-behaved, evenly spaced spikes. But this creates a paradox: if neurons make regular-spaced spikes, where then do the random, irregularly spaced spikes of cortex come from in the first place? Theorists love paradoxes. Paradoxes in science show us where there is a gap in our understanding, and hold out the promise that solving the paradox will create a new view of how the world works. Fittingly, the irregular-spike paradox invoked a pile-on of theorists, and a raft of proposals for what could be making irregular spikes.11

The math is fierce, but the idea is simple. We’ve got our pretend neurons, most excitatory and the rest inhibitory, and we’ve randomly wired them together. Then all we need to do is guarantee that the input to each neuron is more than the neuron needs to make a spike. For this then creates a web of negative feedback loops, of neurons trying to make a spike, but being held back. It works like this. Say some of the excitatory neurons send a lot of spikes. This will drive inhibitory neurons to make spikes—that are fed back to those excitatory neurons and turn down their spikes. But they can’t turn all the excitatory neurons down too much, because then the inhibitory neurons themselves will lose their input and stop firing. But they have to be firing, because the excitatory neurons are firing. This paradox implies there is a self-consistent state for the whole network, where the inhibitory and excitatory neurons are sending just the right amount of spikes each, so the push-pull of the excitation and inhibition in the network is balanced. And we all know by now what balanced excitatory and inhibitory inputs means: irregular spikes. Better still, this theory showed balanced networks are robust old things: you don’t need fine-tuning of the exact strengths of inhibitory and excitatory inputs; nor do you need to fiddle with the details of how the neurons make spikes. Just make the total input to each neuron a lot, add a dash of feedback, and voilà: balance.

Bunched-together inputs come to the rescue here. Big neurons in the cortex, like the layer five pyramidal neurons just below us, rudely jutting their trees past our simple neuron in layer four right up to the ceiling of cortex in layer one, those ignoramuses, have a trick up their sleeve. They add up wrong.27

This supralinear sum is a sudden jump of voltage in the branch where the inputs land. Enough inputs arriving together opens up new holes in the neuron’s skin, allowing extra ions to flow into the neuron, driving up the voltage in that bit of branch. And if you’re thinking that sounds like a spike, you’d not be far off. While not a pretty, peaky thing, this sudden jump in voltage in the tree has the same job: get information from the far reaches of the tree down to the neuron’s body intact. So if a bunch of inputs turn up at same time, they evoke this superblip that rushes down the tree and makes a major contribution to the neuron reaching its tipping point.

These location, location, location dependencies of a single neuron have deep implications for artificial intelligence (AI). AI-brand neural networks are all constructed from the same kind of pretend neuron, a simple thing that just adds up its inputs from other pretend neurons. And once added up, an AI-brand pretend neuron checks if they sum greater than zero, and if so then sends that sum on to all its targets (or else sends zero). The deepest of deep networks are all constructed from millions of these elementary things. But I’ve just spent five thousand and more words telling you that a single neuron in your cortex does not just add up its inputs. How a neuron deals with its input depends on a plethora of things that all interact: the balance and synchrony of its inputs, where they land and how they bunch, whether they sum up wrong, what the voltage is when the inputs land, and what is on the path between the input and the neuron’s body. Current AI networks have barely tapped the surface of what brains do. Indeed, detailed pretend models of neurons have shown us that a single neuron can itself be a two-layer neural network.30 If each branch of the tree has the ability to do those supralinear sums, each branch acts as single AI-brand pretend unit. So their outputs (layer one of the network) converge at the neuron’s body (layer two of the network). And with this comes the ability for single neurons to compute alone many of the functions of logic.31 So each individual neuro...

Belying their name, simple cells are an eclectic bunch. For one thing, they keep the orderly map of the visual world that came from the retina, so nearby simple cells respond to similar positions in the world. For another, tens of channels of information from the retina have slammed into the simple cells that surround us. Thirty-plus channels for every location in visual space, for the middle, the left, the right, up and down, everywhere. So collections of simple cells bunched together are interested in different things about the same location in the world: some in edges at 90 degrees, some at 120 degrees, some at 41.3 degrees; some in edges created by a patch of light above and dark below, some reversed. And all combinations in between.

There is one more type of neuron nearby, a type of neuron that sits in the very first bit of visual cortex but does not particularly care for the view. Everything we’ve landed on so far is an excitatory neuron, a stellate cell with its starburst tree or a pyramidal cell with its above-and-below tree. But as clones of our spike travel along some branches of the axon, they leap gaps onto rare GABA-toting neurons, neurons that get nothing from the retina, and only branch their axons inside the region of cortex their bodies sit in. Hence we call them interneurons. They are the wellsprings of the deadly inhibition we ran into when racing down the tree of our first neuron. They are there to sort the wheat from the chaff.

FAR: HIGHWAY WHAT AND HIGHWAY DO

In tracing this feed-forward circuit, from layer four up to layers two and three, and back down to layers five and six, we’ve run into all three types of pyramidal neurons in cortex.9 All three types use glutamate as their molecules, so all excite neurons they connect to; all connect to neurons of the same type within their own layer, but they are separated by where they send the long branch of their axon. In layer five we watch some of the cloned spikes run into pyramidal tract neurons that send a long branch of their axon all the way through the brain, down to the brain stem, and some onward to the spinal cord. In layer six other cloned spikes jump gaps onto cortico-thalamic neurons, their long branch dropping down out of the cortex into the tiny nugget of midbrain called the thalamus, whose own neurons send their axons all over the cortex, creating complex feedback loops. And in all layers (except layer one) are the cortico-cortical neurons, whose long branch of axon connects regions within the cortex, carrying spikes far across the cortex on this left side of the brain, and on to the cortex on the other side of the brain. Neurons just like the one we’ve been dispatched from.

And just down the temporal lobe from where we’ve landed are neurons that deal with the shapes humans ultimately care most about: faces. And which you also care about deeply right now, to answer the crucial question: are any faces looking at me while I contemplate this cookie? From V2, via V1, the simple but unique conjoined edges of a nose, a brow, a chin, the line of a mouth, the curve of the cheek bones. Combine those with the colors from V4 of the watery pink lips within graying stubble against pale skin, and you get: a Graham. Facing side-on, across the far side of the office, eyes pointed ceiling-ward, idly contemplating existence.

Area MT neurons put the whole picture together; they respond to global motion, across the whole field of view. Some MT neurons respond to a coherent collection of edges and surfaces moving from left to right. Some to such a collection moving from bottom to top. This sensitivity of an MT neuron to global motion in a particular direction likely comes from their integrating the spikes about local directions pouring into area MT from V1 (and V2) neurons.19

Artificial neural networks are not adept at generalizing. An AI researcher may train one of their neural networks on tens of thousands of images so that it learns to classify them: “cars,” “gorillas,” “ice cream vans on fire (irony).” But deep neural networks have many layers of thousands of simple neuron-like units. So they have millions, tens of millions, of connections between those units, and the strength of every connection can be adjusted. Having far more connections to adjust than images to learn means artificial networks are prone to horrible overfitting;23 they learn the fine detail of each image, become fine-tuned to the nuances.

A widely used solution is DropConnect.24 Which does exactly what it says on the box: for every new image presented during training, a bunch of the connections in the network are dropped at random. And only the retained connections are updated by the success or failure in categorizing that image. Repeated for each image, this essentially means that every image is presented to a unique version of the network, stopping the whole network being fine-tuned to the details of each image. And when this network is then tested on unseen images, it does a better job of categorizing them correctly. Dropping connections at random adds noise to the network, noise that lets the network generalize. Your brain faces the same challenges—worse even. Your cortex has billions of connections it could adjust every time you learn something. So how does it not overfit? I suggest synaptic failure. Synaptic failure is exactly the same mechanism as DropConnect: it drops connections between neurons, at random, and temporarily. It adds deliberate noise to the brain exactly where you’d want it to prevent your brain from overfitting. To let it generalize.

The second good reason to add deliberate noise to your brain is to help it search better. Many challenges in machine learning are about finding the optimal solution to a problem given some constraints. Like finding the fastest route between two locations—where fastest often does not mean shortest, and the solutions are constrained by speed limits, traffic, your mode of transport, time of day, likelihood of rampaging sheep escaping a field, and innumerable other factors. A machine solving these problems will explore the space of possible solutions. It will propose a solution, evaluate how good it is, and work out how to adjust that solution to find a better one.25 Told you want to go from London to Paris, it will propose one possible route, work out how long it will take, then look at how it can be adjusted—taking a left turn here to get on to a longer road, but one that has a higher speed limit. And repeat this propose-and-adjust cycle until its proposed solution cannot be bettered. The fundamental challenge for searching is that such problems will have many solutions that are adequate, passable, OK, but which are all surrounded by worse solutions, so that any small adjustments from the adequate solution gets worse results. These are little traps in the space of possible solutions. They make these solutions look like they cannot be bettered—but they can, and there may be vastly better solutions out there, somewhere. The many ways of escaping these traps boil down to the sa...

To me, this suggests the tantalizing idea that synaptic failure is at the heart of the brain’s search algorithm.26 Brains need to seek solutions to many constrained problems.

Neuroimaging—functional MRI—shows us Technicolor images of the cortex, its regions lit up in a swirling riot of poorly chosen colors that make the Pantone people cry into their tasteful coffee mugs. The swirling colors seem to show us that the cortex is a swarm of activity. That when we see a face the visual areas of our cortex bloom with barrages of neural firing, from V1, V4, and down to the face areas of the temporal lobe. That when we hear a swell of strings, the auditory areas of our cortex bloom with barrages of spikes.

But then neuron imaging came along. We point a digital video camera at a bit of brain, and in that bit of brain each neuron contains a chemical we’ve injected that lights up when the neuron is active. Most often, that fluorescent chemical is responding to the amount of calcium in the neuron’s body, lighting up to an influx of calcium with every spike.2 By filming the bit of brain in sharp focus, we can see all the neurons with our eyes, see their outlines. And we can see which ones light up. It turns out for decades we’ve only been recording the tip of the iceberg. Most neurons we can see in these videos are not active.

Largely, it does not. Tomáš Hromádka in Tony Zador’s lab at Cold Spring Harbor patched a collection of neurons in the first bit of auditory cortex (A1) in awake rats and found most of them were silent most of the time.6 And silent regardless of whether the animal was sitting quietly or listening to an extremely dull collection of pure tones. Playing sounds to the bit of the cortex that cares most about sounds evoked very little response. Dan O’Connor, then in Karel Svoboda’s lab at Janelia Farm, patched a collection of neurons in that specialized whisker bit of cortex in mice, mice that were again using a whisker to find a vertical pole.7 Guess what? Most of those neurons were silent most of the time. Even when the whisker was waving back and forth, hitting the pole. And both these and other studies have consistently found silent neurons in all layers of the cortex.8

We traveled along Highway What, and by cloning ourselves traveled along Highway Do at the same time. Highway What has disgorged us into the start of the enigmatic prefrontal region of the cortex, the front third of your cortex, roughly everything forward of your ears. Highway Do left us in part of the parietal regions of cortex, a large strip above and behind your ears (figure 7.5). It becomes harder now to know precisely what any particular small bit of these regions of cortex do. And their roles are intertwined. Neurons in both prefrontal and parietal regions send axons to the other, so we could hop a spike and jump between them if we so wished.

Indeed, the prefrontal and parietal cortices are the brain’s Louvre: vastly too much to take in on one visit, and no matter how niche your tastes you can find something to suit. The prelimbic region, to pick one of many, is like the ceramics gallery: if that’s your bag, great—you could spend all day in here marveling at how its neurons send spikes after a mistake is made, seemingly in order to ensure you take more time before making the same decision again;19 if it’s not your bag, glance at the serried ranks of plates as you hurry to the exit.

These statistics of the visible world are all learned by experience. Raise someone in a world with no vertical lines, and they will not be able to see a vertical object placed before them.10 Raise someone with one eye closed, and when reopened that eye will see nothing.11 In both cases, the neurons in visual cortex have not been able to learn the statistics of the world—deprived of the experience of vertical lines, there are no neurons tuned to vertical lines; deprived of the experience of one eye, there are no neurons tuned to the view from that eye. Neurons learn about edges, corners, and curves through experience. The set of neurons you have in your visual cortex exactly reflects the statistics of the visible world you grew up in.

These predictive spontaneous spikes solve the speed limit problem. We don’t need to get spikes from the retina to alone create the first wave of spikes in V1, to in turn recruit those in V2, then V4 and on and on. Because the spontaneous spikes are already there in each of those regions, are already predicting most of visible world.

And the input from the eye simply adjusts what is wrong with the predictions. As most of the predictions will be correct, because our visual system spent many years learning what is there in the world, there won’t be much to adjust. So most of the spontaneous spikes in your visual cortices will be telling the rest of your brain an accurate version of what is there in the world, before your eye has even “seen” it. That’s the theory, anyway.12 Bayesian hierarchical inference, to the cognoscenti; bootstrapping, if you prefer; educated guessing to the rest of us. Whatever we call it, it means seeing can be so fast you don’t notice it happening; the information is already there and adjusted by the input.

Holding things in your memory buffer is also prediction. There is a strictly limited capacity for what we can keep in that working memory. So for something happening in the world to gain entry to this buffer means it must be likely worth remembering. Placing an event in that buffer is then a prediction that it will be useful to know in the immediate future.