More on this book
Community
Kindle Notes & Highlights
I call this phenomenon the law of accelerating returns (LOAR), and it pertains to both biological and technological evolution. The most dramatic example of the LOAR is the remarkably predictable exponential growth in the capacity and price/performance of information technologies. The evolutionary process of technology led invariably to the computer, which has in turn enabled a vast expansion of our knowledge base, permitting extensive links from one area of knowledge to another.
In The Singularity Is Near I made the case that one corollary of the law of accelerating returns is that other intelligent species are likely not to exist. To summarize the argument, if they existed we would have noticed them, given the relatively brief time that elapses between a civilization’s possessing crude technology (consider that in 1850 the fastest way to send nationwide information was the Pony Express) to its possessing technology that can transcend its own planet.4 From this perspective, reverse-engineering the human brain may be regarded as the most important project in the
...more
In this book I present a thesis I call the pattern recognition theory of mind (PRTM),
IBM is now working with Nuance Speech Technologies (formerly Kurzweil Computer Products, my first company) on a new version of Watson that will read medical literature (essentially all medical journals and leading medical blogs) to become a master diagnostician and medical consultant, using Nuance’s clinical language–understanding technologies.
If understanding language and other phenomena through statistical analysis does not count as true understanding, then humans have no understanding either.
The cerebral cortex, which accounts for 80 percent of the human brain, is composed of a highly repetitive structure, allowing humans to create arbitrarily complex structures of ideas.
British geologist Charles Lyell (1797–1875), however, proposed that it was indeed the movement of water that had carved out these major geological modifications over great periods of time, essentially one grain of rock at a time.
But undoubtedly this was the general belief when the first edition of the present work appeared. . . . Now things are wholly changed, and almost every naturalist admits the great principle of evolution.”
In 1869, only a decade after the original publication of On the Origin of Species, Swiss physician Friedrich Miescher (1844–1895) discovered a substance he called “nuclein” in the cell nucleus, which turned out to be DNA.
In 1927 Russian biologist Nikolai Koltsov (1872–1940) described what he called a “giant hereditary molecule,” which he said was composed of “two mirror strands that would replicate in a semi-conservative fashion using each strand as a template.” His finding was also condemned by many. The communists considered it to be fascist propaganda, and his sudden, unexpected death has been attributed to the secret police of the Soviet Union.
Or does it? Einstein’s conclusion was that since the photon clearly has energy, and has momentum, it must also have a mass equivalent. The energy of the moving photon is entirely equivalent to a moving mass. We can compute what that equivalence is by recognizing that the center of mass of the system must remain stationary during the movement of the photon.
Working out the math, Einstein showed that mass and energy are equivalent and are related by a simple constant. However, there was a catch: The constant might be simple, but it turned out to be enormous; it was the speed of light squared (about 1.7 × 1017 meters2 per second2—that is, 17 followed by 16 zeroes). Hence we get Einstein’s famous E = mc2.10 Thus one ounce (28 grams) of mass is equivalent to 600,000 tons of TNT.
This suggests that our memories are sequential and in order. They can be accessed in the order that they are remembered. We are unable to directly reverse the sequence of a memory.
We can recognize a pattern even if only part of it is perceived (seen, heard, felt) and even if it contains alterations. Our recognition ability is apparently able to detect invariant features of a pattern—characteristics that survive real-world variations.
(This turns out to be true of intellectual perspectives as well.)
First, let me explain why this section specifically discusses the neocortex (from the Latin meaning “new rind”). We do know the neocortex is responsible for our ability to deal with patterns of information and to do so in a hierarchical fashion. Animals without a neocortex (basically nonmammals) are largely incapable of understanding hierarchies.
This thin structure is basically made up of six layers, numbered I (the outermost layer) to VI. The axons emerging from the neurons in layers II and III project to other parts of the neocortex. The axons (output connections) from layers V and VI are connected primarily outside of the neocortex to the thalamus, brain stem, and spinal cord.
The neurons in layer IV receive synaptic (input) connections from neurons that are outside the neocortex, especially in the thalamus.
Vernon Mountcastle
In 1978 he made an observation that is as significant to neuroscience as the Michelson-Morley ether-disproving experiment of 1887 were to physics. That year he described the remarkably unvarying organization of the neocortex, hypothesizing that it was composed of a single mechanism that was repeated over and over again,
American scientist Herbert A. Simon (1916–2001), who is credited with cofounding the field of artificial intelligence, wrote eloquently about the issue of understanding complex systems at the right level of abstraction.
There are about a half million cortical columns in a human neocortex, each occupying a space about two millimeters high and a half millimeter wide and containing about 60,000 neurons (resulting in a total of about 30 billion neurons in the neocortex). A rough estimate is that each pattern recognizer within a cortical column contains about 100 neurons, so there are on the order of 300 million pattern recognizers in total in the neocortex.
Kasparov had learned about 100,000 board positions. That’s a real number—we have established that a human master in a particular field has mastered about 100,000 chunks of knowledge.
The innate ability of humans to learn the hierarchical structures in language that Noam Chomsky wrote about reflects the structure of the neocortex. In a 2002 paper he coauthored, Chomsky cites the attribute of “recursion” as accounting for the unique language faculty of the human species.4 Recursion, according to Chomsky, is the ability to put together small parts into a larger chunk, and then use that chunk as a part in yet another structure, and to continue this process iteratively. In this way we are able to build the elaborate structures of sentences and paragraphs from a limited set of
...more
The ability to recognize patterns even when aspects of them are transformed is called feature invariance, and is dealt with in four ways.
There is a mathematical solution to this optimization problem called linear programming, which solves for the best possible allocation of limited resources (in this case, a limited number of pattern recognizers) that would represent all of the cases on which the system has trained.
Cultural rules are enforced in the neocortex with help from the old brain, especially the amygdala. Every thought we have triggers other thoughts, and some of them will relate to associated dangers. We learn, for example, that breaking a cultural norm even in our private thoughts can lead to ostracism, which the neocortex realizes threatens our well-being. If we entertain such thoughts, the amygdala is triggered, and that generates fear, which generally leads to terminating that thought.
Relaxing professional taboos turns out to be useful for creative problem solving. I use a mental technique each night in which I think about a particular problem before I go to sleep. This triggers sequences of thoughts that will continue into my dreams. Once I am dreaming, I can think—dream—about solutions to the problem without the burden of the professional restraints I carry during the day.
When we used the technique of hierarchical hidden Markov models in order to incorporate the distribution of magnitudes of each input, performance soared.
The cataclysmic Cretaceous-Paleogene extinction event about 65 million years ago led to the rapid demise of many non-neocortex-bearing species that could not adapt quickly enough to a suddenly altered environment. This marked the turning point for neocortex-capable mammals to take over their ecological niche. In this way, biological evolution found that the hierarchical learning of the neocortex was so valuable that this region of the brain continued to grow in size until it virtually took over the brain of Homo sapiens
The central assumption in Hebb’s theory is that the basic unit of learning in the neocortex is the neuron. The pattern recognition theory of mind that I articulate in this book is based on a different fundamental unit: not the neuron itself, but rather an assembly of neurons, which I estimate to number around a hundred.
The “Lego blocks” that Markram proposes are fully consistent with the pattern recognition modules that I have described. In an e-mail communication, Markram described these “Lego blocks” as “shared content and innate knowledge.”
Note that Markram’s estimate of each module’s containing “several dozen neurons” is based only on layer V of the neocortex. Layer V is indeed neuron rich, but based on the usual ratio of neuron counts in the six layers, this would translate to an order of magnitude of about 100 neurons per module, which is consistent with my estimates.
The most powerful argument for the universality of processing in the neocortex is the pervasive evidence of plasticity (not just learning but interchangeability): In other words, one region is able to do the work of other regions, implying a common algorithm across the entire neocortex.
Consider the implications of this study: It means that neocortical regions that are physically relatively far apart, and that have also been considered conceptually very different (primitive visual cues versus abstract language concepts), use essentially the same algorithm. The regions that process these disparate types of patterns can substitute for one another.
There are on the order of a quadrillion (1015) connections in the neocortex, yet only about 25 million bytes of design information in the genome (aft er lossless compression),16 so the connections themselves cannot possibly be predetermined genetically. It is possible that some of this learning is the product of the neocortex’s interrogating the old brain, but that still would necessarily represent only a relatively small amount of information. The connections between modules are created on the whole from experience (nurture rather than nature).
Finally, there is one more piece of corroborating evidence. The techniques that we have evolved over the past several decades in the field of artificial intelligence to recognize and intelligently process real-world phenomena (such as human speech and written language) and to understand natural-language documents turn out to be mathematically similar to the model I have presented above. They are also examples of the PRTM. The AI field was not explicitly trying to copy the brain, but it nonetheless arrived at essentially equivalent techniques.
We are apparently able to keep up to about four items in our working memory at a time, two per hemisphere according to recent research by neuroscientists at the MIT Picower Institute for Learning and Memory.6 The issue of whether the thalamus is in charge of the neocortex or vice versa is far from clear, but we are unable to function without both.
The hippocampus is one of the first regions damaged by Alzheimer’s, so one goal of this research is to develop a neural implant for humans that will mitigate this first phase of damage from the disease.
Fear is the main source of superstition, and one of the main sources of cruelty. To conquer fear is the beginning of wisdom.
One region that is associated with pleasure is the nucleus accumbens.
Once the amygdala does decide that danger is ahead, an ancient sequence of events occurs. The amygdala signals the pituitary gland to release a hormone called ACTH (adrenocorticotropin). This in turn triggers the stress hormone cortisol from the adrenal glands, which results in more energy being provided to your muscles and nervous system.
The system of global neurotransmitter levels, such as serotonin, and hormone levels, such as dopamine, is intricate, and we could spend the rest of this book on the issue (as a great many books have done), but it is worth pointing out that the bandwidth of information (the rate of information processing) in this system is very low compared with the bandwidth of the neocortex. There are only a limited number of substances involved and the levels of these chemicals tend to change slowly and are relatively universal across the brain, as compared with the neocortex, which is composed of hundreds
...more
Our emotional thoughts also take place in the neocortex but are influenced by portions of the brain ranging from ancient brain regions such as the amygdala to some evolutionarily recent brain structures such as the spindle neurons, which appear to play a key role in higher-level emotions.
As mentioned earlier, the insula helps process sensory signals, but it also plays a key role in higher-level emotions. It is this region from which the spindle cells originate. Functional magnetic resonance imaging (fMRI) scans have revealed that these cells are particularly active when a person is dealing with emotions such as love, anger, sadness, and sexual desire. Situations that strongly activate them include when a subject looks at her partner or hears her child crying.
There are relatively few spindle cells: only about 80,000, with approximately 45,000 in the right hemisphere and 35,000 in the left. This disparity is at least one reason for the perception that emotional intelligence is the province of the right brain, although the disproportion is modest. Gorillas have about 16,000 of these cells, bonobos about 2,100, and chimpanzees about 1,800. Other mammals lack them completely.
Anthropologists believe that spindle cells made their first appearance 10 to 15 million years ago in the as yet undiscovered common ancestor to apes and hominids (precursors to humans) and rapidly increased in numbers around 100,000 years ago. Interestingly, spindle cells do not exist in newborn humans but begin to appear only at around the age of four months and increase significantly in number from ages one to three. Children’s ability to deal with moral issues and perceive such higher-level emotions as love develop during this same period.
As American mathematician Norbert Wiener (1894–1964) wrote in his seminal book Cybernetics, published the year I was born (1948): There are fields of scientific work, as we shall see in the body of this book, which have been explored from the different sides of pure mathematics, statistics, electrical engineering, and neurophysiology; in which every single notion receives a separate name from each group, and in which important work has been triplicated or quadruplicated, while still other important work is delayed by the unavailability in one field of results that may have already become
...more
A technique I have used in my own work to combat increasing specialization is to assemble the experts that I have gathered for a project (for example, my speech recognition work included speech scientists, linguists, psychoacousticians, and pattern recognition experts, not to mention computer scientists) and encourage each one to teach the group his particular techniques and terminology. We then throw out all of that terminology and make up our own. Invariably we find metaphors from one field that solve problems in another.
One approach to expand the available neocortex is through the collaboration of multiple humans. This is accomplished routinely via the communication between people gathered in a problem-solving community. Recently there have been efforts to use online collaboration tools to harness the power of real-time collaboration, which have shown success in mathematics and other fields.
