The Singularity is Near: When Humans Transcend Biology
Rate it:
Open Preview
48%
Flag icon
Yes, we have a soul. But it’s made of lots of tiny robots. —GIULIO GIORELLI
48%
Flag icon
Substrate is morally irrelevant, assuming it doesn’t affect functionality or consciousness. It doesn’t matter, from a moral point of view, whether somebody runs on silicon or biological neurons (just as it doesn’t matter whether you have dark or pale skin). On the same grounds, that we reject racism and speciesism, we should also reject carbon-chauvinism, or bioism. —NICK BOSTROM, “ETHICS FOR INTELLIGENT MACHINES: A PROPOSAL, 2001”
48%
Flag icon
A Singularitarian is someone who understands the Singularity and has reflected on its meaning for his or her own life.
48%
Flag icon
But I did not come to my perspective as a result of searching for an alternative to customary faith. The origin of my quest to understand technology trends was practical: an attempt to time my inventions and to make optimal tactical decisions in launching technology enterprises. Over time this modeling of technology took on a life of its own and led me to formulate a theory of technology evolution. It was not a huge leap from there to reflect on the impact of these crucial changes on social and cultural institutions and on my own life. So, while being a Singularitarian is not a matter of faith ...more
48%
Flag icon
Contemporary philosopher Max More describes the goal of humanity as a transcendence to be “achieved through science and technology steered by human values.”5 More cites Nietzsche’s observation “Man is a rope, fastened between animal and overman—a rope over an abyss.” We can interpret Nietzsche to be pointing out that we have advanced beyond other animals while seeking to become something far greater.
48%
Flag icon
BILL GATES: I agree with you 99 percent. What I like about your ideas is that they are grounded in science, but your optimism is almost a religious faith. I’m optimistic also. RAY: Yes, well, we need a new religion. A principal role of religion has been to rationalize death, since up until just now there was little else constructive we could do about it. BILL: What would the principles of the new religion be? RAY: We’d want to keep two principles: one from traditional religion and one from secular arts and sciences—from traditional religion, the respect for human consciousness. BILL: Ah yes, ...more
49%
Flag icon
RAY: Not yet, but there will be. Once we saturate the matter and energy in the universe with intelligence, it will “wake up,” be conscious, and sublimely intelligent. That’s about as close to God as I can imagine. BILL: That’s going to be silicon intelligence, not biological intelligence. RAY: Well, yes, we’re going to transcend biological intelligence. We’ll merge with it first, but ultimately the nonbiological portion of our intelligence will predominate. By the way, it’s not likely to be silicon, but something like carbon nanotubes.
49%
Flag icon
A second scenario is that we could upload the patterns of an actual human into a suitable nonbiological, thinking substrate. A third, and the most compelling, scenario involves the gradual but inexorable progression of humans themselves from biological to nonbiological. That has already started with the benign introduction of devices such as neural implants to ameliorate disabilities and disease. It will progress with the introduction of nanobots in the bloodstream, which will be developed initially for medical and antiaging applications. Later more sophisticated nanobots will interface with ...more
49%
Flag icon
In fact these future machines will be even more humanlike than humans today. If that seems like a paradoxical statement, consider that much of human thought today is petty and derivative. We marvel at Einstein’s ability to conjure up the theory of general relativity from a thought experiment or Beethoven’s ability to imagine symphonies that he could never hear. But these instances of human thought at its best are rare and fleeting. (Fortunately we have a record of these fleeting moments, reflecting a key capability that has separated humans from other animals.) Our future primarily ...more
49%
Flag icon
My point is that we cannot safely dismiss the question of consciousness as merely a polite philosophical concern. It is at the core of society’s legal and moral foundation. The debate will change when a machine—nonbiological intelligence—can persuasively argue on its own that it/he/she has feelings that need to be respected. Once it can do so with a sense of humor—which is particularly important for convincing others of one’s humanness—it is likely that the debate will be won. I expect that actual change in our legal system will come initially from litigation rather than legislation, as ...more
50%
Flag icon
On yet another hand (we’re running out of philosophical hands here), as I pointed out at the beginning of this question, I am in fact being continually replaced as part of a normal biological process. (And, by the way, that process is not particularly gradual but rather rapid.) As we concluded, all that persists is my spatial and temporal pattern of matter and energy. But the thought experiment above shows that gradual replacement means the end of me even if my pattern is preserved. So am I constantly being replaced by someone else who just seems a lot like the me of a few moments earlier?
50%
Flag icon
I do believe that we humans will come to accept that nonbiological entities are conscious, because ultimately the nonbiological entities will have all the subtle cues that humans currently possess and that we associate with emotional and other subjective experiences. Still, while we will be able to verify the subtle cues, we will have no direct access to the implied consciousness.
50%
Flag icon
RAY: Biological humans have about ten trillion cells with their own DNA, but there are about one hundred trillion microorganisms in the digestive tract, basically bacteria. MOLLY 2004: Doesn’t sound very appealing. Are they entirely necessary? RAY: They’re actually part of the society of cells that makes Molly alive and thriving. You couldn’t survive without healthy gut bacteria. Assuming your intestinal flora are in good balance, they’re necessary for your well-being.
50%
Flag icon
Some philosophers hold that philosophy is what you do to a problem until it’s clear enough to solve it by doing science. Others hold that if a philosophical problem succumbs to empirical methods, that shows it wasn’t really philosophical to begin with. —JERRY A. FODOR17
51%
Flag icon
In the late 1960s I was transformed into a radical environmental activist. A rag-tag group of activists and I sailed a leaky old halibut boat across the North Pacific to block the last hydrogen bomb tests under President Nixon. In the process I co-founded Greenpeace…. Environmentalists were often able to produce arguments that sounded reasonable, while doing good deeds like saving whales and making the air and water cleaner. But now the chickens have come home to roost. The environmentalists’ campaign against biotechnology in general, and genetic engineering in particular, has clearly exposed ...more
52%
Flag icon
As another example, Joy advocates not publishing the gene sequences of pathogens on the Internet, which I also agree with. He would like to see scientists adopt regulations along these lines voluntarily and internationally, and he points out that “if we wait until after a catastrophe, we may end up with more severe and damaging regulations.” He says he hopes that “we will do such regulation lightly, so that we can get most of the benefits.”
52%
Flag icon
It’s like arguing in favor of the plough. You know some people are going to argue against it, but you also know it’s going to exist. —JAMES HUGHES, SECRETARY OF THE TRANSHUMANIST ASSOCIATION AND SOCIOLOGIST AT TRINITY COLLEGE, IN A DEBATE, “SHOULD HUMANS WELCOME OR RESIST BECOMING POSTHUMAN?”
52%
Flag icon
We will spend increasing portions of our time in virtual environments and will be able to have any type of desired experience with anyone, real or simulated, in virtual reality. Nanotechnology will bring a similar ability to morph the physical world to our needs and desires. Lingering problems from our waning industrial age will be overcome. We will be able to reverse remaining environmental destruction. Nanoengineered fuel cells and solar cells will provide clean energy. Nanobots in our physical bodies will destroy pathogens, remove debris such as misformed proteins and protofibrils, repair ...more
52%
Flag icon
“Plants” with “leaves” no more efficient than today’s solar cells could out-compete real plants, crowding the biosphere with an inedible foliage. Tough omnivorous “bacteria” could out-compete real bacteria: They could spread like blowing pollen, replicated swiftly, and reduce the biosphere to dust in a matter of days. Dangerous replicators could easily be too tough, small, and rapidly spreading to stop—at least if we make no preparation. We have trouble enough controlling viruses and fruit flies. —ERIC DREXLER
52%
Flag icon
Living creatures—including humans—would be the primary victims of an exponentially spreading nanobot attack. The principal designs for nanobot construction use carbon as a primary building block. Because of carbon’s unique ability to form four-way bonds, it is an ideal building block for molecular assemblies. Carbon molecules can form straight chains, zigzags, rings, nanotubes (hexagonal arrays formed in tubes), sheets, buckyballs (arrays of hexagons and pentagons formed into spheres), and a variety of other shapes. Because biology has made the same use of carbon, pathological nanobots would ...more
52%
Flag icon
Based on this observation we can envision a more insidious possibility. In a two-phased attack, the nanobots take several weeks to spread throughout the biomass but use up an insignificant portion of the carbon atoms, say one out of every thousand trillion (1015). At this extremely low level of concentration the nanobots would be as stealthy as possible. Then, at an “optimal” point, the second phase would begin with the seed nanobots expanding rapidly in place to destroy the biomass. For each seed nanobot to multiply itself a thousand trillionfold would require only about fifty binary ...more
52%
Flag icon
If a little knowledge is dangerous, where is a person who has so much as to be out of danger? —THOMAS HENRY
53%
Flag icon
The Precautionary Principle. As Bostrom, Freitas, and other observers including myself have pointed out, we cannot rely on trial-and-error approaches to deal with existential risks. There are competing interpretations of what has become known as the “precautionary principle.” (If the consequences of an action are unknown but judged by some scientists to have even a small risk of being profoundly negative, it’s better to not carry out the action than risk negative consequences.) But it’s clear that we need to achieve the highest possible level of confidence in our strategies to combat such ...more
53%
Flag icon
My own assessment of this danger is that we are unlikely simply to stumble across such a destructive event. Consider how unlikely it would be to accidentally produce an atomic bomb. Such a device requires a precise configuration of materials and actions, and the original required an extensive and precise engineering project to develop. Inadvertently creating a hydrogen bomb would be even less plausible. One would have to create the precise conditions of an atomic bomb in a particular arrangement with a hydrogen core and other elements. Stumbling across the exact conditions to create a new ...more
53%
Flag icon
Insights from the brain reverse-engineering effort, overall research in developing AI algorithms, and ongoing exponential gains in computing platforms make strong AI (AI at human levels and beyond) inevitable. Once AI achieves human levels, it will necessarily soar past it because it will combine the strengths of human intelligence with the speed, memory capacity, and knowledge sharing that nonbiological intelligence already exhibits. Unlike biological intelligence, nonbiological intelligence will also benefit from ongoing exponential gains in scale, capacity, and price-performance.
53%
Flag icon
Totalitarian Relinquishment. The only conceivable way that the accelerating pace of advancement on all of these fronts could be stopped would be through a worldwide totalitarian system that relinquishes the very idea of progress. Even this specter would be likely to fail in averting the dangers of GNR because the resulting underground activity would tend to favor the more destructive applications. This is because the responsible practitioners that we rely on to quickly develop defensive technologies would not have easy access to the needed tools. Fortunately, such a totalitarian outcome is ...more
54%
Flag icon
Should we tell the millions of people afflicted with cancer and other devastating conditions that we are canceling the development of all bioengineered treatments because there is a risk that these same technologies may someday be used for malevolent purposes? Having posed this rhetorical question, I realize that there is a movement to do exactly that, but most people would agree that such broad-based relinquishment is not the answer.
54%
Flag icon
Relinquishing technological advancement would be economic suicide for individuals, companies, and nations.
54%
Flag icon
A further reason why industrial society cannot be reformed … is that modern technology is a unified system in which all parts are dependent on one another. You can’t get rid of the “bad” parts of technology and retain only the “good” parts. Take modern medicine, for example. Progress in medical science depends on progress in chemistry, physics, biology, computer science and other fields. Advanced medical treatments require expensive, high-tech equipment that can be made available only by a technologically progressive, economically rich society. Clearly you can’t have much progress in medicine ...more
54%
Flag icon
One might counter that computer viruses do not have the lethal potential of biological viruses or of destructive nanotechnology. This is not always the case; we rely on software to operate our 911 call centers, monitor patients in critical-care units, fly and land airplanes, guide intelligent weapons in our military campaigns, handle our financial transactions, operate our municipal utilities, and many other mission-critical tasks. To the extent that software viruses do not yet pose a lethal danger, however, this observation only strengthens my argument. The fact that computer viruses are not ...more
54%
Flag icon
The Threat from Fundamentalism. The world is struggling with an especially pernicious form of religious fundamentalism in the form of radical Islamic terrorism. Although it may appear that these terrorists have no program other than destruction, they do have an agenda that goes beyond literal interpretations of ancient scriptures: essentially, to turn the clock back on such modern ideas as democracy, women’s rights, and education. But religious extremism is not the only form of fundamentalism that represents a reactionary force. At the beginning of this chapter I quoted Patrick Moore, ...more
This highlight has been truncated due to consecutive passage length restrictions.
54%
Flag icon
The availability of Golden Rice has been delayed by at least five years through the pressure of Greenpeace and other anti-GMO activists. Moore, noting that this delay will cause millions of additional children to go blind, quotes the grain’s opponents as threatening “to rip the G.M. rice out of the fields if farmers dare to plant it.” Similarly, African nations have been pressured to refuse GMO food aid and genetically modified seeds, thereby worsening conditions of famine.40 Ultimately the demonstrated ability of technologies such as GMO to solve overwhelming problems will prevail, but the ...more
54%
Flag icon
Certain segments of the environmental movement have become fundamentalist Luddites—“fundamentalist” because of their misguided attempt to preserve things as they are (or were); “Luddite” because of the reflexive stance against technological solutions to outstanding problems. Ironically it is GMO plants—many of which are designed to resist insects and other forms of blight and thereby require greatly reduced levels of chemicals, if any—that offer the best hope for reversing environmental assault from chemicals such as pesticides. Actually my characterization of these groups as “fundamentalist ...more
54%
Flag icon
One of the reasons that calls for broad relinquishment have appeal is that they paint a picture of future dangers assuming they will be released in the context of today’s unprepared world. The reality is that the sophistication and power of our defensive knowledge and technologies will grow along with the dangers. A phenomenon like gray goo (unrestrained nanobot replication) will be countered with “blue goo” (“police” nanobots that combat the “bad” nanobots).
55%
Flag icon
In the case of nanotechnology, the ethics debate has started a couple of decades prior to the availability of the particularly dangerous applications. The most important provisions of the Foresight Institute guidelines include: • “Artificial replicators must not be capable of replication in a natural, uncontrolled environment.” • “Evolution within the context of a self-replicating manufacturing system is discouraged.” • “MNT device designs should specifically limit proliferation and provide traceability of any replicating systems.” • “Distribution of molecular manufacturing development ...more
55%
Flag icon
Inherently there will be no absolute protection against strong AI. Although the argument is subtle I believe that maintaining an open free-market system for incremental scientific and technological progress, in which each step is subject to market acceptance, will provide the most constructive environment for technology to embody widespread human values. As I have pointed out, strong AI is emerging from many diverse efforts and will be deeply integrated into our civilization’s infrastructure. Indeed, it will be intimately embedded in our bodies and brains. As such, it will reflect our values ...more
55%
Flag icon
Decentralization. One profound trend already well under way that will provide greater stability is the movement from centralized technologies to distributed ones and from the real world to the virtual world discussed above. Centralized technologies involve an aggregation of resources such as people (for example, cities, buildings), energy (such as nuclear-power plants, liquid-natural-gas and oil tankers, energy pipelines), transportation (airplanes, trains), and other items. Centralized technologies are subject to disruption and disaster. They also tend to be inefficient, wasteful, and harmful ...more
55%
Flag icon
The implications of dealing with an enemy that does not value its own survival are deeply troublesome and have led to controversy that will only intensify as the stakes continue to escalate. For example, when the FBI identifies a likely terrorist cell, it will arrest the participants, even though there may be insufficient evidence to convict them of a crime and they may not yet even have committed a crime. Under the rules of engagement in our war on terrorism, the government continues to hold these individuals.
55%
Flag icon
We come from goldfish, essentially, but that [doesn’t] mean we turned around and killed all the goldfish. Maybe [the AIs] will feed us once a week…. If you had a machine with a 10 to the 18th power IQ over humans, wouldn’t you want it to govern, or at least control your economy? —SETH SHOSTAK
55%
Flag icon
We need to streamline the regulatory process for genetic and medical technologies. The regulations do not impede the malevolent use of technology but significantly delay the needed defenses. As mentioned, we need to better balance the risks of new technology (for example, new medications) against the known harm of delay.
56%
Flag icon
Technology will remain a double-edged sword. It represents vast power to be used for all humankind’s purposes. GNR will provide the means to overcome age-old problems such as illness and poverty, but it will also empower destructive ideologies. We have no choice but to strengthen our defenses while we apply these quickening technologies to advance our human values, despite an apparent lack of consensus on what those values should be.
56%
Flag icon
The “criticism from analog processing”: Digital computation is too rigid because digital bits are either on or off. Biological intelligence is mostly analog, so subtle gradations can be considered. It’s true that the human brain uses digital-controlled analog methods, but we can also use such methods in our machines. Moreover, digital computation can simulate analog transactions to any desired level of accuracy, whereas the converse statement is not true.
56%
Flag icon
The “criticism from the rich-poor divide”: It’s likely that through these technologies the rich may obtain certain opportunities that the rest of humankind does not have access to. This, of course, would be nothing new, but I would point out that because of the ongoing exponential growth of price-performance, all of these technologies quickly become so inexpensive as to become almost free.
57%
Flag icon
Many of the furious attempts to argue why machines—nonbiological systems—cannot ever possibly compare to humans appear to be fueled by this basic reaction of incredulity. The history of human thought is marked by many attempts to refuse to accept ideas that seem to threaten the accepted view that our species is special. Copernicus’s insight that the Earth was not at the center of the universe was resisted, as was Darwin’s that we were only slightly evolved from other primates. The notion that machines could match and even exceed human intelligence appears to challenge human status once again. ...more
62%
Flag icon
The Criticism from the Rich-Poor Divide Another concern expressed by Jaron Lanier and others is the “terrifying” possibility that through these technologies the rich may gain certain advantages and opportunities to which the rest of humankind does not have access.39 Such inequality, of course, would be nothing new, but with regard to this issue the law of accelerating returns has an important and beneficial impact. Because of the ongoing exponential growth of price-performance, all of these technologies quickly become so inexpensive as to become almost free.
62%
Flag icon
Each example of information technology starts out with early-adoption versions that do not work very well and that are unaffordable except by the elite. Subsequently the technology works a bit better and becomes merely expensive. Then it works quite well and becomes inexpensive. Finally it works extremely well and is almost free. The cell phone, for example, is somewhere between these last two stages. Consider that a decade ago if a character in a movie took out a portable telephone, this was an indication that this person must be very wealthy, powerful, or both. Yet there are societies around ...more
62%
Flag icon
Human life without death would be something other than human; consciousness of mortality gives rise to our deepest longings and greatest accomplishments. —LEON KASS, CHAIR OF THE PRESIDENTIAL COMMISSION ON BIOETHICS, 2003
62%
Flag icon
As I reported in chapter 5, scientists have recently demonstrated the ability to reprogram skin cells into several other cell types. This approach represents the holy grail of cell therapies in that it promises an unlimited supply of differentiated cells with the patient’s own DNA. It also allows cells to be selected without DNA errors and will ultimately be able to provide extended telomere strings (to make the cells more youthful). Even embryonic stem-cell research itself has moved ahead, for example, with projects like Harvard’s major new research center and California’s successful ...more
62%
Flag icon
Ultimately the benefits of technical progress overwhelm such reflexive antitechnology sentiments. The majority of crops in the United States are already GMOs, while Asian nations are aggressively adopting the technology to feed their large populations, and even Europe is now beginning to approve GMO foods. The issue is important because unnecessary restrictions, although temporary, can result in exacerbated suffering of millions of people. But technical progress is advancing on thousands of fronts, fueled by irresistible economic gains and profound improvements in human health and well-being. ...more
62%
Flag icon
As technology becomes more sophisticated it increasingly takes on traditional human capabilities and requires less adaptation. You had to be technically adept to use early personal computers, whereas using computerized systems today, such as cell phones, music players, and Web browsers, requires much less technical ability. In the second decade of this century, we will routinely be interacting with virtual humans that, although not yet Turing-test capable, will have sufficient natural language understanding to act as our personal assistants for a wide range of tasks.