Superintelligence: Paths, Dangers, Strategies

by Nick Bostrom

In addition to faster hardware, a contemporary project would benefit from the great strides that have been made in the many subfields of AI, in software engineering more generally, and in neighboring fields such as computational neuroscience.

Advances in neuroscience and cognitive psychology—

which will be aided by improvements in instrumentation—should eventually uncover the general principles of brain function. This knowledge could then guide AI efforts.

First, a sufficiently detailed scan of a particular human brain is created.

Second, the raw data from the scanners is fed to a computer for automated image processing to reconstruct the three-dimensional neuronal network that implemented cognition in the original brain.

In the third stage, the neurocomputational structure resulting from the previous step is implemented on a sufficiently powerful computer.

Growth in collective intelligence may also come from more general organizational and economic improvements, and from enlarging the fraction of the world’s population that is educated, digitally connected, and integrated into global intellectual

Let us therefore return to the question of how we could safely keep a single superintelligent AI.

forms of AI that mimic some aspects of cortical organization but do not replicate neuronal functionality with sufficient fidelity to constitute a proper emulation.

technology coupling.”11 This refers to a condition in which two technologies have a predictable timing relationship, such that developing one of the technologies has a robust tendency to lead to the development of the other, either as a necessary precursor or as an obvious and irresistible application or subsequent step.

Before you marry your sweetheart, consider the prospective in-laws.

(For example: the McCulloch–Pitts neuron, perceptrons, and other artificial neurons and neural networks, inspired by neuroanatomical work; reinforcement learning, inspired by behaviorist psychology; genetic algorithms, inspired by evolution theory; subsumption architectures and perceptual hierarchies, inspired by cognitive science theories about motor planning and sensory perception; artificial immune systems, inspired by theoretical immunology; swarm intelligence, inspired by the ecology of insect colonies and other self-organizing systems; and reactive and behavior-based control in robotics, inspired by the study of animal locomotion.)