Superintelligence: Paths, Dangers, Strategies

Naturalism -- the doctrine that there is nothing apart from the physical order, and certainly nothing supernatural -- is an incorrigibly incoherent concept, and one that is ultimately indistinguishable from pure magical thinking. The very notion of nature as a closed system entirely sufficient to itself is plainly one that cannot be verified, deductively or empirically, from within the system of nature. It is a metaphysical (which is to say 'extranatural') conclusion regarding the whole of reality, which neither reason nor experience legitimately warrants. . . .

If moreover, naturalism is correct (however implausible that is), and if consciousness is then an essentially material phenomenon, then there is no reason to believe that our minds, having evolved purely through natural selection, could possibly be capable of knowing what is or is not true about reality as a whole. Our brains may necessarily have equipped us to recognize certain sorts of physical objects around them, but there is no reason to suppose that such structures have access to any abstract 'truth' about the totality of things. If naturalism is true as a picture of reality, it is necessarily false as a philosophical precept.

The one thing of which it can give no account, and which its most fundamental principles make it entirely impossible to explain at all, is nature's very existence. For existence is definitely not a natural phenomenon; it is logically prior to any physical cause whatsoever; and anyone who imagines that it is susceptible of a natural explanation simply has no grasp of what the question of existence really is. . . . There is something of a popular impression out there the naturalist position rests upon a particularly sound rational foundation. But, in fact, materialism is among the most problematic of philosophical standpoints, the most impoverished in its explanatory range, and among the most willful and (for want of a better word) magical in its logic. . . . The heuristic metaphor of a purely mechanical cosmos has become a kind of ontology, a picture of reality as such. The mechanistic view of consciousness remains a philosophical and scientific premise only because it is now an established cultural bias, a story we have been telling ourselves for centuries, without any real warrant from either reason or science. . . .

Naturalism commits the genetic fallacy, the mistake of thinking that to have described a thing's material history or physical origins is to have explained that thing exhaustively. We tend to presume that if one can discover the temporally prior physical causes of some object -- the world, an organism, a behavior, a religion, a mental event, an experience, or anything else -- one has thereby eliminated all other possible causal explanations of that object. . . . To bracket form and finality out of one's investigations as far as reason allows is a matter of method, but to deny their reality altogether is a matter of metaphysics. if common sense tells us that real causality is limited solely to material motion and the transfer of energy, that is because a great deal of common sense is a cultural artifact produced by an ideological heritage. . . .

Consciousness is a reality that cannot be explained in any purely physiological terms at all. The widely cherished expectation that neuroscience will one day discover an explanation of consciousness solely within the brain's electrochemical processes is no less enormous a category error than the expectation that physics will one day discover the reason for the existence of the material universe. It is a fundamental conceptual confusion, unable to explain how any combination of diverse material forces, even when fortuitously gathered into complex neurological systems, could just somehow add up to the simplicity and immediacy of consciousness, to its extraordinary openness to the physical world, to its reflective awareness of itself. . . .

Naturalists fall victim to the pleonastic fallacy, another hopeless attempt to overcome qualitative difference by way of an indeterminately large number of gradual quantitative steps. This is the great non sequitir that pervades practically all attempts, evolutionary or mechanical, to reduce consciousness wholly to its basic physiological constituents. At what point precisely was the qualitative difference between brute physical causality and unified intentional subjectivity vanquished? And how can that transition fail to have been an essentially magical one? There is no bit of the nervous system that can become the first step toward intentionality.

One can conduct an exhaustive surveillance of all those electrical events in the neurons of the brain that are undoubtedly the physical concomitants of mental states, but one does not thereby gain access to that singular, continuous, and wholly interior experience of being this person that is the actual substance of conscious thought. . . . There is an absolute qualitative abyss between the objective facts of neurophysiology and the subjective experience of being a conscious self. . . . Consciousness as we commonly conceive of it is also almost certainly irreconcilable with a materialist view of reality, and there is no ‘question’ of whether subjective consciousness really exists -- subjective consciousness is an indubitable primordial datum, the denial of which is simply meaningless.