The Emperor's New Mind: Concerning Computers, Minds, and the Laws of Physics (Oxford Landmark Science)

by Roger Penrose

Even this achievement has been eclipsed in recent years by Penrose’s construction of two shapes that tile the plane, in the manner of an Escher tessellation, but which can tile it only in a non-periodic way. (You can read about these amazing shapes in my book Penrose Tiles to Trapdoor Ciphers.)

highlight

The operationalist would say that the computer thinks provided that it acts indistinguishably from the way that a person acts when thinking.

Now, it might be argued that this test is actually quite unfair on the

computer.

Thus, part of the task for the computer’s programmers is to make the computer appear to be ‘stupider’ than it actually is in certain respects. For if the interrogator were to ask the computer a complicated arithmetical question, as we had been considering above, then the computer must now have to pretend not to be able to answer it, or it would be given away at once! But I do not believe that the task of making the computer ‘stupider’ in this way would be a particularly serious problem facing the computer’s programmers. Their main difficulty would be to make it answer some of the simplest ‘common sense’ types of question – questions that the human subject would have no difficulty with whatever!

Let us set aside, for the moment, the issue of whether, or when, some computer might be made which actually passes the Turing test. Let us suppose instead, just for the purpose of argument, that such machines have already been constructed. We may well ask whether a computer, which does pass the test, should necessarily be said to think, feel, understand, etc. I shall come back to this matter very shortly. For the moment, let us consider some of the implications. For example, if the manufacturers are correct in their strongest claims, namely that their device is a thinking, feeling, sensitive, understanding, conscious being, then our purchasing of the device will involve us in moral responsibilities. It certainly should do so if the manufacturers are to be believed! Simply to operate the computer to satisfy our needs without regard to its own sensibilities would be reprehensible. That would be morally no different from maltreating a slave. Causing the computer to experience the pain that the manufacturers claim it is capable of feeling would be something that, in a general way, we should have to avoid. Turning off the computer, or even perhaps selling it, when it might have become attached to us, would present us with moral difficulties, and there would be countless other problems of the kind that relationships with other human beings or other animals tend to involve us in. All these would now become highly relevant issues. Thus, it would be of great importance for us to kn...

Why AI Isn't necessary

– medical, legal, etc. – is intended to be coded into a computer package! Is it possible that the experience and expertise of human members of these professions might actually be supplanted by such packages? Or is it merely that long lists of factual information, together with comprehensive cross-referencing, are all that can be expected to be achieved?

Note on Expert level AI

This is clearly illustrated by the example of a person touching a hot stove. An involuntary action is set up which causes him to withdraw his hand even before he experiences any sensation of pain. It would seem to be the case that such involuntary actions are very much closer to the responses of our device to its pp-score than are the actual effects of pain or pleasure.

do we retreat from heat prior to the sensation of burning?

a ‘Chinese room

read chinese room AI

Are not these individual Indians more like the individual neurons in a person’s brain than like the whole brain itself? No-one would suggest that neurons, whose firings apparently constitute the physical activity of a brain in the act of thinking, would themselves individually understand what that person is thinking, so why expect the individual Indians to understand the Chinese stories? Searle replies to this suggestion by pointing out the apparent absurdity of India, the actual country, understanding a story that none of its individual inhabitants understands. A country, he argues, like a thermostat or an automobile, is not in the ‘business of understanding’, whereas an individual person is.

cannot have any genuine understanding whatsoever of the tasks that they perform; also, it suggests (but no more) that no algorithm, no matter how complicated, can ever, of itself alone, embody genuine understanding – in contradistinction to the claims of strong AI.

Dualism is a philosophical viewpoint espoused by the highly influential seventeenth century philosopher and mathematician René Descartes,

The same holds for protons and for any other kind of particle, and for whole atoms, molecules, etc. If the entire material content of a person were to be exchanged with corresponding particles in the bricks of his house then, in a strong sense, nothing would have happened whatsoever. What distinguishes the person from his house is the pattern of how his constituents are arranged, not the individuality of the constituents themselves.

Abu Ja’far Mohammed ibn Mûsâ al-Khowârizm who wrote an influential mathematical textbook, in about 825 AD, entitled ‘Kitab al-jabr wa’l-muqabala’. The way that the name ‘algorithm’ has now come to be spelt, rather than the earlier and more accurate ‘algorism’, seems to have been due to an association with the word ‘arithmetic’. (It is noteworthy, also, that the word ‘algebra’ comes from the Arabic ‘al-jabr’ appearing in the title of his book.)

Check this book out

To apply Euclid’s algorithm, we divide one of our two numbers by the other and take the remainder:

concept of a general algorithm dates only from this century.

The most direct and persuasive of these, and also historically the most important, is in terms of the concept known as a Turing machine. It will be appropriate for us to examine these ‘machines’ in some detail.

Turing’s idealization is indeed remarkably well approximated by the electronic computers of today.

This seems to be a reasonable thing to do if we wish our device to function in a reliable and absolutely definite way.

a not uncommon feature of computer programs! (To understand fully why an algorithmic procedure does what it is supposed to involves insights. Are ‘insights’ themselves algorithmic?

Why not allow other symbols from some more complicated numbering system or alphabet? In fact, none of these changes makes the slightest difference to what can be in principle achieved, though some make a certain amount of difference to the economy of the operations (as would certainly be the case if we allowed more than one tape). The class of operations performed, and thus come under the heading of ‘algorithms’ (or ‘computations’ or ‘effective procedures’ or ‘recursive operations’), would be precisely the same as before even if we broadened the definition of our machines in all these ways at once!

Having two separate devices which do not directly communicate with one another achieves no more than having two which do communicate; and if they communicate, then, in effect, they are just a single device!

might think that we could find a Turing machine to churn out all the successive digits, 3, 1, 4, 1, 5, 9, . . ., of the above expansion for π one after the other on the output tape, where we simply allow the machine to run on forever. But this is not allowed for a Turing machine. We must wait for the machine to halt (indicated by the bell ringing!) before we are allowed to examine the output.

The numbers that can be generated in this way are called computable (Turing 1937). Those that cannot (actually the vast majority!) are non-computable.

Computable numbers Are discrete, by extension so are turing numbers

We sometimes loosely refer to the Turing machine whose number is n as the nth Turing machine, denoted Tn. Thus XN + 1 is the 450813704461563958982113775643437908th Turing machine!

correctly specified Turing machine is a sequence made up just of 0s, 10s, 110s, 1110s, and 11110s, and it therefore

111110 indeed signifies that the description of the number n is finished with.)

Tn(m) = .

Halting problem- when the turing machine doesn't end

Turing showed that indeed there is not. His argument was essentially the following. We first suppose that, on the contrary, there is such an algorithm.* Then there must be some Turing machine H which ‘decides’ whether or not the nth Turing machine, when acting on the number m, eventually stops. Let us say that it outputs the tape numbered O if it does not stop and 1 if it does: Here, one might take the coding of the pair (n, m) to follow the same rule as we adopted for the universal machine U. However this could run into the technical problem that for some number n (e.g. n = 7), Tn is not correctly specified; and the marker 111101 would be inadequate to separate n from m on the tape. To obviate this problem, let us assume that n is coded using the expanded binary notation rather than just the binary notation, with m in ordinary binary, as before. Then the marker 110 will actually be sufficient to separate n from m. The use of the semicolon in H(n; m), as distinct from the comma in U(n, m) is to indicate this change. Now let us imagine an infinite array, which lists all the outputs of all possible Turing machines acting on all the possible different inputs. The nth row of the array displays the output of the nth Turing machine, as applied to the various inputs 0, 1, 2, 3, 4, . . . : In the above table I have cheated a little, and not listed the Turing machines as they are actually numbered. To have done so would have yielded a list that looks much too boring to begin with, ...

We now apply a variant of an ingenious and powerful device, the ‘diagonal slash’ of Georg Cantor. (We shall be seeing the original version of Cantor’s diagonal slash in the next chapter.) Consider the elements of the main diagonal, marked now with bold figures:

Diagonals are important fo determining if you can count an infinite set by showing a sequence which does not exist

1 + 0 = . The question of whether or not a particular Turing machine stops is a perfectly well-defined piece of mathematics (and we have already seen that, conversely, various significant mathematical questions can be phrased as the stopping of Turing machines).

It has been helpful in the above description to use the anthropomorphic term ‘know’ in reference to an algorithm. However, is it not we who are doing the ‘knowing’, while the algorithm just follows the rules we have told it to follow? Or are we ourselves merely following rules that we have been programmed to follow from the construction of our brains and from our environment?

At least we should now have some feeling about the meanings of the terms ‘algorithm’ and ‘computability’, and an understanding of some of the related issues.

I feel that it is worth while to give a brief description of Church’s scheme not only because it emphasizes that computability is a mathematical idea, independent of any particular concept of computing machine, but also because it illustrates the power of abstract ideas in mathematics. The reader who is not readily conversant with mathematical ideas, nor intrigued by such things for their own sake, may, at this stage, prefer to move on to the next chapter – and there would not be significant loss in the

What is this strange, varied and most wonderfully intricate land that we have stumbled upon? No doubt many readers will already know. But some will not. This world is nothing but a piece of abstract mathematics – the set known as the Mandelbrot set.1 Complicated it undoubtedly is; yet it is generated by a rule of remarkable simplicity! To explain the rule properly, I shall first need to explain what a complex number is. It is as well that I do so here. We shall need complex numbers later.

The Mandelbrot set

natural numbers, and similarly one might think that the number of fractions is greater than the number of integers. However, this is not the case. According to the powerful and beautiful theory of infinite numbers put forward in the late 1800s by the highly original Russian-German mathematician Georg Cantor, the total number of fractions, the total number of integers and the total number of natural numbers are all the same infinite number, denoted ℵ0 (‘aleph nought’). (Remarkably, this kind of idea had been partly anticipated some 250 years before, in the early 1600s, by the great Italian physicist and astronomer Galileo Galilei.

I think there's some error in this. Take set theory for example: Natural numbers are countably infinite meaning you can have a bijective relationship between one Natural Integer and the positive integers. Real numbers however are not countably infinite because there is no bijection. I just left this note to myself as the distinction between countable and uncountable infinite should be noted. I suppose there's no degree to infinites permitting us to define the numbers as the same "aleph nought". Check this out.

It may be remarked that the computable numbers, on the other hand, are countable. To count them we just list, in numerical order, those Turing machines which generate real numbers (i.e. which produce the successive digits of real numbers). We may wish to strike from the list any Turing machine which generates a real number that has already appeared earlier in the list. Since the Turing machines are countable, it must certainly be the case that the computable real numbers are countable. Why can we not use the diagonal slash on that list and produce a new computable number which is not in the list? The answer lies in the fact that we cannot computably decide, in general, whether or not a Turing machine should actually be in the list. For to do so would, in effect, involve our being able to solve the halting problem. Some Turing machines may start to produce the digits of a real number, and then get stuck and never again produce another digit (because it ‘doesn’t stop’). There is no computable means of deciding which Turing machines will get stuck in this way. This is basically the halting problem.

The real number system is chosen in physics for its mathematical utility, simplicity, and elegance, together with the fact that it accords, over a very wide range, with the physical concepts of distance and time. It is not chosen because it is known to agree with these physical concepts over all ranges. One might well anticipate that there is indeed no such accord at very tiny scales of distance or time. It is commonplace to use rulers for the measurement of simple distances, but such rulers will themselves take on a granular nature when we get down to the scale of their own atoms. This does not, in itself, prevent us from continuing to use real numbers in an accurate way, but a good deal more sophistication is needed for the measurement of yet smaller distances. We should at least be a little suspicious that there might eventually be a difficulty of fundamental principle for distances on the tiniest scale. As it turns out, Nature is remarkably kind to us, and it appears that the same real numbers that we have grown used to for the description of things at an everyday scale or larger retain their usefulness on scales much smaller than atoms – certainly down to less than one-hundredth of the ‘classical’ diameter of a sub-atomic particle, say an electron or proton – and seemingly down to the ‘quantum gravity scale’, twenty orders of magnitude smaller than such a particle! This is a quite extraordinary extrapolation from experience. The familiar concept of real-number distanc...

This reminds me of Gauge theory in mathematics where means of measurement are not universal that discrepancies can occur from two different observations. Common examples would be lbs and kilograms. Measuring quantum behaviour. Measuring the lightyear distance between you and an object from one position in space, with another measuring the same object from another position in space, not accounting for discrepancies in light bending due to the curvature of space time, gravitational waves, so on so forth.

sin (A + B) = sin A cos B + cos A sin B, cos (A + B) = cos A cos B – sin A sin B, are simply the imaginary and real parts, respectively, of the much simpler (and much more memorable!) complex equation* eiA + iB = eiA eiB.

Wow, look over this

Fig. 3.12. A sequence of points in the Argand plane is bounded if there is some fixed circle that contains all the points. (This particular iteration starts with zero and has .) Now if we do this with certain choices of the given complex number c, the sequence of numbers that we get in this way never wanders very far from the origin in the Argand plane; more precisely, the sequence remains bounded for such choices of c which is to say that every member of the sequence lies within some fixed circle centred at the origin (see Fig. 3.12). A good example where this occurs is the case c – 0, since in this case, every member of the sequence is in fact 0. Another example of bounded behaviour occurs with c = –1, for then the sequence is: 0, –1, 0, –1, 0, –1,. . . ; and yet another example occurs with c = i, the sequence being 0, i, i –1, – i, i –1, –i, i –1, – i, . . . However, for various other complex numbers c the sequence wanders farther and farther from the origin to indefinite distance; i.e. the sequence is unbounded, and cannot be contained within any fixed circle.

The Mandelbrot set, that is to say, the black region of our world of Tor’Bled-Nam, is precisely that region of the Argand plane consisting of points c for which the sequence remains bounded.

The view that mathematical concepts could exist in such a timeless, etherial sense was put forward in ancient times (c. 360 BC) by the great Greek philosopher Plato. Consequently, this view is frequently referred to as mathematical Platonism. It will have considerable importance for us later.

‘God-given’ notion. In this chapter I have been arguing that such ‘God-given’ mathematical ideas should have some kind of timeless existence, independent of our earthly selves. Does not this viewpoint

See Mandelbrot (1986). The particular sequence of magnifications that I have chosen has been adapted from those of Peitgen and Richter (1986), where many remarkable coloured pictures of the Mandelbrot set are to be found. For further striking illustrations, see Peitgen and Saupe (1988).

In the late nineteenth century, mathematics had made great strides, partly because of the development of more and more powerful methods of mathematical proof. (David Hilbert and Georg Cantor, whom we have encountered before, and the great French mathematician Henri Poincaré, whom we shall encounter later, were three who were in the forefront of these developments.) Accordingly,

Mathematical proof

However, this confidence was shattered when in 1902 the British logician and philosopher Bertrand Russell produced his now famous paradox (itself anticipated by Cantor, and a direct descendant of Cantor’s ‘diagonal slash’ argument). To understand Russell’s argument, we first need some feeling for what is involved in considering sets as completed wholes. We may imagine some set that is characterized in terms of a particular property. For example, the set of red things is characterized in terms of the property of redness: something belongs to that set if and only if it has redness. This allows us to turn things about, and talk about a property in terms of a single object, namely the entire set of things with that property. With this viewpoint, ‘redness’ is the set of all red things. (We may also conceive that some other sets are just ‘there’, their elements being characterized by no such simple property.)