Rationality: From AI to Zombies

by Eliezer Yudkowsky

if you can see how (not just know that) a higher level reduces to a lower one, they will not seem like separate things within your map; you will be able to see how silly it is to think that your fingers could be in one place, and your hand somewhere else; you will be able to see how silly it is to argue about whether it is your hand that picks up the cup, or your fingers. The operative word is “see,” as in concrete visualization. Imagining your hand causes you to imagine the fingers and thumb and palm; conversely, imagining fingers and thumb and palm causes you to identify a hand in the mental picture. Thus the high level of your map and the low level of your map will be tightly bound together in your mind. In reality, of course, the levels are bound together even tighter than that—bound together by the tightest possible binding: physical identity. You can see this: You can see that saying (1) “hand” or (2) “fingers and thumb and palm,” does not refer to different things, but different points of view.

1) XYZ is not conceivable if you can see why - and not just know that - H2O is water. 2) Composition as identity?