Rationality: From AI to Zombies

by Eliezer Yudkowsky

If, for example, someone believes the authority of “Thou Shalt Not Kill” derives from God, then there are several and well-known things to say that can help set up a line of retreat—as opposed to immediately attacking the plausibility of God. You can say, “Take personal responsibility! Even if you got orders from God, it would be your own decision to obey those orders. Even if God didn’t order you to be moral, you could just be moral anyway.” The above argument actually generalizes to quite a number of metaethics—you just substitute Their-Favorite-Source-Of-Morality, or even the word “morality,” for “God.” Even if your particular source of moral authority failed, couldn’t you just drag the child off the train tracks anyway? And indeed, who is it but you that ever decided to follow this source of moral authority in the first place? What responsibility are you really passing on?