It is important to understand that self-preservation doesn’t have to be any sort of built-in instinct or prime directive in machines. (So Isaac Asimov’s Third Law of Robotics,8 which begins “A robot must protect its own existence,” is completely unnecessary.) There is no need to build self-preservation in because it is an instrumental goal—a goal that is a useful subgoal of almost any original objective.9 Any entity that has a definite objective will automatically act as if it also has instrumental goals.
