“Paper-clip problem. Convergent instrumental goals. Value loading. All the AI theory that has been developed over the centuries tells us that even if an AI isn’t explicitly and deliberately antagonistic, it may still perform actions that are harmful to the human race in pursuit of its own goals. And it has every advantage over us that we do over humanity, plus more. It can multitask in ways we simply can’t. It won’t have any emotional or psychological issues with cloning. It can self-modify to improve itself and add capabilities. It could, in the most trivial example, start harvesting stellar
...more
