Superintelligence: Paths, Dangers, Strategies

by Nick Bostrom

A system motivated to promote our interests might be making a mistake if it allowed us to shut it down or to construct another, potentially unfriendly AI.)