Superintelligence: Paths, Dangers, Strategies

by Nick Bostrom

The AI thus must be endowed with a criterion that it can use to determine which percepts constitute evidence in favor of some hypothesis about what the ultimate goal is, and which percepts constitute evidence against.