Superintelligence: Paths, Dangers, Strategies

by Nick Bostrom

In such circumstances to select a final value based on our current convictions, in a way that locks it in forever and precludes any possibility of further ethical progress, would be to risk an existential moral calamity.