Superintelligence: Paths, Dangers, Strategies

by Nick Bostrom

One feature of a malignant failure is that it eliminates the opportunity to try again. The number of malignant failures that will occur is therefore either zero or one.