Superintelligence: Paths, Dangers, Strategies

by Nick Bostrom

Since then, the expected arrival date has been receding at a rate of one year per year;

The Google search engine is, arguably, the greatest AI system that has yet been built.

apex cogitator

Opportunities for private individuals to reduce the overall amount of existential risk from a potential intelligence explosion are therefore greatest in scenarios in which big players remain relatively oblivious to the issue, or in which the early efforts of activists make a major difference to whether, when, which, or with what attitude big players enter the game. Activists seeking maximum expected impact may therefore wish to focus most of their planning on such high-leverage scenarios, even if they believe that scenarios in which big players end up calling all the shots are more probable.

A more benign course of action, which might also have had a chance of working, would have been to use its nuclear arsenal as a bargaining chip to negotiate a strong international government—a veto-less United Nations with a nuclear monopoly and a mandate