Genius Makers: The Mavericks Who Brought A.I. to Google, Facebook, and the World

by Cade Metz

The same tweet urged his followers to read Superintelligence:8 Paths, Dangers, Strategies, a recently published tome from an Oxford University philosopher named Nick Bostrom. Like Shane Legg, the founder of DeepMind, Bostrom believed that superintelligence could secure the future of humanity—or destroy it.9 “This is quite possibly the most important and most daunting challenge humanity has ever faced,”10 he wrote. “And—whether we succeed or fail—it is probably the last challenge we will ever face.” His concern was that scientists would design a system to perfect a particular part of our lives without realizing it would one day wreak havoc in ways no one had the power to stop. His oft-repeated metaphor was a system designed to create as many paperclips as possible. Such a system, he said, could transform “first all of earth and then increasing portions of space into paperclip manufacturing facilities.”