Genius Makers: The Mavericks Who Brought A.I. to Google, Facebook, and the World

by Cade Metz

As Google and other tech giants adopted the technology, no one quite realized it was learning the biases of the researchers who built it. These researchers were mostly white men, and they didn’t see the breadth of the problem until a new wave of researchers—both women and people of color—put a finger on it. As the technology spread even further, into healthcare, government surveillance, and the military, so did the ways it might go wrong. Deep learning brought a power even its designers did not completely understand how to control, particularly as it was embraced by tech superpowers driven by an insatiable hunger for revenue and profit.

As Google and other tech giants adopted the technology, no one quite realized it was learning the biases of the researchers who built it. These researchers were mostly white men, and they didn’t see the breadth of the problem until a new wave of researchers—both women and people of color—put a finger on it. As the technology spread even further, into healthcare, government surveillance, and the military, so did the ways it might go wrong. Deep learning brought a power even its designers did not completely understand how to control, particularly as it was embraced by tech superpowers driven by an insatiable hunger for revenue and profit.

One day, he said, the Perceptron would travel into space and send its observations back to Earth. When the reporter asked if there was anything the Perceptron was not capable of, Rosenblatt threw up his hands. “Love. Hope. Despair.19

Human nature, in short,” he said. “If we don’t understand

the human sex drive, how should we expect...

“The perceptron program is not primarily concerned with the invention of devices for34 ‘artificial intelligence,’ but rather with investigating the physical structures and neurodynamic principles which underlie ‘natural intelligence,’ ” he wrote. “Its utility is in enabling us to determine the physical conditions for the emergence of various psychological properties.”

The answer, Rumelhart suggested, was a process called “backpropation.” This was essentially an algorithm, based on differential calculus,

that sent a kind of mathematical feedback cascading down the hierarchy of neurons as they analyzed more data and gained a better understanding of what each weight should be.

Hinton liked to say that “old ideas are new”—that scientists should never give up on an idea unless someone had proven it wouldn’t work.

LeCun and his colleagues also designed a microchip they called ANNA. It was an acronym inside an acronym. ANNA was the acronym for Analog Neural Network ALU,5 and ALU stood for Arithmetic Logic Unit, a kind of digital circuit suited to running the mathematics that drove neural networks. Instead of running their algorithms using ordinary chips built for just any task, LeCun’s team built a chip for this one particular task. That meant it could handle the task at speeds well beyond the standard processors of the day: about 4 billion operations a second. This fundamental concept—silicon built specifically for neural networks—would remake the worldwide chip industry, though that moment was still two decades away.

Across the field, neural networks showed up in less than 5 percent of all published research papers. When submitting papers to conferences and journals, hoping to improve their chances of success, some researchers would replace the words “neural network” with very different language, like “function approximation” or “nonlinear regression.” Yann LeCun removed the word “neural” from the name of his most important invention. “Convolutional neural networks” became “convolutional networks.”

One year, Clément Farabet,8 one of his PhD students, built a neural network that could analyze a video and separate different kinds of objects—the trees from the buildings, the cars from the people. It was a step toward computer vision for robots or self-driving cars, able to perform its task with fewer errors than other methods and at faster speeds, but reviewers at one of the leading vision conferences summarily rejected the paper. LeCun responded with a letter to the conference chair saying that the reviews were so ridiculous, he didn’t know how to begin writing a rebuttal without insulting the reviewers. The conference chair posted the letter online for all to see, and though he removed LeCun’s name, it was obvious who had written it.

When Hinton gave a lecture at the annual NIPS conference, then held in Vancouver, on his sixtieth birthday, the phrase “deep learning” appeared in the title for the first time. It was a cunning piece of rebranding. Referring to the multiple layers of neural networks, there was nothing new about “deep learning.” But it was an evocative term designed to galvanize research in an area that had once again fallen from favor. He knew the name was a good one when, in the middle of the lecture, he said everyone else was doing “shallow learning,” and his audience let out a laugh. In the long term, it would prove to be a masterly choice. And it immediately fed the reputation of this tiny group of researchers working on the fringes of academia.

What their prototype still lacked was the extra processing power needed to analyze all that data. In Toronto, Hinton made use of a very particular kind of computer chip called a GPU, or graphics processing unit. Silicon Valley chip makers like Nvidia originally designed these chips as a way of quickly rendering graphics for popular video games like Halo and Grand Theft Auto, but somewhere along the way, deep learning researchers realized GPUs were equally adept at running the math that underpinned neural networks. In 2005, three engineers had tinkered with the idea inside the same Microsoft lab where Deng and Hinton would later build their speech prototype,3 and a team at Stanford University stumbled onto the same technical trick around the same time.4

At first, Deng balked at the price. His boss, Alex Acero, who would one day oversee Siri, the digital assistant on the Apple iPhone, told him this was an unnecessary expense. GPUs were for games, not AI research. “Don’t waste your money,” he said, telling Deng to skip the expensive Nvidia gear and buy generic cards at the local Fry’s Electronics store. But Hinton urged Deng to push back, explaining that cheap hardware would defeat the purpose of the experiment. The idea was that a neural network would analyze Microsoft’s speech data over the course of several days, and those generic cards might flame out if they ran that hot for that long. But his larger point was that neural networks thrived on extra processing power. Deng needed to buy not only a $10,000 GPU card but maybe more than one, plus a specialized server that could run the card. That would cost just as much as the card. “It would cost you about $10,000,” Hinton said in an email to Deng. “We are about to order 3, but then we are a well funded Canadian university, not a poor software seller.” In the end, Deng bought the necessary hardware.

Dahl pushed Hinton’s speech prototype beyond the performance of anything else under development at the company. What he and Mohamed and Hinton showed was that a neural network could sift through a sea of very noisy speech and somehow find the stuff that mattered, the patterns that no human engineer could ever pinpoint on their own, the telltale signs that distinguished one subtle sound from another, one word from another. It was an inflection point in the long history of artificial intelligence. In a matter of months, a professor and his two graduate students matched a system that one of the world’s largest companies had worked on for more than a decade. “He is a genius,” Deng says. “He knows how to create one impact after another.”

Hawkins invented the PalmPilot, a forerunner of the iPhone from the 1990s, but what he really wanted to do was study the brain. In his book he argued that the whole of the neocortex—the part of the brain that handled sight, hearing, speech, and reason—is driven by a single biological algorithm. If scientists could re-create this algorithm, he said, they could re-create the brain.

They also hired additional researchers from Stanford and Toronto as the project graduated from Google X to a dedicated AI lab, Google Brain. The rest of the industry, and even parts of Google Brain, didn’t quite realize what was about to happen. Just as the lab reached this key moment, Andrew Ng decided to leave. He had another project in the works, and it needed his attention. He was building a start-up, Coursera, that specialized in MOOCs, or Massive Open Online Courses, a way of delivering a university education over the Internet. In 2012, it was one of those Silicon Valley ideas that entrepreneurs, investors, and journalists were convinced would completely change the world. At the same moment, Sebastian Thrun was creating a similar start-up called Udacity. Neither project, though, could begin to compete with what was about to unfold inside Google Brain.

At first, Malik said deep learning would have to master a European dataset called PASCAL. “PASCAL is too small,” Hinton told him. “To make this work, we need a lot of training data. What about ImageNet?” Malik told him they had a deal. ImageNet was an annual contest run by a lab at Stanford,15 about forty miles south of Berkeley. The lab had compiled a vast database of carefully labeled photos, everything from dogs to flowers to cars, and each year, researchers across the globe competed to build a system that could recognize the most images. Excelling at ImageNet, Hinton thought, would certainly clinch the argument. What he didn’t tell Malik was that his lab was already building a neural network for the upcoming competition, and that thanks to two of his students—Ilya Sut skever and Alex Krizhevsky—it was nearly finished.

“He’s got three things,” Hinton says. “He’s very bright, he’s very competitive, and he’s very good at social interactions. That’s a dangerous combination.”

Hassabis had two obsessions. One was designing computer games. In his gap year,5 he helped celebrated British designer Peter Molyneux create Theme Park, in which players build and operate a sprawling digital simulation of a Ferris-wheel-and-rollercoaster amusement park. It sold an estimated 10 million copies, helping to inspire a whole new breed of game—“sims” that re-created huge swaths of the physical world.

Most people couldn’t see beyond what they were already doing. The trick, Alan Eustace believed, was to surround yourself with people who could apply new kinds of expertise to problems that seemed unsolvable with the old techniques. “Most people look at particular problems from a particular point of view and a particular perspective and a particular history,” he says. “They don’t look at the intersections of expertise that will change the picture.”

“Deep learning should not be called AI,” Krizhevsky says. “I went to grad school for curve setting, not AI.” What he did, first at Google Brain and then inside the self-driving car project, was apply the math to new situations. This was very different from any effort to re-create the brain—and far from vague fears that machines would someday spin outside our control. It was computer science. Others agreed, but this was not a view that made headlines.

The strength of the system, he told his audience, was its simplicity. “We use minimum innovation for maximum results,” he said, as applause rippled across the crowd, catching even him by surprise.

But Sutskever didn’t see it merely as a breakthrough in translation. He saw it as a break-through for any AI problem that involved a sequence, from automatically generating captions for photos to instantly summarizing a news article with a sentence or two.

“The real conclusion is that if you have a very large dataset and a very large neural network,” he told his audience, “then success is guaranteed.”

Now Dean and Hölzle tapped into this talent for the new chip project while also hiring seasoned chip engineers from Silicon Valley companies like HP. The result was the tensor processing unit, or TPU. It was designed to process the tensors—mathematical objects—that under-pinned a neural network. The trick was that its calculations were less precise than typical processors.20 The number of calculations made by a neural network was so vast, each calculation didn’t have to be exact. It dealt in integers rather than floating point numbers. Rather than multiply 13.646 by 45.828, the TPU lopped off the decimal points and just multiplied 13 and 45.

The conversation stayed private for years, until Vance published his biography of Musk, but soon after their dinner, Musk was saying much the same thing across both national TV and social media. During an appearance on CNBC, he invoked The Terminator.6 “There have been movies about this,” he said. On Twitter, he called artificial intelligence “potentially more dangerous than nukes.”7

That fall, Musk appeared onstage at a Vanity Fair conference in New York City, warning author Walter Isaacson about the dangers of artificial intelligence designed for11 “recursive self-improvement.” If researchers designed a system to fight email spam,12 he explained, it could end up deciding that the best way of eliminating all the spam was just to remove all the people.

Even as Musk sounded the alarm that the race for artificial intelligence could destroy us all, he was joining it. For the moment, he was chasing the idea of a self-driving car, but he would soon chase the same grandiose idea as DeepMind, creating his own lab in pursuit of AGI. For Musk, it was all wrapped up in the same technological trend. First image recognition. Then translation. Then driverless cars. Then AGI.