Genius Makers: The Mavericks Who Brought A.I. to Google, Facebook, and the World

by Cade Metz

Google Brain had already explored a technology called “word embeddings.”14 This involved using a neural network to build a mathematical map of the English language by analyzing a vast collection of text—news articles, Wikipedia articles, self-published books—to show the relationship between every word in the language and every other word. This wasn’t a map you could ever hope to visualize. It didn’t have two dimensions, like a road map, or three dimensions, like a video game. It had thousands of dimensions, like nothing you’ve ever seen, or could ever see. On this map, the word “Harvard” was close to the words “university,” “Ivy,” and “Boston,” even though those words weren’t linguistically related. The map gave each word a mathematical value that defined its relationship to the rest of the language. This was called a “vector.” The vector for “Harvard” looked a lot like the vector for “Yale,” but they weren’t identical. “Yale” was close to “university” and “Ivy,” but not “Boston.”