The Worlds I See: Curiosity, Exploration, and Discovery at the Dawn of AI

by Fei-Fei Li

Although it had taken more than a half century for the necessary preconditions to align—historic milestones in the evolution of algorithms, large-scale data, and raw computing power, all converging at the dawn of the 2010s—it took less than a half decade for the capabilities they unleashed to change the world.

I wrote that the true impact of AI on the world would be largely determined by the motivation that guided the development of the technology—a disturbing thought in an era of expanding facial recognition and targeted advertising. But if we were to broaden our vision for AI to explicitly include a positive impact on humans and communities—if our definition of success could include such things—I was convinced that AI could change the world for the better. I still am.

As it happened, however, I was past the midpoint of a twenty-one-month sabbatical from my professorship at Stanford and was serving as chief scientist of AI at Google Cloud—placing me well within its epicenter.

The founding ideals of this country, however imperfectly they’ve been practiced in the centuries since, seemed as wise a foundation as any on which to build the future of technology: the dignity of the individual, the intrinsic value of representation, and the belief that human endeavors are best when guided by the many, rather than the few.

It matters what motivates the development of AI, in both science and industry, and I believe that motivation must explicitly center on human benefit.

I believe our civilization stands on the cusp of a technological revolution with the power to reshape life as we know it.

This revolution must build on that foundation, faithfully. It must respect the collective dignity of a global community. And it must always remember its origins: the restless imagination of an otherwise unremarkable species of hominid, so mystified by its own nature that it now seeks to re-create it in silicon. This revolution must, therefore, be unequivocally human-centered.

My mother in particular took pride in a subtle twist on the day’s parenting norms: I was expected to work hard and pursue the fullness of my potential, of course, but not for the sake of anyone or anything else.

Cultural fault lines notwithstanding, my parents’ love for me was sincere. Both were hardworking providers, even if my mother often took her responsibilities to perfectionist extremes that contrasted awkwardly with my father’s nonchalance.

“I asked the girls to leave because the time has come to tell you that your performance as a group is unacceptable. As boys, you’re biologically smarter than girls. Math and science are fundamental parts of that, and there’s just no excuse for your average exam scores to be lower than theirs. I’m deeply disappointed in you all today.”

Increased attention was being paid to algorithms that solved problems by discovering patterns from examples, rather than being explicitly programmed—in other words, learning what to do rather than being told. Researchers gave it a fitting name: “machine learning.”

Moreover, the structure of these networks is almost entirely learned, or at least refined, long after the brain’s initial formation in utero. It’s why, although our gray matter may appear anatomically indistinguishable, our personalities, skills, and memories are unique.

Hubel and Wiesel’s epiphany was that perception doesn’t occur in a single layer of neurons, but across many, organized in a hierarchy that begins with the recognition of superficial details and ends with complex, high-level awareness.

Finally, Yann LeCun, one of Hinton’s first students, famously applied it all to an impressively practical task: reading handwritten ZIP codes. In less than a decade, machine learning had progressed from a tenuous dream to a triumph in the real world.

It was as if he wanted to catalog the world—not by any formal means, and not even for any particular reason, but simply because he found joy in the process.

he read through Chinese classics like Dream of the Red Chamber, Romance of the Three Kingdoms, and Journey to the West.

My reading list grew more and more eclectic. I dove into Douglas Hofstadter’s Gödel, Escher, Bach: An Eternal Golden Braid, and was swept away by the range and depth of Roger Penrose’s The Emperor’s New Mind.

“Learning a new language,” she said, “is like opening a door to a new world.”

Even the term “artificial intelligence” itself—seen by many as hopelessly broad, if not delusional—was downplayed in favor of narrower pursuits like decision-making, pattern recognition, and natural language processing, which attempted to understand human speech and writing. “Artificial intelligence” seemed destined to remain the domain of science fiction writers, not academics.

Neural networks, simply put, were a good idea born at the wrong time.

My job included building the experiment’s apparatus from the ground up: researching the hardware, tracking down the proper electrodes, comparison shopping for the amplifiers and loudspeakers we’d use to listen to their output, then putting it all together from end to end. It was intense, and often stressful, but never boring.

In other words, the LGN sits somewhere between what the eye senses and what the brain understands; our goal was to decode the signals that pass through it along the way.

Even more than Princeton, it was the place that most fully demonstrated what my parents sought in coming to this country: the freedom to recognize a passion, and to live it, without compromise.

After all, what are thoughts if not reactions to stimuli, whether direct or otherwise?

It was less an exogenous force than an internal one, according to Parker, who identifies the fuse that ignited the Cambrian explosion as the emergence of a single capability: photosensitivity, or the foundation of the modern eye.

Vision, therefore, is not merely an application of our intelligence. It is, for all practical purposes, synonymous with our intelligence.

Vision Science,

Vision, therefore, isn’t just a question of the details of what we see. While images can be broken down and examined in the granular terms proposed by researchers like Treisman, especially in tightly controlled lab conditions, the vision we rely on to survive in a chaotic world deals in things—objects, people, and places. Indeed, from the earliest stages of processing, we perceive our surroundings not as an assemblage of colors and contours, but in terms of categories.

Our results joined the ranks of decades of work suggesting there was a simple idea at the heart of human visual perception: that above all else, our vision is based on an awareness of well-defined categories. On the recognition of things.

a data set now officially known as “Caltech 101”

Nevertheless, Biederman’s number—a potential blueprint for what our ambitions as researchers demanded—was big. Really big. It wasn’t 1,000, 2,000, or even 5,000. And it certainly wasn’t the 101 we spent months cataloging. It was 30,000.

It would be like a map of everything humans value—everything we’ve come to describe with a word—arranged in one connected space. This, in a nutshell, was WordNet.

First, there was WordNet: a lexical database of almost indescribable ambition that seemed to capture the entirety of the world’s concepts, organized in a natural hierarchy of human meaning. Then there was ImageNet: an attempt to assign a single picture to each concept. Both projects seemed like responses to the yawning, mysterious space that Biederman’s number had created in my thoughts.

All told, we cut the majority of WordNet’s 140,000 entries, leaving a visual, countable subset just north of 22,000.

Mechanical Turk,

Andrew Ng, Daphne Koller, and Sebastian Thrun,

By June 2009, due in large part to the infusion of new research funding provided by Stanford, the inaugural version of ImageNet was complete. In spite of the many challenges we’d faced along the way, we’d actually done it: fifteen million images spread across twenty-two thousand distinct categories, culled from nearly a billion candidates in total, and annotated by a global team of more than forty-eight thousand contributors hailing from 167 countries.

“I don’t think ImageNet will make today’s algorithms better,” I said. “I think it will make them obsolete.”

Our world evolved fast, and by the 2010s, most of us saw the neural network—that biologically inspired array of interconnected decision-making units arranged in a hierarchy—as a dusty artifact, encased in glass and protected by velvet ropes.

The winner was dubbed AlexNet, in homage to both the technique and the project’s lead author, University of Toronto researcher Alex Krizhevsky.

The project was helmed by the eponymous Alex Krizhevsky and his collaborator, Ilya Sutskever, both of whom were smart but young researchers still building their reputations. The third name, however, caught my attention instantly: Geoffrey E. Hinton. The same Hinton who’d made his name as an early machine learning pioneer with the development of backpropagation in the mid-1980s, the breakthrough method that made it possible to reliably train large neural networks for the first time. The Hinton who had mentored Yann LeCun when he was still a student in his lab. The Hinton who, like his protégé, had refused to give up on the study of neural networks, even when it made him a near outcast among his colleagues.

In yet another twist of fate, the style of number-crunching favored by neural networks is functionally similar to the kind used in rendering the graphics for video games—a multibillion-dollar industry that had been driving the advancement and commercialization of custom hardware since the 1990s, fueling the growth of megabrands like Nvidia, the company at the forefront of the field. By 2012, such hardware—specialized processors known as “graphics processing units,” or GPUs—had attained affordable, consumer-level status. For Hinton’s lab, that meant the silicon needed to bring AlexNet to life was no longer an investment requiring a government grant and construction permits. It was available off the shelf at Best Buy.

Now in its final form, the diversity that it took a world of crowdsourced volunteers to curate has forged a topology so varied, so robust, that a kind of holy grail has been attained. This neural network, the largest our field has ever seen, trained by more data than any in history, can generalize.

Because we intended to use cars as a proxy for exploring larger socioeconomic questions—correlating them with aspects of their owners’ identities like income, education, and occupation—we had to face the fact that dramatic gaps in monetary value often translate to only subtle differences in outward appearance. So while we had little trouble distinguishing a Cadillac sedan from a Toyota pickup truck, early experiments suggested it was perilously easy for a “naively” trained classifier to mistake a Cadillac for, say, a Honda Accord, especially when the cars were painted in similar colors—precisely the kind of mistake we sought to avoid.

The 1998 article, entitled “Visual Memory: What Do You Know About What You Saw?,” was written in an almost colloquial tone, but its conclusions were incisive. Seeing an image, as he put it, prompted our brains to “remember the gist of the scene.”

Back in the 1970s, the researcher and mathematician Anatol Holt summed up this myopia by saying that AI was a technology that can make a perfect chess move while the room is on fire. How relevant that diagnosis still felt, even now. Modern AI behaved like a kind of game-playing savant, mastering isolated tasks that lent themselves to narrow metrics like “error rate,” while failing to notice the burning embers that were falling on the board.

If Einstein, Bohr, and Wheeler were cosmic dreamers, students like Andrej were different, cut from the same cloth, perhaps, as Edison or the Wright brothers.

Some were amusing confirmations of stereotypes, such as our finding about a city’s ratio of sedans to pickup trucks: when the former is higher, the city is 88 percent likely to vote Democrat; when the latter is higher, it’s 82 percent likely to vote Republican. But that was just the beginning.

I wondered, in fact, if the true value of the North Star as a metaphor wasn’t just its ability to guide but the fact that its distance remains perpetually infinite. It can be pursued until the point of exhaustion, the object of a lifetime’s obsession, but never be reached. It’s a symbol of the scientist’s most distinctive trait: a curiosity so restless that it repels satisfaction, like opposing magnets, forever. A star in the night, a mirage in the distance, a road without end. This, I realized, was what AI was becoming for me.

Our rival wasn’t some mysterious team of researchers at another university. It was Google.