The Thinking Machine Quotes

“The truth, not widely understood until later, was that the deep-learning revolution was as much a revolution in hardware as software. It was the product of not one but two unpopular, cast-off, discredited, and cash-starved technologies whose ideal form could only be revealed in synthesis. Neural nets running on parallel computers: these tightly coupled technologies were the twin strands of DNA for a new and powerful organism, looking to consume all the data in the world.”
― Stephen Witt, The Thinking Machine: Jensen Huang and Nvidia, the company shaping the future of AI

“Nickolls was obsessed with getting the CUDA platform to work. Friends sometimes asked him why he was working for a video game company when he didn't play video games. Nickolls informed them he wasn't working on video games; he was working on one of the most important technologies of all time. He was building a platform so fast it would make every other computer look like a calculator watch. "Few inventions will have the impact on the world that CUDA will ultimately have", he would say.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“Is it going to—is it going to destroy jobs?” Huang asked, his voice crescendoing with anger. “Are calculators going to destroy math? That conversation is so old, and I’m so, so tired of it,” he said. “I don’t want to talk about it anymore! It’s the same conversation over and over and over and over and over again. We invented agriculture and then made the marginal cost of producing food zero. It was good for society! We manufactured electricity at scale, and it caused the marginal cost of chopping down trees, lighting fires, carrying fires and torches around to approximately zero, and we went off to do something else. And then we made the marginal cost of doing calculations—long division! We made it zero!” He was yelling now. “We make the marginal cost of things zero, generation after generation after generation, and this exact conversation happens every single time!” [...] “This cannot be a ridiculous sci-fi story,” he said. He gestured to his frozen PR reps at the end of the table. “Do you guys understand? I didn’t grow up on a bunch of sci-fi stories, and this is not a sci-fi movie. These are serious people doing serious work!” he said. “This is not a freaking joke! This is not a repeat of Arthur C. Clarke. I didn’t read his fucking books. I don’t care about those books! It’s not—we’re not a sci-fi repeat! This company is not a manifestation of Star Trek! We are not doing those things! We are serious people, doing serious work. And— it’s just a serious company, and I’m a serious person, just doing serious work.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“Garlick was given access to Nvidia’s code base. He was appalled at what he saw.
“Basically, it was cancer,” he said. “Y’know, cancer cells aren’t efficient. They just mutate,
grow, and expand.” At 3dfx, Garlick had taken pride in the elegance of his programming,
developing orderly systems with lucid commenting, allowing other programmers to easily
maintain and improve his work. “In the time we spent making it clean, we went out of
business,” he said. Nvidia’s approach was slapdash, with blocks of code written during some
delirious midnight crunch serving as the foundation for critical systems. “What a shit show!
The code was crap, the tool-chain was a mess, and the thing was, they didn’t give a shit!”
Garlick said. “They didn’t give a shit about anything but the next tape-out.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“He could talk extemporaneously for more than two hours at a time, and in these presentations he would often revisit the same themes: the importance of the “speed of light” scheduling concept, the pursuit of the fabled “zero-billion-dollar market,” and above all, the ever-present danger of creeping bureaucracy.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“what set Huang apart was his ability to avoid dead ends. “Similar people, they get lost, right?” Horstmann said. “They just get lost in these deep, deep ratholes. He doesn’t. He has a great sense of seeing when a problem has reached a certain level of complexity, and he can’t easily make further progress, and he has to go in a different direction.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“Huang was athletic and had good reflexes, but his unique quality was his exceptional focus. When he set his mind to self-improvement, the rest of the world faded away. He outworked everyone; he didn’t seem to get frustrated or stuck; he never hit a plateau. Instead, Huang watched, with measured satisfaction, as his patient dedication to the fundamentals slowly manifested itself as skill.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“Desperation, not inspiration, was the mother of victory. Huang encouraged his employees to preserve the mindset they’d adopted during the Riva crunch, asking them to constantly behave as if the company was teetering on the verge of bankruptcy even when it was making massive profits. For years to come, Jensen opened staff presentations with the words “Our company is thirty days from going out of business.” Even today at Nvidia, this sentence remains the corporate mantra.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“Now I saw where the fear was coming from. The executives were more afraid of Jensen yelling at them than they were of wiping out the human race.”
― Stephen Witt, The Thinking Machine: Jensen Huang and Nvidia, the company shaping the future of AI

“But the final word went to Morris Chang. He didn’t attribute Huang’s success to his work ethic, which, at TSMC, would have been considered slightly above average—nor did he find him especially adaptable. Chang was ninety-two years old when I spoke with him, wearing a purple corded sweater and sitting in front of a striking piece of abstract art, his face serene, his hair completely white. In seventy years of corporate life across two continents he had seen every manner of executive there was to see. To him, the explanation was simple, and there was no secret: “His intellect is just superior.”
― Stephen Witt, The Thinking Machine: Jensen Huang and Nvidia, the company shaping the future of AI

“For Dally, it was Huang’s tireless work ethic that made Nvidia succeed. Even Dally, who left no spare second in his day, could not quite believe the superhuman efforts of his boss. “The rest of us are just here to reduce the bandwidth demands on Jensen,” Dally said. “I mean, when does he sleep?” Diercks agreed: “His hobbies are work, email, and work.”
― Stephen Witt, The Thinking Machine: Jensen Huang and Nvidia, the company shaping the future of AI

“Krizhevsky himself escaped the dragnet. Rarely saying a word to anyone, he was not the ideal collaborator, and he departed Google in 2017. His share of the auction money was just under $15 million, enough not to work anymore, especially given the asceticism of his lifestyle. In 2019 he granted a Japanese news crew a visit inside his Bay Area home. Krizhevsky lived like a Benedictine monk, in a spartan apartment above a Vietnamese restaurant. The walls inside were completely bare; the only items of furniture were a desk, a couch, a digital piano, and a television; the only sign of life in the place was his house cat. Krizhevsky, the Orville Wright of the neural net, told the news crew he had walked away from the technology. “Maybe it’s just my personality,” he said, “but when I spend a very long time specializing in something, after about ten years I start to lose interest.”
― Stephen Witt, The Thinking Machine: Jensen Huang and Nvidia, the company shaping the future of AI

“The executives were more afraid of Jensen yelling at them than they were of wiping out the human race.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“Catanzaro was uncorked now—I sensed that he didn’t often get to share this perspective at his job. “It doesn’t need to inhabit this biosphere. In fact, it doesn’t need to be on the Earth, either, because the thing about artificial intelligence is that it travels at the speed of light. Humans, you know, we actually have to lug bodies around. Artificial intelligence can move along a radio signal as long as there’s an antenna on the other side.” Free of the limitations of biology, Catanzaro explained, AI would rapidly spread throughout the solar system and beyond. “Humans are naturally confrontational—like, we’re territorial animals, and it’s built into our limbic system to defend our turf,” he said. “AI, if it’s truly intelligent, the things that it’s interested in are so much bigger than the little thin crust of Earth that the humans live on. I don’t think that it’s going to be interested in taking that from us. Rather, I feel like AI is going to want to take care of us.” • • • Serving as a zoo animal for a space computer was an experience to be savored at leisure.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“The bias against neural nets, Hinton felt, was “ideological,” a word he pronounced in the same venomous tone that Huang had used to say “political.” The ideology of the research community at the time was that it was not enough that AI be useful. Instead, AI should somehow “unlock” the secrets of intelligence and encode them in math. The standard, 1,100-page AI textbook of the time was a survey of probabilistic reasoning, decision trees, and support-vector machines. The neural nets got just ten pages, with a brief discussion of backgammon up front. When Hinton’s colleague designed a neural net that outperformed state-of-the-art software for recognizing pedestrians, he couldn’t even get his paper admitted to a conference. “The reaction was well, that doesn’t count, because it doesn’t explain how the computation is done—it’s just not telling us anything,” Hinton said.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip

“But as Garlick acclimated to these shortcuts, he came to see the value of the Nvidia approach. “There was a bizarre brilliance to it all: just iterate, iterate, iterate, execute, execute, execute,” he said. “The way I see it now, tech debt is the battle scar of the survivor.”
― Stephen Witt, The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip