Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI

by Karen Hao

Q* had not factored into the board’s decision. But, as Pachocki alluded to in his talk, Research was indeed treating the new algorithm with intense importance. The algorithm had been a brainchild of Sutskever’s, rooted in research he had been developing since 2021 to advance OpenAI’s models without a need for more data. The idea was to get a deep learning model to make better use of its existing data by using more compute at inference time to deliver better results. It broke the logic of the original scaling laws, which tied together a model’s performance with three inputs used during training: data, parameters, and compute. With this new method, Sutskever hoped to improve a model’s performance by further scaling the one ingredient he had always believed to be the most important: compute—but inference compute rather than training compute; in other words, the compute used to generate the model’s responses.