To get a sense of one petaFLOP, imagine a billion people each holding a million calculators, doing a complex multiplication, and hitting “equals” at the same time. I find this extraordinary. Not long ago, language models struggled to produce coherent sentences. This is far, far beyond Moore’s law or indeed any other technology trajectory I can think of. No wonder capabilities are growing.
