compute used to train the best AI models has increased by nine orders of magnitude—going from two petaFLOPs to ten billion petaFLOPs. To get a sense of one petaFLOP, imagine a billion people each holding a million calculators, doing a complex multiplication, and hitting “equals” at the same time. I find this extraordinary.
