The bare facts are these: (1) ever since Google’s landmark transformer paper, AI has been progressing at a much faster rate than nearly anybody save for Yudkowsky expected; (2) Silicon Valley is flooring the accelerator—Altman reportedly wants to raise $7 trillion for new facilities to manufacture semiconductor chips; (3) and yet, the world’s leading AI researchers don’t even understand very much about how any of this works. It is not only rational to have some fear about this; it would be irresponsible not to have some.
