On the Edge: The Art of Risking Everything

by Nate Silver

are usually more tokens than words in any given sentence. (For instance, punctuation marks are tokens, and compound words like “snowboard” might be broken into multiple tokens.) When you ask ChatGPT a question, its transformer encodes each token into vector space. Vector space is like a graph with two or more dimensions. For instance, one way to encode “Paris” is with the coordinates 48.9, −2.4. This represents its longitude and latitude.[*25] However, there are dozens of other attributes you could describe for Paris. Paris might rate highly on the “fashionability” axis but poorly on “friendliness” (i.e., it is stereotypically snobby). Imagine you overhear this conversation between two friends: Alice: How was that new French place you tried? The one on Fifth Avenue? Bob: Well, darling. It wasn’t Paris, exactly. But it’s not half bad. “Paris” is thick with meaning here. Bob is saying the restaurant wasn’t particularly sophisticated or authentic, but was unpretentious and had reasonably good food. He could say that outright, but invoking Paris is more artful. Say you fed this conversation to ChatGPT and asked it to continue the dialog. In comparing itself to a symphony orchestra, ChatGPT imagines that each musician receives a set of instructions from a conductor[*26] analogous to a series of tokens from the chat window. Some tokens (“Fifth” “Avenue”) are straightforward, like receiving sheet music, while others (“darling”) are more ambiguous and require more context. The mus...