Brave New Words: How AI Will Revolutionize Education (and Why That's a Good Thing)

by Salman Khan

How good these systems can get is usually related to the complexity and architecture of the underlying “model.” Think of a model as a computational representation that tries to mimic or simulate something in the real world. For example, when meteorologists try to predict the path of a hurricane, they use weather models that contain a software representation of billions or trillions of smaller volumes of the atmosphere and forecast how those smaller volumes would likely interact with one another. In the case of large language models, they are specifically designed to model associations between words. In this case we’re modeling not atmospheric conditions but neurons and synapses. Large language models such as GPT-4, short for Generative Pre-trained Transformer, are essentially big, powerful—albeit digital—“word brains” trained on a colossal amount of information from books, articles, websites, and all sorts of written material. By analyzing and processing this vast amount of text, the language model learns the patterns, the language, and the context of how words, sentences, and paragraphs fit together. If you were to ask a large language model like GPT-4 a question, it would know what to reply based on its training from all those books, web pages, video transcripts, and social media posts. What it lacks in real-world sensory experiences of the human brain, it compensates for by having exposure to more language than any human might hope to read, watch, or listen to in multip...