Most AIs, though, train in a phase distinct from the operational phase: their learned models—the parameters of their neural networks—are static when they exit training. Because an AI’s evolution halts after training, humans can assess its capacities without fear that it will develop unexpected, undesired behaviors after it completes its tests. In other words, when the algorithm is fixed, a self-driving car trained to stop at red lights cannot suddenly “decide” to start running them. This property makes comprehensive testing and certifications possible—
