Human Compatible Quotes

“Before we can understand how to create intelligence, it helps to understand what it is. The answer is not to be found in IQ tests, or even in Turing tests, but in a simple relationship between what we perceive, what we want, and what we do. Roughly speaking, an entity is intelligent to the extent that what it does is likely to achieve what it wants, given what it has perceived.”
― Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control

“Uncertainty about objectives implies that machines will necessarily defer to humans: they will ask permission, they will accept correction, and they will allow themselves to be switched off.”
― Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control

“Machines are beneficial to the extent that their actions can be expected to achieve our objectives.”
― Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control

“Like any rational entity, the algorithm learns how to modify the state of its environment—in this case, the user’s mind—in order to maximize its own reward.”
― Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control

“Let me reemphasize a point made earlier: suitably designed machines will not behave like those they observe, even if those machines are learning about the preferences of sadistic demons.”
― Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control

“Alas, the human race is not a single, rational entity. It is composed of nasty, envy-driven, irrational, inconsistent, unstable, computationally limited, complex, evolving, heterogeneous entities.”
― Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control

“On a small island off the coast of Panama lives the pygmy three-toed sloth, which appears to be addicted to a Valium-like substance in its diet of red mangrove leaves and may be going extinct. Thus, it seems that an entire species can disappear if it finds an ecological niche where it can satisfy its reward system in a maladaptive way.”
― Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control

“Whereas a human can read and understand one book in a week, a machine could read and understand every book ever written—all 150 million of them—in a few hours. This requires a decent amount of processing power, but the books can be read largely in parallel, meaning that simply adding more chips allows the machine to scale up its reading process. By the same token, the machine can see everything at once through satellites, robots, and hundreds of millions of surveillance cameras; watch all the world’s TV broadcasts; and listen to all the world’s radio stations and phone conversations. Very quickly it would gain a far more detailed and accurate understanding of the world and its inhabitants than any human could possibly hope to acquire.”
― Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control

“Typically, such algorithms are designed to maximize click-through, that is, the probability that the user clicks on presented items. The solution is simply to present items that the user likes to click on, right? Wrong. The solution is to change the user’s preferences so that they become more predictable. A more predictable user can be fed items that they are likely to click on, thereby generating more revenue. People with more extreme political views tend to be more”
― Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control