AI can also learn biases, errors, and falsehoods from the data it sees. Just out of pretraining, the AI also doesn’t necessarily produce the sorts of outcomes that people would expect in response to a prompt. And, potentially worse, it has no ethical boundaries and would be happy to give advice on how to embezzle money, commit murder, or stalk someone online.
