You Look Like a Thing and I Love You: How Artificial Intelligence Works and Why It's Making the World a Weirder Place

by Janelle Shane

As we’ve seen, there are many ways to accidentally sabotage an AI by giving it faulty or inadequate data. But there’s another kind of AI failure in which we discover that they’ve succeeded in doing what we asked, but what we asked them to do isn’t what we actually wanted them to do.

As a species, we suck at specifying what we want. This is a professional opinion. We are doomed.

1. They develop their own ways of solving a problem rather than relying on step-by-step instructions from a programmer. 2. They lack the contextual knowledge to understand when their solutions are not what humans would have preferred.

the programmer still has to make sure the AI has actually solved the correct problem.

Again, the primary requirement of a consumer or buyer of anything is to confirm just what you got relative to what you asked for.

“I’ve taken to imagining [AI] as a demon that’s deliberately misinterpreting your reward and actively searching for the laziest possible local optima. It’s a bit ridiculous, but I’ve found it’s actually a productive mindset to have,” writes Alex Irpan, an AI researcher at Google.

An increased tension to a very old view of computer programming. The computer will do exactly what you told it to do with the worst possible result.