Thrun explained that AI systems could act in unpredictable ways to achieve their “fitness function,” or goal. If an AI was designed with a fitness function to survive and reproduce, it might inadvertently wipe out all biological life on Earth, Thrun said. This didn’t mean the AI was bad. It was just unaware of the gravity of what it was doing. Its motives weren’t all that different from ours when we washed our hands. We didn’t hate the bacteria on our skin and want to destroy it. We just wanted clean hands.
