A smart automaton would blend in. It would observe those around it, mimic their behavior, act just like everyone else. All the while completely unaware of what it was doing. Unaware even of its own existence.” “Why would it bother? What would motivate it?” “As long as you pull your hand away from an open flame, who cares whether you do it because it hurts or because some feedback algorithm says Withdraw if heat flux exceeds critical T? Natural selection doesn’t care about motives. If impersonating something increases fitness, then nature will select good impersonators over bad ones.
