complex goals. Since we humans don’t always have the resources to figure out the truly optimal replication strategy, we’ve evolved useful rules of thumb that guide our decisions: feelings such as hunger, thirst, pain, lust and compassion. We therefore no longer have a simple goal such as replication; when our feelings conflict with the goal of our genes, we obey our feelings, as by using birth control. We’re building increasingly intelligent machines to help us accomplish our goals. Insofar as we build such machines to exhibit goal-oriented behavior, we strive to align the machine goals with
...more
