Micah Newman

37%
Flag icon
To be a safe fulfiller of a wish, a genie must share the same values that led you to make the wish. Otherwise the genie may not choose a path through time that leads to the destination you had in mind, or it may fail to exclude horrible side effects that would lead you to not even consider a plan in the first place. Wishes are leaky generalizations, derived from the huge but finite structure that is your entire morality; only by including this entire structure can you plug all the leaks. With a safe genie, wishing is superfluous. Just run the genie.
Micah Newman
AI
Rationality: From AI to Zombies
Rate this book
Clear rating
Open Preview