Rationality: From AI to Zombies

by Eliezer Yudkowsky

when we have a hidden motive for choosing the “best” current option, we have a hidden motive to stop, and choose, and reject consideration of any more options.