Rationality: From AI to Zombies

by Eliezer Yudkowsky

When “no one can prove to you” that your precious idea isn’t right, it means you don’t have enough information to strike a small target in a vast answer space. Until you know your idea will work, it won’t.