In Montezuma’s Revenge, you start in a room filled with obstacles. In each direction is another room, each with its own obstacles. There is no sign or clue as to which direction is the right way to go. The first reward is earned when you find your way to a hidden door in a faraway hidden room. This makes the game particularly hard for reinforcement learning systems: the first reward occurs so late in the game that there is no early nudging of what behavior should be reinforced or punished. And yet somehow, of course, humans beat this game.
