researchers began applying Sutton’s temporal difference learning to all kinds of different games. And one by one, games that had previously been “unsolvable” were successfully beaten by these algorithms; TD learning algorithms eventually surpassed human-level performance in video games like Pinball, Star Gunner, Robotank, Road Runner, Pong, and Space Invaders. And yet there was one Atari game that was perplexingly out of reach: Montezuma’s Revenge.
