I have to admit that the Forgotten Realms is my least favorite campaign setting only because I don't have an opinion about Greyhawk. Now, my experience may be colored by the people who I gamed with at the time I was playing in the Realms, but I did not enjoy the excruciating level of detail that the setting has accumulated over the years. My first foray into Dungeons and Dragons was Game Mastering, and I hadn't yet developed the GM chops to shut down the irritating players in my group. Having to deal with the one Canon Lawyer picking apart every single plot and description made the game nothing but a chore.
I think the biggest issue that the later era Realms setting has going against it was that TSR decided that all novels became canon to the setting. It filled the setting with all powerful demi-gods that rendered most adventurers moot and filled in way too much of the map.