Builders of the World: UNITE!

So, if you pay attention to this blog at all, you've probably recognized that I, and a lot of people who talk on this blog, do a lot of bashing on worldbuilding. For what we assume to be good reason.

The argument is pretty easy to see if you peruse this blog: worldbuilding adds unnecessary fluff that deviates from plot, character development and overall deadens the sense of wonder by way of murdering with detail.

That's the argument.  I'm not sure it's right, but that's the one I've been sticking with.

It occurs to me, though, that we don't see a lot of talk about it from the other side: those who praise worldbuilding, who drool over magic systems and gush about the language of the birds and all the subtle nuances of the tone and touch of a Wombatman.  Perhaps it's that I'm simply missing something.  Maybe there's some glory to worldbuilding that I'm just not clear on?  Maybe it adds in a way I hadn't considered before?

This is my humble request, if you are indeed one of those people who clings to the notion: what is it?  What am I missing?  Am I using the entirely wrong definition of worldbuilding?  Am I just too dense to grasp it?  In the interests of discussing this and furthering everyone's collective knowledge, I'd genuinely like to know.

So tell me.  Either here, in email, whatever you want.  We should know this.