Literal Idioms

Somebody once accused me of taking idioms too literally.  I told him that wasn't a very nice thing to say, as the P.C. term is "Intelligently Handicapped."  I also pointed out to him that just because someone is Intelligently Handicapped doesn't mean that they're illiterate.  Some people are just so intolerant.