Children's book I read around 2001-04. Fairytale compilation but I only really remember Rumpelstiltskin

[Chandler]

It seemed old to me, so I would say it definitely was published pre 2000. The most distinguishing characteristic was the physical book. It was long and skinny, not the shape of a normal book (think more long, skinny, vertical rectangle rather than square). It was clothbound with a red spine and I think the cover was tan.
Some more hazy memories: My mom read it to me, so I'm uncertain if it had illustrations but it certainly wasn't a picture book. But it wasn't a ton of pages either.
Random assumption: It was in the children's section of my library, low enough on the shelf that I was able to grab it, indicating to me that its author was likely towards the middle of the alphabet

[Ellie]

It might be Rumpelstiltskin by Paul O Zelinsky. The book cover matches your description and it’s only 40 pages. It was published in 1996.