Guilt at NOT Finishing Reading a Book?

Until now I've always finished reading even mediocre books, conscious of the amount of work any writer puts into their project. But recently I've become aware that I can't read everything, and that I'd rather read books which are well written and/or give unusual insights.

Has anyone else made the decision NOT to finish a book, and still felt a bit guilty about that? Even if it is an Emperor's New Clothes moment, when it's a best seller and having read the book, you can't work out why it has been acclaimed and wonder if others just read the media release?
That's why I appreciate honest reviews with reasons by other readers and then there's a thrill of finding a genuinely interesting but lesser known book.