An analysis: my ratings vs. Goodreads ratings

[YouKneeK]

I think it’s useful to see 1 star ratings if they’re accompanied by a review explaining that the reviewer couldn’t finish the book. Much like other people have already said, I’ll typically skim through the reviews intentionally looking for 1 star reviews to see if I can get a consensus about what people disliked. If a lot of people couldn’t finish the book, I can make a more informed decision as to whether the premise interests me enough to risk suffering through a slow or unpleasant beginning.

Some people don’t bother writing reviews at all, but I think those people should still have the right to indicate they disliked the book by giving a low rating even if they didn’t finish it. If somebody offers me a tootsie pop and I discover that it tastes like dirt, I shouldn’t have to endure sucking on dirt until I get to the yummy tootsie roll in the center before I can say the tootsie pop is horrible. It’s true that a book can have a bad beginning but eventually improve enough that those who persevered are glad they did. But not everybody is willing to eat dirt to get to the tootsie roll and, for them, the taste of dirt deserves a one star rating no matter how good things might get later.

A good author should be able to find a way to keep the reader interested in the beginning. Just because a book doesn’t have likeable characters in the beginning, to use an earlier poster’s example, doesn’t mean the beginning can’t be “good”. There are many other things that can capture my interest in the beginning and make me want to keep reading: an interesting plot, an intriguing question I want to find the answer to, a cool world, enjoyable humor, etc. Likeable characters are high on the list of what’s important to me, and it will probably affect my over-all rating if I didn’t like the characters at all, but it won’t prevent me from reading the entire book if the book offers other things that hold my interest.

Disclaimer: I of course don’t mean to say that I think everybody should rate books they didn’t finish if they don’t want to. I only mean to offer my perspective on why it can be appropriate to rate an unfinished book with one star if a person so chooses.

[Neal]

colleen the contrarian ± (... never stop fighting) ± wrote: "Out of curiosity - what defines a "fair shot"?"

For me it's 100 pages. That's my point of no return. If I turn to page 101 I'm in until the end, even if it turns south. It may take me 6 months to finish it though. =)

[Eliene]

For me a fair shot is when I read at least 40% of the way through. If I drop it after that point, I definitely rate it 1-2. Sometimes I even give a 3 for a dropped book because I know I probably would have liked it and finished it if I'd read it at a different point in my life... A bit odd, but I can't really explain it.
On rare occasions, I rate books low even before I hit the 40% mark but only if happen hate every page outright.

I like reading 1 and 2 reviews. It really helps me decide if I want to pick up a book. I try to look at at least 2 rave reviews and 2 bad reviews if I'm on the fence about a book.

[colleen the convivial curmudgeon]

Personally, I don't think I have a hard limit on what I consider a fair shot. Probably more than 30%, but it depends on the book.

***

I wanted to go back to an earlier comment/discussion thread, because a book I was read today made me think of it:

Michel wrote: "This discussion leaves me wondering what proportion of readers promote the interest/enjoyment of a book's plot over the quality of the writing itself. I have read books that were enjoyable/entertaining despite having what some would qualify as poor grammar or syntax, while other books by masters of the English language bored me to tears. So, how many readers rate a book solely on how interesting/enjoyable/entertaining it was? Does 'bad' mean for some 'boring', while 'amazing' would describe the degree to which the reader enjoyed the book, despite any potential flaws in the written English? If it is so, it could explain why there are such rating disparities between readers of the same book. "


For me, I will definitely say that I rate based on entertainment: bad = boring.

But I'm not sure I can really separate the writing itseld from the enjoyment factor.

I don't think it's my highest criteria, because I've read beautifully lush books that I had to 2-star for lack of characters I cared about or a plot...

But I also think that good writing can elevate an otherwise pretty standard story for a 3 to a 4.

But I don't just mean correct grammar and things like that. Obviously bad grammar and people who can't write a sentence, let alone a paragraph, are automatically bad...

But, for me, it's more the tone and texture of the writing - and that is, of course, entirely subjective, but it's something which definitely matters to me.

If the voice of a story works for me, then it tends to be a book that's easier for me to fall into. It can't, alone, outweight the importance of character or plot... but if I get a story I like writtin in a voice that resonates... well, those are the books/stories that linger.

[Kateb]

colleen the contrarian ± (... never stop fighting) ± wrote: "Kateb wrote: "I find that if a read more than a 1/4 of the book I have to find out what happens , even if I dont want to read the book. I will flick through to find out the basic plot . My friends ..."

What a great idea, that is to put them aside for a while, I will start that now!!! Just had my nephew clean out his study and i have been given 100 new books. We will see how that goes

[Neal]

More data crunching since the weekend is upon us.

I exported the 890 books I have on my shelves (again, mostly SF&F, and because it's convenient to export) and plotted a histogram. Keep in mind that my purpose of this thread isn't to try and dictate how people should/shouldn't choose to rate their books, but just understand what the ratings mean on GR when I look at them since they seem so skewed.


Nice. Pretty normal, albeit tight, distribution with a few outliers.

x-axis = average GR rating
y-axis = frequency, 0.1 rating bins

Rating Avg = 3.84
Rating St Dev = 0.29
These numbers are slightly different than the first post as I've added a handful of books to my shelves since that post, so the data shifted around slightly.



Nearly all the books are receiving 3 or 4 star ratings, with a few 5 stars tossed in to pull the mean + 3sigma over 4.0.

So, in general, GR effectively uses a thumbs up (4 or occasional 5 star) and thumbs down (3 star) rating system. :-p

Or, any book rated over ~3.84 is considered better than average by readers, rated below ~3.84 is considered worse than average by readers.

Anything rated over 4.13 (+1 sigma) should probably be considered a GOOD book, anything rated over 4.42 (+2 sigma) is rare and should be considered an AMAZING book, or has a low number of votes.

Anything under 3.55 (-1 sigma) should probably be considered a BAD book, anything under 3.26 is rare and should be considered an TERRIBLE book, or has a low number of votes.

In general, per your genre tastes, yadda, yadda, disclaimer, etc...

It would be interesting to get access to all the GR data, HUGE sample size, what the distributions look like that make up each book average, what different genre averages look like, etc. But I don't see that happening. /nerd

[Cheryl]

If a book has a *lot* of votes (at least a couple of hundred) and the community rating is under 3.3, I'm more suspicious of rave reviews.

But I will repeat myself. I look at reviews, especially those written by friends and those I follow. I don't consider ratings relevant.

Different people like different books. You seem to be willing to follow the herd. I'm not.

[Kateb]

Gail wrote: "Kateb wrote: "colleen the contrarian ± (... never stop fighting) ± wrote: "Kateb wrote: "I find that if a read more than a 1/4 of the book I have to find out what happens , even if I dont want to ..."
so far very teenagey, will work my way through them. After all I introduced him to sci fi and fantasy years ago

[V.W.]

I only rate books I liked, since I don't rate unless I'm prepared to write a review saying why I liked it. So all my ratings are 4 or 5 star.

In general I never go past the first page of a book if what I'm reading doesn't appeal to me. I see no point in rating these.

Some are enjoyable at the start but go seriously wrong a bit further in, but most of these are "wrong" because of things like improper use of weaponry or combat technique, poor history, (carelessly) rubbish science, and so forth. There are readers who don't care about such things and would enjoy the book anyway, so I don't rate these either.

[Robin]

colleen the contrarian ± (... never stop fighting) ± wrote: "Personally, I don't think I have a hard limit on what I consider a fair shot. Probably more than 30%, but it depends on the book.

***

I wanted to go back to an earlier comment/discussion thread,..."

I totally agree with this. An example of a story with an awesome tone and characters at least for me is Jim Butcher's Dresden Files series. Without Harry's distinctive and loveable character voice the story would just be a typical urban fantasy series. But Harry the protagonist makes it very special.

[Neal]

Cheryl wrote: "You seem to be willing to follow the herd. I'm not. "

Not really. From my posts here I thought it would be clear that I don't trust the GR rating data. I feel it's biased, skewed, and generally a misleading rating system. /shrug

[Tommy]

Wouldn't trusting ratings and/or reviews be following the herd though? If you'll read a book based on what a grouping of people you deem reliable--let's call them your herd--would that not be following them?

Heck, wouldn't joining the millions of readers on Goodreads be following the herd in a larger sense?

I just like what I like. I pay no mind to ratings or reviews, aside from those by people who I frequently speak to. And even then, they'll rave about books that sound uninteresting to me at times and I'll pass, and other times talk about how much they hated a book, and though their hatred is the first thing I ever hear about the book, it may interest me and I may love it(this has happened dozens upon dozens of times). Even then, while my opinions are unaffected by them, I'm reading books based on the opinions and reading lists of others, so am I not following a herd?
It just feels less insulting than it sounds to me, personally, but our need to believe we're all lone wolves makes it sting.

Either way, while I don't fully agree with the topic, I don't see the need to attempt to insult each other for differing opinions.

[Trike]

Neal wrote: "Cheryl wrote: "You seem to be willing to follow the herd. I'm not. "

Not really. From my posts here I thought it would be clear that I don't trust the GR rating data. I feel it's biased, skewed, and generally a misleading rating system.."

"You forgot stupid, lazy..." -- The Breakfast Club ;)

[Tommy]

Trike wrote: "Neal wrote: "Cheryl wrote: "You seem to be willing to follow the herd. I'm not. "

Not really. From my posts here I thought it would be clear that I don't trust the GR rating data. I feel it's bi..."

Dang it, that was so much better than what I said haha.

[Cheryl]

Neal wrote: "Cheryl wrote: "You seem to be willing to follow the herd. I'm not. "

Not really. From my posts here I thought it would be clear that I don't trust the GR rating data. I feel it's biased, skewed,..."


But you said earlier:
"
Anything rated over 4.13 (+1 sigma) should probably be considered a GOOD book, anything rated over 4.42 (+2 sigma) is rare and should be considered an AMAZING book, or has a low number of votes.

Anything under 3.55 (-1 sigma) should probably be considered a BAD book, anything under 3.26 is rare and should be considered an TERRIBLE book, or has a low number of votes."

So, you didn't mean that??

[James]

I rate Cheryl's response a 4.43

[Neal]

Cheryl wrote: "So, you didn't mean that?? "


I think you're reading far too much into what I've posted. =)

My point is the data indicates people SHOULD take these ratings with a large grain of salt and not follow the "herd."

[Cheryl]

:throws hands in the air:
unsubscribing from thread

[Aaron]

Quality of a rating also has to do with how effective it was at hittings it target audience. See how getting nominated for a Hugo can plummet your rating because while it might of had enough people in it's target audience who liked it...the whole Hugo's community was not part of it's target audience and it will get low ratings because of that.

Different genre's also tend to get different ratings...seriously point me to a Sci-fi book with over 1000 reviews that has broken a 4.5 I don't have any on my list some of the Vorkosigan books come close but that's about it. Meanwhile in fantasy you have stuff like Words of Radiance with a 4.76/5 rating.

[Alan]

Neal wrote: "More data crunching since the weekend is upon us...."
Nice analysis! The trouble is that a statistical analysis is only as good as the data it is based on. Now, at risk of repeating (in slightly different form) what other people have already written, here is my position . . .
I am unlikely to publish a review of something I have not read to the end. I am also unlikely to reach the end of something for which I would do a 1 or 2 start review.
Those rules are not absolute - I have given a couple of 2 stars - but they are a guide to the way in which the figures could become distorted.
I could smooth out this bias by rating without reviewing - or by writing a review that just says "Didn't (couldn't!) finish this crap" . . . but to what extent should I consider myself obliged to do so?
In the mean time, I suspect many people take a similar position - so the statistics get distorted. It doesn't mean that "Anything under 3.55 (-1 sigma) should probably be considered a BAD book, anything under 3.26 is rare and should be considered an TERRIBLE book" - it means that enough people have felt good enough about it to give it a 3 or 4, but we don't know how many people reckoned it so bad they did not finish it.

As always, statistics need careful interpretation - and anyway, who knows whether or not your tastes in literature will resemble mine? Individual opinions are a good guide when you know the tastes of the reviewer - otherwise these figures all need taking with a pinch of salt!

[James]

Aaron wrote: "seriously point me to a Sci-fi book with over 1000 reviews that has broken a 4.5"

That's an arbitrary number. I can point to lots that are over 4.

[Bill]

Interesting Neal. I find it more useful to choose books based on reviews than numerical ratings, especially those from people i know. I only read books I have bought, usually hardcovers if one is available, and I don't buy books unless I'm fairly sure I will enjoy them. Libraries are few, small and distant from where I live. My GR avg is 4.13.

I personally rate on a 10 point scale and I suspect my avg is close to a 7 or 3.5. I usually give 7s a 4 here. I also rate on how badly I want to re-read a book rather than how much I liked it. I've found some of the most worthwhile books require re-reading, but may not be the most likable or enjoyable. Dhalgren, eg, is a tough read. I've only read it once and need and want to read it a few more times. It's not the most fun book I've read, but still a favorite.

[Aaron]

Kernos wrote: "Interesting Neal. I find it more useful to choose books based on reviews than numerical ratings, especially those from people i know. I only read books I have bought, usually hardcovers if one is a..."

FYI assuming that every single one of your 5s is a 9/10... 4s are 7/10 and so on and so forth. You would have an average rating of 7.25.

If you took the middle ground of 9.5/10=5 and 7.5/10=4. You would have a 7.75...assuming a direct translation from the 10 point to the 5 point system which I feel isn't a good idea because most 10 point scales on put average at like 7/10, I would bet you would have a 7.6-7.7.