The Sword and Laser discussion
Are 5 star rating systems useless?
date
newest »
newest »
Mark wrote: "We can just adapt it. Like this http://xkcd.com/1098/"Yeah, that pretty much sums it up.
I think it's also helpful to recognize the difference between an Amazon star rating and one on Goodreads. Goodreads star ratings are exclusively intended for books, whereas Amazon star ratings are meant to cover a multitude of retail items. Someone's experience with an Amazon product might actually have been harmful, whereas the worst you can (usually) do with a book is maybe drop it on your toe (unless--like that one time special edition of Ray Bradbury's Fahrenheit 451--it comes with an asbestos cover). Which explains why Goodreads star ratings tend to be skewed in one direction, whereas Amazon's are skewed in the other.Here's what a given star rating means on Amazon:
1 star - I hate it
2 stars - I don't like it
3 stars - It's OK
4 stars - I like it
5 stars - I love it
Here's what a given star rating means on Goodreads:
1 star – didn’t like it
2 stars – it was OK
3 stars – liked it
4 stars – really liked it
5 stars – it was amazing
I've seen some reviewers who've taken this into account when transcribing reviews between the two services, bumping a Goodreads rating up a star (when appropriate) when forwarding said rating to Amazon.
Which might explain (at least in part) why Sean is getting a 3.9 star average. Books readers hate probably don't get finished more often than not, but a certain percentage of those readers may be resistant to rating a book they haven't finished, because even though they didn't like what they did read, they may feel it's unfair to rate a book they never actually completed. So the overwhelming number of ratings may be falling between 2 and 5 stars, which would average out to be 3.5 stars.
No statistics to cite on the above, just seems a reasonable assumption (at least to me, YMMV).
Walter wrote: "I think it's also helpful to recognize the difference between an Amazon star rating and one on Goodreads. Goodreads star ratings are exclusively intended for books, whereas Amazon star ratings are ..."Amazon (Kindle Store) ratings are based on very small numbers of reviews, even for extremely popular titles. That makes them meaningless, statistically speaking. The titles I chose all had 100 or more votes for this reason. Some had hundreds of thousands.
What I'm saying is that there are no books rated 1 or 2 stars on Goodreads, unlike movies, for example. It is the average that people see, and that is being skewed into a narrow range.
I find the S&L discussions much more useful than the ratings or even reviews. I was very bummed when I rated enough books to qualify for Recommendations and I'd already read or heard of them all. The stars are pretty much meaningless to me at this point. I generally hear about a book, look it up to see what it's about and glance over a few non-spoilery reviews. If the reviews are decent and not 5 stars from the author's mom/beloved one or a 1 star hater from the opposite end of the political spectrum, I'll give the book a shot.
xkcd does pretty much sum it up. If I know nothing about a book, I look to see if it has 4 stars or more (with more than 100 reviews, to ensure it's a fairly good average). If it has more than four stars and the reviews tend to jive with what I like, I'll give the book a shot.
Rochelle wrote: "xkcd does pretty much sum it up. If I know nothing about a book, I look to see if it has 4 stars or more (with more than 100 reviews, to ensure it's a fairly good average). If it has more than fo..."
The perception is that a book with a 4+ rating is in the top 20% of books. The reality is that it's in the top 50%.
Should boil it down to this.[Read it]
[Maybe read it]
[Don't read it]
The end.
The flaw is in GoodReads' options. They're inherently skewed to try to make us give a neutral or better response by making 2* out of 5 to be what most people would normally give a 3* rating to and then giving 3 positive responses and only 1 negative response.Most people are inclined to give a 3* rating to something that illicits a mostly neutral response eg. "It's not bad/it's okay/it didn't make me want to claw my eyes out".
It's an excellent way of manipulating the data from responses because I'm willing to bet that most folk don't pay attention to what GoodReads label the star ratings, and so folks are rating things higher than they normally would.
I think you might be overstating the importance of the star rating average. Do you think most people really pick the books they read based on what its average score is? I think most are smart enough to recognize that they tend to fall in a very narrow range.I might look to see what an individual has rated a book we're talking about but almost never notice what the average is.
The goodreads rubric is not the problem, it's people applying their own values to the scale. A three star rating doesn't help a potential reader much if seventy-five percent of those who rated it use three stars when they hate a book.
Phil wrote: "I think you might be overstating the importance of the star rating average. Do you think most people really pick the books they read based on what its average score is? I think most are smart enough..."I'm sure there a small number of people out there who say, "I won't read a book unless it has 4 stars", but they are only hurting themselves. I think that it may influence purchasing decisions though. Someone might be more tempted to take a lower rated title out of the library than purchase it outright, for instance. Or perhaps, all other factors being equal, given a choice between two books, someone might choose the higher rated book. It's certainly not the only factor in people's decision making, but if it's having an influence and is based on faulty data, we should examine it.
But that was the question. If it's pointless, why use it, and is there a better system?
I think those that point out that people don't just look at the star rating are important points. Most read reviews, blurbs, and even may place higher value on the ratings and reviews of their GR "friends", especially those that consistently recommend books with similar tastes to yours.Additionally, people it makes sense that the books people read are rating higher. I like certain things and unless the author really screws something up I'm not going to give it a 1 star. There are other things when reading I avoid. I'm sure that if I were to read those books, many would receive 1/2 star ratings. But I select them out knowing its not my thing. I don't waste my time unless I reasonably feel that there is a really good chance I'll like a book (my time is precious). The 1/2 star reviews come from people who thought they'd like something, but it turned out not as they expected, and not in a good way.
Just remember that Goodreads ratings are not a measure of the objective quality of a book. And frankly, I'm not sure that there is a way to objectively measure the quality of a book given that a reader's engagement with it is such a personal thing, beholden to their tastes and other personal baggage. On the whole, I'd imagine that people tend not to branch out too much and mostly stick to reading books that they enjoy, which skews ratings towards the higher end of the scale.
I think that by proclaiming "the system is broken", perhaps what you really mean is, "the system doesn't do what I would like it to".
Anybody who truly uses GR to find stuff to read knows that the reviews and comments from other members is where the real information is, because you can see why someone gave a book a particular rating.
Alicja wrote: I think those that point out that people don't just look at the star rating are important points.I think this is a good point. Again, no statistics to go on here, but it seems to me that sales for books which prove to be popular amongst readers (or a particular subset of readers) tend to be influenced far more by word-of-mouth and/or reviews, rather than ratings.
Also seems to me that ratings on Goodreads exist in order to provide a feedback option for those folks who are uncomfortable writing reviews (or who don't feel they have time to compose a review). To the best of my recollection, Amazon doesn't allow only star ratings, you must include a minimum of a twenty word review along with your rating (unless things have changed recently).
As a side note on ratings, it also seems to me that the number of ratings plays a crucial part as well; at least, speaking anecdotally. If I see a five star average rating on a book with only a handful of ratings, I'm less likely to be impressed than if I see a four star average on a book when the actual number of ratings is in the thousands, or tens of thousands.
And Sean, please understand that my post wasn't intended to be a 'point/counterpoint' sort of thing. I simply intended a side comment of sorts to point out the differences between Amazon and Goodreads star ratings. Wasn't addressing your post specifically (except for my speculations on the star averaging thing).
I think the main problem boils down to this: When I give a book 3 stars, it means it was decent book that I enjoyed reading. (Which is also more or less what 3 stars is supposed to indicate according to GR itself). However when a book averages 3 stars on Goodreads, it means it's more often than not barely readable crap. As is the case everywhere with these sorts of online ratings (except on IMDb, probably) people are way to generous with their ratings. People, especially people who don't tend to take ratings as seriously as the people spending their time in discussion groups and threads like these :p, tend to just give everything they like a high rating, without any sort of nuance.
Another thing that skews the averages IMO, is that many people feel that they need to finish a book before they can rate it. Finishing a book which usually means a commitment of a fair amount of time. And people aren't all willing to spend that time on something they're not really enjoying. (unlike a movie which you can finish in 1.5 hours and usually is seen as a social event with other people which increases the likelyhood of people sitting it through till the end.) So a lot of "bad reads" go unrated. I don't agree with this. If a book is so poorly written/boring that I can't make it all the way through, it fully deserves the 1 star I'm giving it.
Was going to say about IMDb myself, but the opposite--that their ten-point scale doesn't do a lot better than the five-star system because it has such a long tail. People seem to rate between 7-10, unless they really hate something. In both cases it's a problem with the raters not using the whole scale correctly, but if that's the case the scale needs to be re-thought. But I think any system will be susceptible to the same issues.The stars on GR aren't as important because (afaik) you can't call up some mega list and rank it on stars. You CAN do that on Amazon, so ratings there are really vital for authors. I tend to be more critical in GR because it's a book review site, but wouldn't like to be too harsh on Amazon where it might be more damaging.
The problem with star ratings is that everyone has the own scale they measure against. Also as I found when I worked on project to add star ratings and reviews to airline and insurance purchases most people to give a 1 star rating and complain only a few gave 4 or 5 stars if they true loved the service.I think a simple liked, average, disliked system would be best with the number of people who voted each.
100 - liked this book
25 - thought it was average
50 - would not recommend it
Getting people to leave reviews is a whole other problem entirely. When I was looking at building a indie book site aimed at connecting readers and writers. I thought a karma system might encourage people to leave reviews. They would earn karma for sharing and reviewing books, which they can exchange for things. But the idea never got off the initial concept stage.
Alicja wrote: "I think those that point out that people don't just look at the star rating are important points. Most read reviews, blurbs, and even may place higher value on the ratings and reviews of their GR "..."I don't look at the star ratings when choosing a book. Almost everything on my to read list is stuff from authors I already know I like, or stuff I've seen a recommendation or a review for (from S&L, Scalzi's Big Idea guest posts).
Star ratings mean nothing to me when choosing a book. I know that my taste is different than others, and what they hate, I may love. I try to use Goodreads suggested tooltip criteria for giving ratings.
I tend to use the system for my own use and follow the Amazon rating system. 1 = the worst, 3 = average 5 = the best. I rate books honestly how I feel about them and get the expected distribution. So far this year I have rated 36 books.
1 star = 2
2 star = 10
3 star = 14
4 star = 8
5 star = 2
I generally pick up a book based on the premise. I'll look at the 1, 2, & 3 star reviews to see if folks are complaining about the kinds of things I would complain about. If not, I might just buy the book. The overall average rating for a book doesn't register unless it's extremely high or extremely low (then I think something's fishy).
I use other people's ratings in part to choose reviews to read. If I'm interested in a book on goodreads I read a couple of positive and a couple of negative reviews so the stars help me whiz by a bunch of repetitive 5-star reviews. Other than that, I use the star system to fine tune the recommendations made to me. If I hated a book and want to see less of its particular sub-genre in my recommendations list, I am more likely to rate the book than to just ignore it.
It should be obvious to anyone who spends time on Goodreads that the average ratings fall in a very narrow range, making them all but useless.You might remember that YouTube used to employ a star rating system for videos, but found it broken for similar reasons, and switched to a binary like/dislike rating system. Here's a blog post where they explain the shortcomings of star ratings:
http://youtube-global.blogspot.com.au...
Another example of a failed 5-star rating system, Yahoo! Sites:
http://buildingreputation.com/writing...
Apparently the tendency for aggregate ratings to clump around 4.5 is called a "J curve". People tend to only rate things that they like, which skews the average toward the upper end of the scale.
Also, I think many people are overly generous with 5 star ratings on Goodreads. A 5 star book ought to an absolute classic of its kind - something you would recommend without reservation and which stands above other books in its category. The YouTube and Yahoo! Sites examples suggest that for many people 5 stars simply means "I liked it".
I still think star ratings on Goodreads are useful, just not as a statistical average. Here's how I use star ratings to help me find books I will enjoy:
- Compare a selection of 5 star reviews with a selection of low star reviews. Use those reviews to get a sense of the common criticisms and compliments the book receives, and use that information to decide if the book appeals to me. This is the method described by Alan and Gregor in the posts directly above this one.
- Look at how my friends rate a book.
- Look at how my favourite reviewers rate a book. For those reviewers I will often ignore what anyone else has to say about the book, and trust their rating.
I find star ratings on Amazon more useful, perhaps because only someone who writes a review can rate a book, and as others have mentioned in this thread, reviews are far more helpful than ratings. I wonder if the average Goodreads rating would be different if only ratings accompanied by a review were taken into account?
An aside: The average of my 286 Goodreads ratings is 3.99, which is higher than the 3.9 average that Sean calculated in this thread's first post. I suppose that makes me just another example of how broken the system is!
To satisfy my own curiosity I did an experiment to see if the average ratings differ when we only consider ratings attached to a review. My theory was that someone who takes the time to write a review has considered a book's merits and weaknesses more carefully than someone who simply rates a book, and that reviewers' ratings are therefore more meaningful.I analysed 3 books: one that is widely considered a classic (Lord of the Rings), one of debatable quality (Twilight), and one chosen at random (The Light Between Oceans). I looked at the first 5 pages of reviews, ordered by newness.
LotR review average: 4.4 (higher than the overall rating of 4.29)
Twilight review average: 3.14 (lower than the overall rating of 3.56)
The Light Between Oceans review average: 3.93 (lower than the overall rating of 3.97)
I don't know if any conclusions can be drawn from this small data set, except perhaps that Twilight is a worse book than it's already low Goodreads rating would suggest! (ducks for cover) In any case, there does seem to be a small variation between reviewers' ratings and overall ratings, but probably not enough to be statistically meaningful.
Lots of people look only at stars. To that end, it's helpful for people to use them in the same way. But people don't.I personally like what Goodreads does with stars and use that when I'm writing reviews here. I mostly don't write reviews anywhere else. That said, I hope that people read my actual reviews if they're looking for an opinion on a book. The star ratings are there whether people use them or not; I use the ones I give as a reminder to myself at a glance what I thought of the book.
Personally, I don't look to generic user reviews or star ratings to determine if a book is something I might want to read. I look for reviews from users I trust and discussions like ones in this group to steer me towards a book (or away from one). Generic user reviews are usually too vague or otherwise focus on things that shouldn't be in a review. But if I know of a user and know that their book interests generally align with mine, I'll take their opinion with much more credibility.
Sorry this is rambling trying to multitask and failing. :)
terpkristin wrote: "The star ratings are there whether people use them or not; I use the ones I give as a reminder to myself at a glance what I thought of the book."That's what I use them for as well.
The problem with the average rating system isn't the skew toward the higher numbers, or even the non-comparability of ratings by different people using their own scales. The big problem is that different numbers of people have rated different things. This makes comparing ratings between books meaningless. In particular, you'll find that the first book in a series almost always has a lower rating than later books, and the most famous book by an author has a lower rating than his lesser-known books, even if it's widely acknowledged as his masterpiece. This is because what decides the average rating of a book comes down to how many people who aren't going to like the book have been persuaded to read the book. Famous books, and first books in series, have lower averages because lots of people who aren't going to like them have bee convinced to read them; little-known books, and book 10 in a series, have higher ratings because the only people who read them are those who are sure to like them. I call this the Bananafish Problem... so-called because JD Salinger's masterpiece, "The Catcher in the Rye" has a goodreads average of 3.77, whereas his rarely-heard-of short story "A Perfect Day for Bananafish" has an average of 4.35. The utter incomparability of these scores can be seen when you look at how many have rated them: Bananafish has 3.5k, whereas Catcher has 1.4M - three orders of magnitude higher!
It's probably not the case with Bananafish, but the real nonsense of this sort of comparison comes when everyone prefers one book to another. Suppose that EVERY SINGLE PERSON who read both Catcher and Bananafish preferred Catcher. This would still have no impact on the average scores, which would continue to rate Catcher higher, since the Bananafish readers would simply be drowned out. There's something wrong in a system of recommendation where every single person who tries two things prefers A to B, yet B ends up recommended over A...
(another example: 'Thud!' has almost exactly the same average rating as 'Small Gods'. I have never met anybody who thinks 'Thud!' is a better book than 'Small Gods'. But Small Gods has twice as many reviews as Thud!, so more of the reviewers aren't hardcore Pratchett fans... (mind you, that doesn't explain why 'Going Postal' has a higher rating than 'Small Gods', but a similar number of raters. That's just an abomination. But that's because although they're the same number, they're numbers of different things - the people who rated Small Gods are a different crowd altogether from the ones who rated Going Postal))
However, stars are still useful. You can always look at people you know and see how they rated things, with some sense in mind of how they tend to rate things. It's just not very useful when you aggregate large numbers of ratings and compare between books with different audiences.
I don't think that using books which are ludicrously famous/notorious such as Catcher in the Rye is a good basis for your argument. A book which is so well known, so widely read that it is achieving rating entropy is not one readers will check the star rating of before deciding to read.
I discussed this with a group once and someone brought up the point that just by reading a blurb on the back, taking recommendations, reading books specifically in genres you enjoy, etc., you naturally weed out books you would consider 1 or 2 stars and never pick them up to read. So most books you would read would be in the average to pretty good range, hence why averages would be closer to 3.5-4.
Darren wrote: "I don't think that using books which are ludicrously famous/notorious such as Catcher in the Rye is a good basis for your argument. A book which is so well known, so widely read that it is achievin..."That's not the point - the problem isn't Catcher-specific. It's that averages aren't comparable when they're averages of different sizes of sample. Taking a less famous example: RA Salvatore's Drizzt novels. The original one, 'The Crystal Shard', has a rating of 4.11. Later books have ratings that go up and down, but by the time you get near the end, it seems that the books just get better and better: #24 has a rating of 4.35. But it also has only 1800 ratings, compared to The crystal shard's 22600. So do the books get better, or is it just that only hardcore fans are still reading? Indeed, there's a perverse effect potentially going on here: if a series gets worse as it goes on, many readers will stop reading following books, so that the readership becomes distilled into a more and more hardcore group of fans - so the closer a book is to being so shit that almost everyone gives up on the series from that point on, the higher the average ratings for the following books!
Wastrel wrote: "Darren wrote: "I don't think that using books which are ludicrously famous/notorious such as Catcher in the Rye is a good basis for your argument. A book which is so well known, so widely read that..."The thing about a series is that I don't think most people look to reviews for later books in the series to determine if they'll continue to read. If I loved a book, then I'm reading the next one even if the rating is low. If I'm not sure whether to read it or not, then I'd want to look at reviews and discussions about why people loved or hated it. Something that turns off some people may leave me drooling with love. I think the rating system has its limitations. I also don't think that most GR users look at only the stars. They have their limited uses and that's great.
Also, I wanted to point out that GR isn't just a rating site but also a book-related social networking site. The stars aren't just meant for others, but they are also to catalogue one's like/dislike of a book in their personal profile. They are also there to inform you how you felt about a particular book at a particular point in time. Some people have more in-depth thoughts and others are fine with just the star system. And as a social-networking site, we share not only reviews but also our star ratings with reading buddies. So the purpose of this site is also different to that of amazon, which focuses in selling products.
For science fiction we should have a short book list for readers to rate. Then have a function built into the site that finds readers who gave similar ratings to those books. Then the user could find books with high ratings given by people with similar tastes.Like I gave a low rating to Hitch Hiker's Guide to the Galaxy even though it has a high rating from most readers. My tastes might be quite different from someone who gave it a high rating. So other people who gave it a low rating might have tastes similar to mine so I would be interested in what they gave a high rating.
Wastrel wrote: " Indeed, there's a perverse effect potentially going on here: if a series gets worse as it goes on, many readers will stop reading following books, so that the readership becomes distilled into a more and more hardcore group of fans - so the closer a book is to being so shit that almost everyone gives up on the series from that point on, the higher the average ratings for the following books! "Maybe sometimes. Here's a pretty famous counter to that.
Casino Royale Bond #1, 22K+ Ratings (3.7 stars)
Octopussy & the Living Daylights Bond #14 2.8K ratings (3.4 stars)
I guess the number of reviews is almost as important as the stars it gets. Mostly I pick my books by friends and blogs suggestions, and that rules out most of the poor books. I don't hesitate to give a 1-star to a book I dislike, but the selection method rules out most of them. I save 5-stars for very special books, but most of my rates are indeed 4, with equal amounts of 3's and 5's after that.
Janet wrote: "I discussed this with a group once and someone brought up the point that just by reading a blurb on the back, taking recommendations, reading books specifically in genres you enjoy, etc., you natur..."This is definitely my experience -- I've given out very, very few (if any) 1- or 2-star ratings simply because I self-select those books right out of my queue before they get read, at least in most cases.
The other thing, at least for me, is that rating/reviewing a book seems much more ... personal? than rating/reviewing a movie. Maybe because for a book I know it's primarily the work of a single individual (the author). Both Netflix and Goodreads use 5-star scales, but I think if you compare my average Netflix rating to my average Goodreads rating, Netflix would skew quite a bit lower. Possibly because, in part, I don't expect Seth Rogen to actually be checking his Netflix ratings obsessively and be hurt by my 1-star on Green Hornet or whatever.
Joseph wrote: "Both Netflix and Goodreads use 5-star scales, but I think if you compare my average Netflix rating to my average Goodreads rating, Netflix would skew quite a bit lower."I think one difference between movies and films is that because films only take a few hours to watch, I am more likely to take a risk on a movie. This means I'm going to watch a lot more bad movies than I will read bad books.
When selecting a book I will spend a longer time vetting a title, since reading a book is a big time commitment, and I don't want to waste that time. This is the "weeding out" process that Janet described, which I agree is a really good explanation for the overall high ratings on Goodreads.
First of all, what you're seeing has been studied by scientists and I remember reading about it a few years ago. It's a mostly unconscious confirmation bias. With the exception of trolls or people who are purposely picking books to challenge their palettes, you will rate things you buy highly because you bought them. To rate them lowly means you suck at making decisions about what to buy. Your brain, therefore, doesn't let you rate things lowly unless it REALLY sucked or pissed you off in some way. Over time this thread morphed into people talking about whether star ratings affect them. For me, when it comes to things I pay attention to star ratings. A shirt is a shirt; a hard drive is a hard drive. When the rating doesn't match my preconceptions (eg a hard drive brand I love has low ratings) I dive into the words. If I see that the low ratings are for dumb things like the person not knowing what they were doing or shipping - then I ignore it. If it's DOAs then that means maybe a bad batch or they aren't up to snuff anymore.
When it comes to subjective things like books and movies I rarely listen to the star ratings unless
a) it's someone I trust (personally or a critic who matches me most of the timg)
b) It's 5/5 or 1/5 - because it's pretty rare for something to be 1/5 and I like it.
I generally ignore the book ratings because when you actually read them, people get completely stupid about why they rate things lowly. I've seen books about The Buddha rated lowly by religious fundamentalists of other religions. And people rate political books lowly because they are from the other party. Or, in general, because the books have sex, profanity, LGBT members, tall people, fat people, people of different races.
So I tend to stick to things like S&L where they interview the author and I get a better idea of whether I'd like it, to reviewers I trust, to family/friends with the same taste, and to the text of Goodreads reviews. Also, once I find an author I like, I tend to like all they put out - especially in the same universe or genre. Sometimes when they cross genres I don't like it as much.
So I thought I'd take a look at S&L reads to see of group opinion (as I remember it) at all reflects average rating. A dance of cloaks, which was the strongest disliked since I've been a member, ranks in at 3.72
Among others, which seemed to be pretty dang beloved was a 3.69
Earthsea, which is of course a beloved classic of the genre, generally liked here but also the recipient of a lot of S&L criticism has a 3.96
Promise of blood, which most people seemed to enjoy, no one seemed to intensely love, and a few people felt fell flat, is at a wop
Ing 4.17
Lady Trent, which I remember being broadly liked, occasionally loved, and rarely criticized, is at 3.76
Ancillary justice, which people adored or hated has a 3.98! Good for you, Breq!
So yeah... Little correlation at all. Of course this isn't scientific at all as I'm just going from my only recollection of how threads went.
Oh, and Ulysses has the exact same rating as dance of cloaks.
Rob wrote: "So I thought I'd take a look at S&L reads to see of group opinion (as I remember it) at all reflects average rating. A dance of cloaks, which was the strongest disliked since I've been a member, ..."
Don't forget, the loudest people are the most passionate, but rarely reflect the majority.
I see the problems with these. I personally prefer a straight "thumbs up/down" sort of system. It loses the nuance of a rating system, but I think most people don't need that nuance or if they do they can actually read a review. I think a straight up or down system allows for more flexibility where you can determine that that X% of people like a book. The system Rotten Tomatoes has is good I think.
We do a thumbs up/down for our local book club, usually people are pretty sure about it but we have allowed 'thumbs sideways' for folks who have mixed feelings about books. 22% have unanimously positive ratings, 70% have half or more liking the book, and we've had 2 books no one liked. It's actually a pretty good system, although we've got a small sample size.
The Sword and Laser
Books mentioned in this topic
Casino Royale (other topics)Octopussy and the Living Daylights (other topics)
Have you ever noticed that whenever you check the rating for a book, it will almost always be between 3.25 and 4.5 stars. Now, there are many factors that would account for this. 1 and 2 star books get abandoned because of their poor quality, and people don't bother rating them or adding them to their shelves. (It's true. The worst thing you can do to an author is ignore them.)
Some people are generous with their ranking because they don't want to appear overly critical or hurt the author's feelings, even if they’re not aware they’re doing it. Additionally, tastes differ and what may be the "best novel ever written" for one person is someone else's "meh". Even leaders in their genre like Tolkien and GRRM rarely rise above a 4.5. This is what happens when you try to apply the Law of Large Numbers to personal taste.
I thought it was strange, so I did a little experiment. I searched for 100 random common words and took 1 book (with >100 raters) from each list. From this I determined that the average rating for books on Goodreads is actually around 3.9 stars instead of the 3.0 you would expect. I repeated this with other lists, both random and selected, and they all came to similar numbers. (Mean~3.9, stdev~0.30. Try it yourself)
What does this mean? It turns out that about 70% of all titles are rated between 3.6 and 4.2 stars, while 99% fall between 3.0 and 4.8 stars. If 3 stars is average, about 99% of books are above average. So, should we adjust the scale so that 3 becomes a 1 and 4.5 becomes 5? While this might be more useful, it is practically impossible. Also, it makes fair to good books seem less appealing that they might actually be. The system is broken.
What can you do?
-Rate all your books, especially the lousy ones. And be honest. You're not doing anyone any favors by being nice all the time.
-Double it. It may seem a little silly and even counter-intuitive, but it's easy and helps put a rating in perspective by watering down the bias. Thus a 3 becomes an 6, and a 4.5 becomes a 9.
-Read reviews. Notice the plural. Don't base your choices on one person's experience, even if it's someone you trust. Look for a consensus opinion.
-Write reviews. When someone takes the time to sit down and write a well thought out commentary, it says a lot about the book. Titles with few or no reviews are to be avoided.
-Encourage Goodreads and others to adopt a more representative metric of a book's quality than a simple average of votes.
What do you think? Should we give up on star ratings altogether?