Blog Post: Goodreads Transitions to New Data Sources

Goodreads Transitions to New Data Sources

Posted by Otis on January 30, 2012 1

We're happy to announce that Goodreads has reached a new milestone: We are no longer dependent on Amazon for book information. Ingram, the largest wholesaler of books in the U.S., is our new primary data partner, and our agreement with them will ensure accurate and reliable information for the foreseeable future. In addition to Ingram, we are working with dozens of other open data sources, such as libraries, to find book records from all over the world (see below for a list). This is an exciting moment, and we're confident that this change will be better for the members of Goodreads and long-term success of the site.

What has changed? Not as much as you might think.
Fewer than 2 percent of our 7 million users have books currently affected. However, you may notice some missing cover images. We are still in the process of importing millions of images from several sources, so please be patient—most of those images will be back very soon.

Today, 99 percent of the transition to new data sources is now complete. For the remaining 1 percent of books without new sources, book records may appear blank: listed as "Unknown Title" and "Unknown Author." Once we've found a new source of data for those books, they will be restored. Please be assured that all of your reviews and ratings will continue to be available during the transition.

What can I do right now?
Here's what you can do to help make the switch as smooth as possible:

If you see a blank book record—one listed as "Unknown Title" and "Unknown Author"—and you know what book it is (for example, if you reviewed the book and can recognize it from your review), please report it in the Goodreads Librarians Group or apply to become a Librarian yourself. If you're a librarian and you see a blank book record that you can identify, please feel free to enter new information using one of the approved sources listed in this post.

If you see a book missing a cover, feel free to find a new one on the author's or publisher's Web site and upload it to Goodreads. While many of the missing cover images will be included very soon in our ongoing imports, we also welcome book information from Goodreads members.

Why did we do this?
Quality book information—such as titles, authors, publication dates, and cover images—is the life blood of a site dedicated to book discovery and literary discussion. Since its inception, Goodreads has relied on Amazon's public API for much of this information, and while Amazon data was free, it came with many restrictions. For instance, we couldn't use their information for our mobile apps or link to competing bookstores. If you have any additional questions about the transition, please feel free to contact us.

We'd like to say a big thank you to the many Goodreads Librarians and Goodreads Authors who have worked tirelessly to help with the transition over the past few weeks. Thank you again for continuing to make Goodreads the amazing and unique place it is. We look forward to providing you with the best, most accurate, and most all-encompassing book database in the world for many years to come.

Our sources of data
Ingram
WorldCat
ISBNdb
All Major American Publishers
Many Smaller American Publishers
The Library of Congress
The German National Library
The British National Library
Goodreads Authors
(And growing!)

Authors and Publishers, would you like to be a source?
We are very proud of the enormous community of authors on the site, and keeping your book information accurate has always been a priority of ours. We are most likely already importing a feed of data for your books (either from Ingram or directly from your publisher), but if you would like to confirm this, or offer us a feed of data to ensure that your books will always be available to the millions of readers on Goodreads, please contact us.

Twitter_icon 28 likes · like

Comments (showing 1-50 of 67) (67 new)


message 1: by Isavarg (new)

Isavarg Is this why a load of covers on my book list has disappeared? It would be nice to have them back. Having a blank cover looks awful and if you aren't a goodreads librarian, you can't upload it without making a new edition as far as I can tell and I would like the cover displayed to be of the book I actually own.


message 2: by Patrick (new)

Patrick Brown Isavarg wrote: "Is this why a load of covers on my book list has disappeared? It would be nice to have them back. Having a blank cover looks awful and if you aren't a goodreads librarian, you can't upload it witho..."

As noted in the post above, most book covers will be back in a few days. We're importing them now.


message 3: by Sarah (new)

Sarah Yep, I can see loads of my covers that weren't there this morning, are back now.
I can only imagine this has been an enormous task, my thanks to everybody involved!


message 4: by Bex (new)

Bex I just want to say thank you to all the librarians who rescued many many books each. I only managed a small handful myself but everyone did their bit enormously.

Also I (and I'm sure others) intend on going back in there in the next few days to update spare fields which we skipped over during our epic rescue attempts. Nobody likes a missing cover ;)


message 5: by MissSusie (new)

MissSusie Thanks for the update!


message 6: by Nell (last edited Jan 30, 2012 12:39PM) (new)

Nell I'm assuming that Kindle book images will not return since this is an Amazon product. For 2012 books, I switched to the generic ebook image if one was available. Is that a correct assumption?


message 7: by Cheryl (new)

Cheryl Isavarg wrote: "Is this why a load of covers on my book list has disappeared?

Um, yes. That's what this post says.

It would be nice to have them back.

Again, this post states they will be restored. Did you not read the whole article????


message 8: by Kara (new)

Kara Nell wrote: "I'm assuming that Kindle book images will not return since this is an Amazon product. For 2012 books, I switched to the generic ebook image if one was available. Is that a correct assumption?"

The authors of Kindle editions can upload their cover images to Goodreads themselves. Or Goodreads Librarians can get the image from the author's website or library websites. Taking the cover from the ebook edition on Goodreads should be okay, but it's a good idea to double-check elsewhere that they're the same.


message 9: by Shelley (new)

Shelley Goodreads is awesome! Thanks for all you do!


message 10: by Rachel (new)

Rachel I was wondering if you're actively pursuing a Canadian source for data? I noticed it's all my Canadian books whose covers have disappeared, or else they're now an American edition cover. Just curious.


message 11: by Patrick (new)

Patrick Brown Rachel wrote: "I was wondering if you're actively pursuing a Canadian source for data? I noticed it's all my Canadian books whose covers have disappeared, or else they're now an American edition cover. Just curious."

Yes, in fact, we are working on a new and very comprehensive Canadian data source as I write this.


message 12: by Louise (new)

Louise Forster I don't know why I'm continuously gobsmacked at Amazon's muscle. I commend you for taking this step. I applaud the goodreads site. It has helped me to understand the Amazon juggernaut.


message 13: by mlady_rebecca (new)

mlady_rebecca Thanks for the update and all the hard work.


message 14: by Suzanne (new)

Suzanne At what point to we assume the cover art and other data, will not be coming back for a Particular book? Do we assume that if this hasn't reappeared by the end of the week, that we need to request that book art etc is added? I have a lot of books where the cover art (and other data is mising), partucarly on box sets. It seems that you are still looking for other sources of information, I would have thought book depository would be a good replacement for Amazon.


message 15: by Trice (new)

Trice Suzanne wrote: "It seems that you are still looking for other sources of information, I would have thought book depository would be a good replacement for Amazon.
"


book depository doesn't seem to have book summaries available though, at least from what I've seen


message 16: by Moloch (new)

Moloch Isn't Bookdepository a part of Amazon now?


message 17: by Dark (new)

Dark Ape At a guess at the trouble I am having and the widgets page is having, I believe this transition is also impacting the results from some of the widgets, at least for now, but I am always glad for more reliable data! :)


message 18: by Lisa M (new)

Lisa M So, let me get this straight. Most of my books are Kindle books. That means that most of my books will NOT have covers unless I become a librarian and add them myself, or ask some poor librarian to add 150+ book covers for me. Is that right?


message 19: by Terri (new)

Terri Now I can understand what to do. Thanks for update!


message 20: by Ellie (new)

Ellie Loredan The German National Library! YAAAY! Thanks so much!


Ms Bubbles SockieP Otis - couldn't you put a link to your blog under Explore? I never think about it unless it's mentioned and linked to in a thread. Alternatively perhaps put it as a banner when you write a new blog. Thanks.


message 22: by Ridley (new)

Ridley Is GR getting a feed from Harlequin? During this whole song and dance I found myself manually adding a lot of Harlequin ebooks to rescue their Kindle editions.

Since they publish something like 100 romance novels a month, that'd be a nice chunk of books to play with.


message 23: by Patrick (new)

Patrick Brown Ridley wrote: "Is GR getting a feed from Harlequin? During this whole song and dance I found myself manually adding a lot of Harlequin ebooks to rescue their Kindle editions.

Since they publish something like 10..."


We are getting a feed from Harlequin, but it does not include Kindle editions, so if that's primarily what you're reading, that may be the issue.


message 24: by Ridley (new)

Ridley I was putting the ISBN ebooks in, not Kindle editions. Is the feed new? Maybe it's not retroactive.


message 25: by Patrick (new)

Patrick Brown Ridley wrote: "I was putting the ISBN ebooks in, not Kindle editions. Is the feed new? Maybe it's not retroactive."

It's coming from an aggregator, so we'll probably need to check with them about it. Thanks for reporting that.


message 26: by Patrick (new)

Patrick Brown Ridley wrote: "I was putting the ISBN ebooks in, not Kindle editions. Is the feed new? Maybe it's not retroactive."

And actually, if you have some examples, that would be great too. We were finding successful ONIX feeds for most harlequin books.


message 27: by Ridley (last edited Jan 31, 2012 12:17PM) (new)

Ridley Honestly, I can go on the Harlequin ebook site, copy and paste ISBNs at random into Goodreads, and 9 times out of 10 they won't be in GR's database.

For example: This book only lists the paperback, but Harlequin shows an ebook version here.

ETA: And of course every other ISBN I've tried just now has worked. And the example above is for an ebook that releases tomorrow, So maybe it's a smaller issue than I've made it out to me.


message 28: by Patrick (new)

Patrick Brown Ridley wrote: "Honestly, I can go on the Harlequin ebook site, copy and paste ISBNs at random into Goodreads, and 9 times out of 10 they won't be in GR's database.

For example: This book only lists the paperback..."


It looks like we might be a little bit behind on processing the feeds. Since this is a newer book, it should get picked up in the next pass. We'll eventually get a good, efficient schedule for processing the feeds sorted out, but until then, you may occasionally see books like this that haven't made it onto the site yet.


message 29: by Ridley (new)

Ridley Yeah, as I looked at it more, GR seems to just miss the ones released in the past few months. Anything newer than October 2011 is spotty.


message 30: by Katharina (new)

Katharina Gerlach I tried to rescue a book and got a message saying it had already been rescued, but the title, author, and cover on the book page were still "Unknown ..." Does that mean the book hasn't been rescued despite the claim? Or hasn't the data been posted yet?


message 31: by Patrick (new)

Patrick Brown Katharina wrote: "I tried to rescue a book and got a message saying it had already been rescued, but the title, author, and cover on the book page were still "Unknown ..." Does that mean the book hasn't been rescued..."

That's a result of the sort of in-between state. We should have a rescue page that lets you re-source the data fairly soon. Sorry for the inconvenience.


message 32: by Alessandra (new)

Alessandra Petra X wrote: "Otis - couldn't you put a link to your blog under Explore? I never think about it unless it's mentioned and linked to in a thread. Alternatively perhaps put it as a banner when you write a new blo..."

I second this. I wasn't even aware of this blog until I saw a link in a post.


message 33: by Felicity (new)

Felicity Heaton What about future books? Will you continue to bring in the books and links for Amazon Kindle, BN, CreateSpace paperback etc versions?

Personally I make over 80% of my royalties and sales on Amazon, so if Amazon books aren't going to be listed on here anymore, it sort of turns me off Goodreads. I buy all my books on Amazon too, so if I'm browsing books on GR, I like to have the ability to buy from them with a link from your site.

Everything I've read about the move on the web makes it sound very anti-Amazon.


message 34: by Otis (new)

Otis Chandler Felicity wrote: "What about future books? Will you continue to bring in the books and links for Amazon Kindle, BN, CreateSpace paperback etc versions?"

Our goal is and has always been to have all books on Goodreads available to review. As Amazon isn't giving us a feed of KDP or CreateSpace titles, we will have to manually add those until we find a better solution. We will continue to link to Amazon - we never said we wouldn't do that. This change is not about us being anti-Amazon. It's about us having more freedom to build our business with our own data.


message 35: by Pinky_rocks (new)

Pinky_rocks it is nice that you are trying hard to stop being monopoled by amazon. and to see that there are a lot of ambitious people trying to make this site better. but i would think if you start to develop something new for an already existing site, you would do a beta-version and work on it, and change everything only when you're really done with it.
I'm just a little annoyed. I don't really care, that some book covers are momentarily not visible, as i hope they will come back soon (though i kind of have my doubts about all of my german books...as all of the covers have gone awol), I just don't like the fact that some books have completely changed editions! this is making it very unorganized for me, and I use goodreads for organizing my books...i'd rather have no cover and the actual edition that i have on my shelf, than a completely wrong edition with the wrong page count and just everything.

But I'll have trust and think you'll work on everything! keep going goodreads! :) just ranting


message 36: by Troy (new)

Troy Blackford This is AWFUL! There are FIVE of my last 25 read books totally missing, none of which have any text reviews, and therefore I haven't the FOGGIEST idea what they are.

I mean I would know if I had any idea what the title, author, or topic was... but as all I can see is 'Unknown Book 10204912024 by Unknown Author 193102,' I am drawing a complete blank. Hope this gets fixed in... ever.


message 37: by Carolyn (new)

Carolyn Troy wrote: "This is AWFUL! There are FIVE of my last 25 read books totally missing, none of which have any text reviews, and therefore I haven't the FOGGIEST idea what they are.

I mean I would know if I had a..."


I looked at the first one and it's a Kindle edition - can you look on your Kindle to see a date read or anything like that? (I don't have a Kindle, so I have no idea.) Or can you look it up by it's ASIN, or by it's publication date (June 2011)?

This is a tremendous PITA, I know, but it's just a bump in the road. It will get better. = )


message 38: by Ciara (last edited Feb 02, 2012 09:14AM) (new)

Ciara hey goodreads, what the fuck is up with having ads along the side of the page that BLOCK THE FUNCTIONS of the website? i tried pulling down the "explore" menu & an ad for some horror movie that i will never see was blocking the list of choices. there was no way to close the ad or move the "explore" functions menu so that i could access it. i couldn't click on what i wanted through the ad because it would just take me to the ad. similarly, when i pulled up a screen to write a book review (& bear in mind that i have written over 800 reviews on your website, i'm one of your top 150 contributers, generating free content & links to your website), the ad was blocking the text box. i could write, but i couldn't see what i was writing, so if i paused to gather my thoughts, i couldn't see where i'd left off to make sure i was punctuating properly & all that. i don't care if you need to sell ad space on the site, but maybe don't have it interfere with the actual site functionality.


message 39: by Otis (new)

Otis Chandler Ask Rivka


message 40: by Pinky_rocks (new)

Pinky_rocks @carolyn: it wouldn't be a problem looking for the book missing by the ASIn or anything else, but I just found a "missing book" on my own list and the details say it's in croatian! I have not a single book in Croatian standing here since I cant understand it. so there's no relying on the little information in the books, and if you have a huge list of books it is very very difficult to find out just which one is missing... now I'm off to find that croatian book on my shelf :)


message 41: by Carolyn (last edited Feb 02, 2012 02:24PM) (new)

Carolyn Thanks Pinky, but I was only addressing the Kindle issue that Todd was having with his books.
Overall, I think with the immense amount of work done by staff and volunteer librarians that a LOT of data was saved in the immediate present. Yes, there is plenty of work to do to clean things up, and yes, it will take some time, but overall it could have been much much worse. Keep in mind, there was no 'beta test' that could have been prepared for - when GR decided to move away from Amazon, they set a totally unrealistic (and cruel) deadline of wiping all their data from the site. As an example of how they should have acted, when GR lost Barnes&Noble as a data source, they just shut the feed off, but didn't ask that everything be scrubbed!

You should be able to find them pretty easily. Go to your "all" shelf and sort by author, then page or scroll to the 'U's for Unknown. You can do the same thing for title, and catch them both ways. HTH


message 42: by [deleted user] (new)

Sucks. Totally destroyed my author's page and nothing has been restored.

If it isn't broke, fix it. Right.


message 43: by Pascale (new)

Pascale Almost all my French books are not to be found in the renewed database... :'(
It's too bad GR doesn't get more international. I feel that beside English/American books, there is not much that is done for other languages... But maybe I'm wrong, I hope I am.


message 44: by Otis (new)

Otis Chandler Pascale wrote: "Almost all my French books are not to be found in the renewed database... :'(
It's too bad GR doesn't get more international. I feel that beside English/American books, there is not much that is do..."


We were able to get as many international books as we'd like. Although Worldcat and other library sources had quite a few. We intend to work on more and better international databases this year. If you know any good sources, do please let us know!


message 45: by Pascale (new)

Pascale Thank you for your answer. :)
I'll be sure to inform the GR staff if I know of any good sources other than Amazon.


message 46: by Pascale (last edited Feb 03, 2012 01:09AM) (new)

Pascale Otis wrote: "Pascale wrote: "Almost all my French books are not to be found in the renewed database... :'(
It's too bad GR doesn't get more international. I feel that beside English/American books, there is not..."


Found professional databases for french books : http://www.electre.com/
http://www.bibliopoche.com/


message 47: by Carolyn (new)

Carolyn Pascale wrote: "Otis wrote: "Pascale wrote: "Almost all my French books are not to be found in the renewed database... :'(
It's too bad GR doesn't get more international. I feel that beside English/American books,..."


Pascale - It's great you're finding sources! You'll want to send an email to the "contact us' link with the information as well (if you haven't already), to make sure TPTB see it asap. = )


message 48: by Emmanuel (new)

Emmanuel For french books, the bibliothèque nationale de France should also be a good source: http://catalogue.bnf.fr/


message 49: by [deleted user] (new)

Isavarg wrote: "Is this why a load of covers on my book list has disappeared? It would be nice to have them back. Having a blank cover looks awful and if you aren't a goodreads librarian, you can't upload it witho..."

Missing book covers doesn't bother me. Missing titles AND authors so I can't even tell WHICH book has been deleted -- that's ridiculous.


message 50: by Whitney (new)

Whitney Moore You might be happy to announce that you are "no longer dependent on Amazon for book data," but some of us (i.e. me!) are not. I woke up this morning and all of my books had disappeared from my author profile except for one that is out of print and 2 that I originally had to add manually.

Total bollocks.


« previous 1
back to top