Large Book Data Import

[Tiffany]

Scott wrote: "GoodEats"

:)

[Michael]

We definitely need a [different] permanent thread for "funniest finds"...

[Carin]

Ingram has a division (Spring Arbor) that sells religious items as well as books. I'm sure that's where the communion wafers etc. are coming from.

[Scott]

Ellie [The Empress] wrote: "Lobstergirl wrote: "Oh now I want to shelve the communion wafers, dammit."

I would be funny if they get a genre box soon. :D"

Badly Behaving Saviors

[Banjomike]

I've been looking at the current edits being done by amazon_kcw and there are a lot like these with two apparently identical cover image updates being done at the same time (consecutive edit numbers)

https://www.goodreads.com/book/edits/...
https://www.goodreads.com/book/edits/...
https://www.goodreads.com/book/edits/...

[Sarah]

Hi there - as a part of improving imports we're trying make sure that the librarian change log is working as expected. Has the 'undo' for book descriptions always been broken or did it break somewhere along the line? If the latter, when did it stop working?

[Plethora]

I ran into what Banjomike referenced in #255 as well yesterday.

I noticed a book on my shelf had changed covers. I went to the edit screen and it had three consecutive overwrites from Amazon_KCW. After undoing each of those the original cover was restored.

Why is it STILL overwriting covers????? These were recent, this month.

[Lobstergirl]

https://www.goodreads.com/book/show/2...

Yes, this description is an amazon import. I wish I were joking.

[Empress]

https://www.goodreads.com/book/edits/...

[Michael]

Sarah wrote: "Hi there - as a part of improving imports we're trying make sure that the librarian change log is working as expected. Has the 'undo' for book descriptions always been broken or did it break somewh..."

I am just a newbie, starting as librarian in October 2013. It did not work then, either. But I had already given up on reporting bugs due to overwhelming reactions from GR.

[Michael]

https://www.goodreads.com/book/edits/...

Dec 11, 2013: amazon_kcw changes [perfect] cover provided by ingram

Jan 10, 2014: amazon_kcw changes its own change

Feb 16, 2014: I revert both.

[Sarah]

I just made a change that I'm hoping (fingers crossed) will stop amazon_kcw from overwriting covers and other data coming in from our other data feeds. It may still 'overwrite' its own data (usually with the data), but I'm hoping this will alleviate some of the pain here.

[Lobstergirl]

Is there any way that Amazon imports for the description field can be turned off? They're just so awful. Every one of them makes me cry.

[Tiffany]

Lobstergirl wrote: "Is there any way that Amazon imports for the description field can be turned off? They're just so awful. Every one of them makes me cry."

Yes, yes, yes! This most recent set of imports, especially. Uck!

[Scott]

Amazon_kcw seems to have imported a bunch of Michael McDowell's Blackwater novels that are duplicates of existing entries:

https://www.goodreads.com/author/list...

(see the batch at the bottom of page 1)

[Lobstergirl]

https://www.goodreads.com/book/show/4...

Normal ISBN-10 from amazon-kcw, severely abnormal ISBN-13 from amazon-sable. Can't imagine how that could be correct...according to normal rules the ISBN-13 should be 9780470372272.

[Empress]

Amazon did a lot of overrides of it's own information on this book. Ones it changes the publisher from 'Chelsea Green Publishing' to '"Chelsea Green Publishing"' and ones from '"Chelsea Green Publishing"' to 'Chelsea Green Publishing'
(I hope that is the fix actually), but the description. It would be nice if it just doesn't override if there is any description.

[Sarah]

Lobstergirl wrote: "https://www.goodreads.com/book/show/4...

Normal ISBN-10 from amazon-kcw, severely abnormal ISBN-13 from amazon-sable. Can't imagine how that could be correct...according to ..."

The priority for amazon_kcw and amazon_sable have been lowered below our other data sources - it should no longer be overwriting fields that already exist. Let me know if you seen any instances of this happening in the future.

For overwrites done in the past, it's probably best just to undo the incorrect overwrites.

[Sarah]

Ellie [The Empress] wrote: "Amazon did a lot of overrides of it's own information on this book. Ones it changes the publisher from 'Chelsea Green Publishing' to '"Chelsea Green Publishing"' and ones from '"Chelsea Green Publ..."

The removal of the quotation marks was indeed a fix we ran :-)

[Sarah]

Scott wrote: "Amazon_kcw seems to have imported a bunch of Michael McDowell's Blackwater novels that are duplicates of existing entries:

https://www.goodreads.com/author/list...

(see the ba..."

It is now a requirement that books created by amazon imports must contain either an isbn or an asin - so that should limit the number of duplicates in the future.

We've run a script to remove most of the duplicates produced, but I need to do a second pass to remove duplicates which were subsequently shelved by users (to move the shelvings to the correct book) and to clean up books missed in the first go.

[Plethora]

Sarah -

What date do we consider as new instances of the problem? I had a cover overwrite 10 days ago that I fixed that was from earlier in the month.

It would be helpful to know that if we see cover changes after X date and description changes after X date, etc, that these are indeed new/on-going that still need to be brought to your attention.

[Sarah]

Hi Bookworm - I'd say anything February 18 or later should be considered a new instance of the issue. I released a change last Monday (Feb 17) that should have fixed the issue.

Also - I believe a developer pushed changes that may have fixed 'undo' for descriptions, can someone test it out (preferably on a description that actually needs to be undone!) and let me know if it's working?

Thanks!

[Empress]

Sarah wrote: "Hi Bookworm - I'd say anything February 18 or later should be considered a new instance of the issue. I released a change last Monday (Feb 17) that should have fixed the issue.

Also - I believe a ..."

I did "undo" for the description in the same book from post 267 today and it worked just fine. The amazon script had changed it from a valid description to a description of condition for the paperback.

Off-topic:
I just couldn't remove the description on the Spanish edition. I'm not sure how to remove a description set as default, I tried by "editing" all descriptions but that did not work.

[Plethora]

Sarah wrote: "Hi Bookworm - I'd say anything February 18 or later should be considered a new instance of the issue. I released a change last Monday (Feb 17) that should have fixed the issue.

Also - I believe a ..."

Thanks, I was losing track of when changes were applied.

[Emy]

Ellie [The Empress] wrote: "Off-topic:
I just couldn't remove the description on the Spanish edition. I'm not sure how to remove a description set as default, I tried by "editing" all descriptions but that did not work. "

With that I find I have to "edit" the default description, and scroll to the bottom of the box and then add a carriage return (just a blank enter) at the end, then save. It usually works. Not always though :/

[Empress]

I noticed onyx_ingram is overwriting it's own descriptions with info when the book was first published and imprint series.

Those are all recent changes and it changed the book description three times. Can you stop it from overwriting description and re-do all the ones it's done recently?

Here is the change log: https://www.goodreads.com/book/edits/...

Edit: Thank you Emy. I was thinking of doing an edit in the description field. Will try it next time.

[Sarah]

Ellie [The Empress] wrote: "I noticed onyx_ingram is overwriting it's own descriptions with info when the book was first published and imprint series.

Those are all recent changes and it changed the book description three t..."

I think this is intentionally being allowed. Sometimes we get updates to old information or fixes to incorrect information via onix feeds. The current import logic allows any data source with equal or greater priority to the previous updater to overwrite existing data.

If there's a consensus that overwrites from sources with equal priority are causing more harm than good, I can further the discussion with some other developers/product managers.

[Empress]

Thank you for replying. I have just two question about that:

If I revert the changes will the script rewrite my change?

Is it re-writing user descriptions as well?

[Sarah]

Ellie [The Empress] wrote: "Thank you for replying. I have just two question about that:

If I revert the changes will the script rewrite my change?

If you revert the change - it should theoretically attribute the change to you, making it impossible to overwrite by a non-user. But let me check that this is what is actually happening :-)

Is it re-writing user descriptions as well?"
No - a non-user datasource should *never* overwrite a user entered field.

[Lobstergirl]

amazon_sable updated the book Strange Death of Liberal England, Nineteen Hundred and Ten Thru Nineteen Hundred and Fourteen by George Dangerfield
title: 'The Strange Death of Liberal England 1910-1914' to 'Strange Death of Liberal England, Nineteen Hundred and Ten Thru Nineteen Hundred and Fourteen'

Oh, brilliant. Such an improvement.

[Empress]

Sarah wrote: "Ellie [The Empress] wrote: "Thank you for replying. I have just two question about that:

If I revert the changes will the script rewrite my change?

If you revert the change - it should theoreti..."

yes, an undo put me as the author of the description. Thank you.

[Keith]

Here's an odd one that also seems to result from either sable or ingram imports:

https://www.goodreads.com/book/edits/...

Lee added secondary authors Feb-2012, then ingram and sable made updates without any record of removing those names, but they were gone when I got there. I have restored them, along with fixing the rest of the record, but this may be indicative of an area where (a) there is insufficient change logging, and (b) where scripts are still overwriting user changes.

[Elizabeth (Alaska)]

Note odd description for this edition:

Divisadero

I can fix the description, but wanted to alert regarding the import.

[Susie]

Some cover changes are still going through:
onix ingram updated the book The Atrocity Exhibition by J.G. Ballard
image_uploaded_at: 'Tue Dec 25 05:20:19 -0800 2012' to 'Sun Mar 16 11:11:55 -0700 2014'
https://www.goodreads.com/book/show/3...

[Plethora]

Can MR be set as a flag to not import from the feeds in the Author names?

[Empress]

Bookworm R wrote: "Can MR be set as a flag to not import from the feeds in the Author names?"

As well as N/A. Could also ??????? be set as a flag when in titles?
https://www.goodreads.com/book/edits/... and various from Unknown

[Paw3pals]

Bookworm R wrote: "Can MR be set as a flag to not import from the feeds in the Author names?"

Suggest it be tied to at least 2 following names, i.e., MR Stevens would not be stopped, but MR Mark Stevens would be stopped, or ideally just Drop the MR.

And add the following, all with and without an ending period: MS, Mrs, Dr, MD, Prof, Professor, RN, Rev, Reverend, PHD, PH D, Hon, Honorable.

[Empress]

Paw3pals wrote: "Bookworm R wrote: "Can MR be set as a flag to not import from the feeds in the Author names?"

Suggest it be tied to at least 2 following names, i.e., MR Stevens would not be stopped, but MR Mark S..."

I never thought to suggest any of these, because I though if they can drop them, they would.

[Susie]

Again on the cover updates still going through - this one was from last Sunday:

onix ingram updated the book Aesop's Fables by Aesop
image_uploaded_at: 'Sun Mar 23 10:59:25 -0700 2014' to 'Sun Mar 23 14:36:21 -0700 2014'

[rivka, Former Moderator]

Images from the same source will indeed replace. Nothing has changed there. What has changed is that Amazon (Sable and KCW) will not replace images from other import sources. See https://www.goodreads.com/topic/show/... and https://www.goodreads.com/topic/show/...

[Susie]

Ah, that's good to know. Just means then that, in some cases, we'll have to be more vigilant. But it is only from Onix where I've seen the cover overwrites still happening and luckily not from Amazon.

Does that mean, then, if I'm reading that correctly, that even the Onix import will respect any changes made by librarians?

[rivka, Former Moderator]

Absolutely. User edits have higher priority than all imports.

[Keith]

rivka wrote: "Absolutely. User edits have higher priority than all imports."

Though see my Mar 14, 2014 07:43PM report in comment 282 in this thread above (which may have been fixed).

[rivka, Former Moderator]

Because of the logging gap, I'm not sure what happened there.

[Amanda]

I'm noticing a lot of books that appear to be imported by onix smashwords that don't have any covers. Is there a reason the covers aren't importing? When I check the smashwords page for the book, the cover is there.

[Sarah]

Amanda wrote: "I'm noticing a lot of books that appear to be imported by onix smashwords that don't have any covers. Is there a reason the covers aren't importing? When I check the smashwords page for the book, t..."

We're currently having a bit of an issue with the images that smashwords is sending us - there's a developer working on sorting things out, but thanks for catching that!

[Amanda]

Sarah wrote: "Amanda wrote: "I'm noticing a lot of books that appear to be imported by onix smashwords that don't have any covers. Is there a reason the covers aren't importing? When I check the smashwords page ..."

Thanks for getting back to me.

[Susie]

Sarah wrote: "We're currently having a bit of an issue with the images that smashwords is sending us - there's a developer working on sorting things out, but thanks for catching that!"

I've noticed that when I've downloaded books from Smashwords, that they don't use a jpg format but instead use an alternative format that isn't accepted by the GR upload (.jfif). It might have something to do with that.

[Empress]

Amazon adds description with EXCERPTS in it?

Told entirely in email messages, the way i see it captures the raw emotion and angst of high school.

Sarah is caught between two guys. One, named Eric, is sweet and smart, and she's head over heels for him. One, named Mick, is her father's boss' son. He wants Sarah bad. When Sarah picks Eric over Mick, Mick uses his position as the son of a powerful man to get revenge on Sarah. One by one, her friends turn against her, until there's only one obstacle in Mick's way--Eric.

Loosely based on The Illiad, this novella chronicles a conflict between high school friends.

--------------------------
EXCERPT:

From: Dana Hellerman
To: Sarah Rook
Date: Fri 31 Jan 2003 10:09:38
Subject: (none)

sarah,
what's up? i am in ms. cooke's class but she can't see that i am typing. i'm supposed 2 be working on some dumb short story. this will have to be quick. are u going 2 the party this weekend? Tristan and Eric are going to be there, atleast that's what Tristan told me yesterday on the phone. if u check this during study hall e-mail me back. I will be on the computer at lunch and then we can talk about it after school.
did u see what cara wilson is wearing? i was going to buy a shirt like that at the mall, but now that i see it on her, i am not sure. she is alot thinner than me and she looks horrible!! i don't know. what do u think? well, i will see u after school, girl. hope ur coming this weekend.
l8er,
dana


From: Sarah Rook
To: Dana Hellerman
Date: Fri, 31 Jan 2003 12:44:01
Subject: Where is the party?

Dana,

I don't know if I can get out this weekend. My dad says that I can only have the car once every two weeks and you know I took it last week. You would have to pick me and then someone would have to get me home by curfew.

I will try. Where is the party? Is it in Reetown again? I don't know why you keep telling me about Eric, because you know I don't like him. He is a huge dork and I don't care if he IS your boyfriend's best friend, that does not mean you have to foist him on me.

I thought Cara looked cute. You think she looks horrible? Anyway, even if she is thinner, you are much better proportioned, if you know what I mean, and I'm sure the shirt would look fine on you. Cara looks like a clothes hanger. We have had this discussion before. You know that you are really hot and stop being such an idiot about the whole thin thing. You know that you look bigger because you are bigger--where it counts!! Ask Tristan.

Okay, well, this is study hall, and I have a major test in Trig next period. So I have to go. See you after school!!

LYLAS,
Sarah(less)

https://www.goodreads.com/book/edits/...

[Empress]

Amazon script is re-writing user-uploaded covers (March 2014)
https://www.goodreads.com/book/edits/...
https://www.goodreads.com/book/edits/...