Goodreads Developers discussion

2860 views
examples / showcase > Python wrapper

Comments Showing 1-42 of 42 (42 new)    post a comment »
dateDown arrow    newest »

message 1: by Sefa (last edited Mar 08, 2015 07:26PM) (new)

Sefa (sefakilic) | 6 comments Hi all,

I've started a Python wrapper for Goodreads API. It aims to provide full coverage of Python interface for API methods. Any suggestions or contributions are welcomed!

https://github.com/sefakilic/goodreads


message 2: by [deleted user] (new)

Thanks for the contribution Sefa!


message 3: by Ettore (new)

Ettore Pasquini Thanks Sefa. I've moved your post to the examples/showcase folder and marked it as sticky, so other people can find it easily. This kind of question has come up frequently.

Again, thanks so much!


message 4: by Sefa (new)

Sefa (sefakilic) | 6 comments Thanks for moving it to the examples/showcase folder. I must say that it is far from complete yet, but I am planning to complete it soon with proper documentation and testing.

Hope it will be useful for Goodreads people!

Cheers,


message 5: by Mohan (new)

Mohan (mohansn) | 5 comments Sefa wrote: "Thanks for moving it to the examples/showcase folder. I must say that it is far from complete yet, but I am planning to complete it soon with proper documentation and testing.

Hope it will be usef..."


Thank you!


message 6: by Sefa (new)

Sefa (sefakilic) | 6 comments Hi again,

I guess the python module (https://github.com/sefakilic/goodreads/) is in good shape. Any feedback/contribution is more than welcome!

Thanks!


message 7: by David (new)

David (davidyoud) | 1 comments Nice! I'll be trying this out at some point.


message 8: by Sebastian (last edited Apr 22, 2015 04:54AM) (new)

Sebastian | 2 comments Hi! Junior developer here. :-)

First off, thanks for writing this!

I seem to be getting an " ImportError: No module named 'session' " error on the first line, "from goodreads import client"

I'm on Windows 8.1, running Python 3.4.3. I installed the dependencies prior to installation with pip.

I'll continue to try to find a way to work around it.

Below is a log of the installation:

C:\downloads\goodreads-master>python setup.py install
running install
running bdist_egg
running egg_info
writing goodreads.egg-info\PKG-INFO
writing requirements to goodreads.egg-info\requires.txt
writing dependency_links to goodreads.egg-info\dependency_links.txt
writing top-level names to goodreads.egg-info\top_level.txt
reading manifest file 'goodreads.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'goodreads\apikey.py'
writing manifest file 'goodreads.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_py
creating build\bdist.win-amd64\egg
creating build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\author.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\book.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\client.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\comment.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\event.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\group.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\owned_book.py -> build\bdist.win-amd64\egg\goodreads

copying build\lib\goodreads\request.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\review.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\session.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\user.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\user_status.py -> build\bdist.win-amd64\egg\goodreads
copying build\lib\goodreads\__init__.py -> build\bdist.win-amd64\egg\goodreads
byte-compiling build\bdist.win-amd64\egg\goodreads\author.py to author.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\book.py to book.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\client.py to client.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\comment.py to comment.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\event.py to event.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\group.py to group.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\owned_book.py to owned_book.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\request.py to request.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\review.py to review.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\session.py to session.cpython-34.pyc
File "build\bdist.win-amd64\egg\goodreads\session.py", line 34
print self.request_token, self.request_token_secret
^
SyntaxError: Missing parentheses in call to 'print'

byte-compiling build\bdist.win-amd64\egg\goodreads\user.py to user.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\user_status.py to user_status.cpython-34.pyc
byte-compiling build\bdist.win-amd64\egg\goodreads\__init__.py to __init__.cpython-34.pyc
creating build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\PKG-INFO -> build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\SOURCES.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\dependency_links.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\requires.txt -> build\bdist.win-amd64\egg\EGG-INFO
copying goodreads.egg-info\top_level.txt -> build\bdist.win-amd64\egg\EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist\goodreads-0.2.3-py3.4.egg' and adding 'build\bdist.win-amd64\egg'
to it
removing 'build\bdist.win-amd64\egg' (and everything under it)
Processing goodreads-0.2.3-py3.4.egg
Removing c:\python34\lib\site-packages\goodreads-0.2.3-py3.4.egg
Copying goodreads-0.2.3-py3.4.egg to c:\python34\lib\site-packages
goodreads 0.2.3 is already the active version in easy-install.pth

Installed c:\python34\lib\site-packages\goodreads-0.2.3-py3.4.egg
Processing dependencies for goodreads==0.2.3
Searching for rauth==0.7.1
Best match: rauth 0.7.1
Adding rauth 0.7.1 to easy-install.pth file

Using c:\python34\lib\site-packages
Searching for requests==2.6.0
Best match: requests 2.6.0
Adding requests 2.6.0 to easy-install.pth file

Using c:\python34\lib\site-packages
Searching for xmltodict==0.9.2
Best match: xmltodict 0.9.2
Adding xmltodict 0.9.2 to easy-install.pth file

Using c:\python34\lib\site-packages
Searching for nose==1.3.6
Best match: nose 1.3.6
Processing nose-1.3.6-py3.4.egg
nose 1.3.6 is already the active version in easy-install.pth
Installing nosetests-3.4-script.py script to C:\Python34\Scripts
Installing nosetests-3.4.exe script to C:\Python34\Scripts
Installing nosetests-script.py script to C:\Python34\Scripts
Installing nosetests.exe script to C:\Python34\Scripts

Using c:\python34\lib\site-packages\nose-1.3.6-py3.4.egg
Finished processing dependencies for goodreads==0.2.3

C:\Python34\Scripts>python
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from goodreads import client
Traceback (most recent call last):
File "", line 1, in
File "C:\Python34\lib\site-packages\goodreads-0.2.3-py3.4.egg\goodreads\client.py", line 2, in
from session import GoodreadsSession
ImportError: No module named 'session'
>>>


message 9: by Sefa (new)

Sefa (sefakilic) | 6 comments Hi, thanks for your interest.

It seems you're using Python 3. The wrapper supports 2.7 only for now.

For further discussion/bug reports, you can create an issue at https://github.com/sefakilic/goodread...

Thanks!


message 10: by Sebastian (new)

Sebastian | 2 comments Ooooh I see.
I'll try installing Python 2.

Thank you so much!


message 11: by Sahar (new)

Sahar Pirmoradian (spirmora) | 8 comments I have written a simple Jupyter notebook to show how to use the Goodreads Python library for retrieving basic information from Goodreads:

https://github.com/pirmoradian/Goodre...

Hope that helps.
Sahar


message 12: by Jay (new)

Jay Moskowitz (chiefwizard) | 7 comments Does the Python wrapper provide the 1 request per second limitation of the terms of service or should the program using the wrapper ensure calls are not made more often than this limit?


message 13: by frando (new)

frando (broadishreads) | 11 comments It's been a bit since this Python wrapper was updated and is currently broken now that the Goodreads API enforces using https. I recently detached a fork of the project and changed a few things around. Please check out https://github.com/thejessleigh/bette... and let me know if you have any suggestions, or maybe open up a pull request! :)

You can install the package with pip
`pip install betterreads`


message 14: by Alexis (new)

Alexis (lexish) | 12 comments Hi Jess, you rock. I'm gonna check it out!


message 15: by Michael (new)

Michael Davie (michaeldavie) | 1 comments I'm trying to use your library, but I'm getting 403 forbidden errors for any requests that require authentication. However, when I try the same request with requests_oauthlib directly it works. I'd appreciate any guidance you could provide.


message 16: by frando (new)

frando (broadishreads) | 11 comments Hi Michael. Which library are you using? The goodreads and goodreads2 libraries don’t use https for their authentication requests. I’ve created an updated library called betterreads that should fix the 403 problem.

If you are seeing 403s using betterreads please either file a bug or direct message me and I’ll try to troubleshoot with you.


message 17: by frando (new)

frando (broadishreads) | 11 comments I meant to say or send me a direct message. You can view the code and file bug reports at github.com/thejessleigh/betterreads


message 18: by أحمد (new)

أحمد الغامدي (ahalghamdi) | 1 comments Jess wrote: "It's been a bit since this Python wrapper was updated and is currently broken now that the Goodreads API enforces using https. I recently detached a fork of the project and changed a few things aro..."

Hi, Jess. I have got this error when I try to install betterreads

ERROR: Command errored out with exit status 1:
command: 'c:\users\dell\anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\DELL\\AppData\\Local\\Temp\\pip-install-6dofct_j\\backports-datetime-fromisoformat\\setup.py'"'"'; __file__='"'"'C:\\Users\\DELL\\AppData\\Local\\Temp\\pip-install-6dofct_j\\backports-datetime-fromisoformat\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\DELL\AppData\Local\Temp\pip-wheel-_h6wchyp' --python-tag cp36
cwd: C:\Users\DELL\AppData\Local\Temp\pip-install-6dofct_j\backports-datetime-fromisoformat\
Complete output (12 lines):
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.6
creating build\lib.win-amd64-3.6\backports
copying backports\__init__.py -> build\lib.win-amd64-3.6\backports
creating build\lib.win-amd64-3.6\backports\datetime_fromisoformat
copying backports\datetime_fromisoformat\__init__.py -> build\lib.win-amd64-3.6\backports\datetime_fromisoformat
running build_ext
building 'backports._datetime_fromisoformat' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/vi...
----------------------------------------
ERROR: Failed building wheel for backports-datetime-fromisoformat
ERROR: Command errored out with exit status 1:
command: 'c:\users\dell\anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\DELL\\AppData\\Local\\Temp\\pip-install-6dofct_j\\backports-datetime-fromisoformat\\setup.py'"'"'; __file__='"'"'C:\\Users\\DELL\\AppData\\Local\\Temp\\pip-install-6dofct_j\\backports-datetime-fromisoformat\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\DELL\AppData\Local\Temp\pip-record-eiwjaqe4\install-record.txt' --single-version-externally-managed --compile
cwd: C:\Users\DELL\AppData\Local\Temp\pip-install-6dofct_j\backports-datetime-fromisoformat\
Complete output (12 lines):
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.6
creating build\lib.win-amd64-3.6\backports
copying backports\__init__.py -> build\lib.win-amd64-3.6\backports
creating build\lib.win-amd64-3.6\backports\datetime_fromisoformat
copying backports\datetime_fromisoformat\__init__.py -> build\lib.win-amd64-3.6\backports\datetime_fromisoformat
running build_ext
building 'backports._datetime_fromisoformat' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/vi...
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\dell\anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\DELL\\AppData\\Local\\Temp\\pip-install-6dofct_j\\backports-datetime-fromisoformat\\setup.py'"'"'; __file__='"'"'C:\\Users\\DELL\\AppData\\Local\\Temp\\pip-install-6dofct_j\\backports-datetime-fromisoformat\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\DELL\AppData\Local\Temp\pip-record-eiwjaqe4\install-record.txt' --single-version-externally-managed --compile Check the logs for full command output.


message 19: by frando (new)

frando (broadishreads) | 11 comments Hi! I have not tried the installation on windows. It seems like there might be a compatibility issue with one of the dependencies on windows. I will take a look over the next couple of days. Thanks for letting me know.


message 20: by Hannu (new)

Hannu Kokko (hkokko) | 1 comments How can I get all owned books of a user. By using the default listofbooks = user.owned_books()
I get only five books or so.

listofbooks = user.owned_books(page=2)
causes an error message...

What am I doing wrong

using the latest better reads with python 3.7 on Mac


message 21: by frando (new)

frando (broadishreads) | 11 comments Owned books is a specific goodreads construct for when someone checks “I own this book” in their review. It creates a record for the condition and purchase location of the book.

What exactly are you trying to get? All of the books a user has read? All of the books a user has ever shelved?


message 22: by Shane (new)

Shane Phillips | 24 comments Does this wrapper support me getting all my shelves and or all books on one of my shelves?


message 23: by Shane (new)

Shane Phillips | 24 comments I am looking to extract all books on all my shelves into a database


message 24: by frando (new)

frando (broadishreads) | 11 comments Hi Shane,

You should be able to get all of the books and reviews from one shelf using this function. https://betterreads.readthedocs.io/en...

You can get all of the books associated with a user by running this function for each of their "exclusive" shelves, as each of their shelved books will belong to one of these. Let me know if you encounter any difficulties with this.

Be aware that if you have a shelf with more than a couple hundred books, it could take a minute or more to complete the operation.

As far as extracting all books and storing them in your own database, just make sure you're familiar with the developers terms of service :) https://www.goodreads.com/api/terms


message 25: by Shane (last edited Nov 03, 2019 06:37PM) (new)

Shane Phillips | 24 comments Thank you so much Jess. I used to program many years ago and now do dba work. I am learning python. I used to export goodreads to csv weekly and import into my personal db. Only my data for my personal analysis. I would like to make a python program that extracts the data into my db so that I can remove the manual step.


message 26: by Shane (new)

Shane Phillips | 24 comments Jess,

I am new to python. I got betterreads installed and I got connected to goodreads and was able to return a single book or user name.

however, I cannot get a list of shelves and their properties returned.
I have tried various print or for loops to access the shelves but no luck.
I am trying to get a comma separated list of 1 rows per shelf with all its properties.
gid, name, exclusive, ......

sample:
from betterreads import client
gc = client.GoodreadsClient('key', 'secret')
user = gc.user(user_id=10961320)
s = user.shelves
print (s[0])

any help or sample would be much appreciated.
Shane


message 27: by frando (new)

frando (broadishreads) | 11 comments Hi Shane.

It's been a little bit since I worked on this library, so pardon me if I ask some dumb questions.

Did you try using the Client's authenticate method to get user authentication and access their data? Some information in the Goodreads API is gated behind the user agreeing to allow your third party app to access it. I definitely need to update the documentation to reflect this.

Take a look at this section in the README and let me know if you need additional help. Also happy to chat via DM or via Github issues. Github is a bit easier since it supports code formatting.

https://github.com/thejessleigh/bette...


message 28: by Shane (last edited Dec 02, 2019 05:18PM) (new)

Shane Phillips | 24 comments Jess,

(posted in github too)
I apologize if I am just missing something as a new python user.
Your documentation for user:
https://betterreads.readthedocs.io/en...

I can user example and get user.user and user.user_name to work however nothing below your sample code works.
list_groups, owned_books, shelves, etc do not work.

here is test commands( name, link work, but shelves and list_groups do not):

>>> from betterreads import client
>>> gc = client.GoodreadsClient('key', 'secret key')
>>> user = gc.user(user_id=10961320)
>>> user.name
'Shane Phillips'
>>> user.shelves
Traceback (most recent call last):
File "", line 1, in
TypeError: __repr__ returned non-string (type int)
>>> user.link
'https://www.goodreads.com/user/show/1...'
>>> user.list_groups
Traceback (most recent call last):
File "", line 1, in
TypeError: __repr__ returned non-string (type int)


message 29: by frando (new)

frando (broadishreads) | 11 comments Hey Shane - I just looked at a recent pull request on the project and it looks as though some integration tests that were previously passing are now failing. I wonder if that has something to do with the issues you are seeing. It's possible something has changed in the API that I haven't accounted for. I'm very sorry - I've been only moderately available due to the holiday week last week. Let me poke around and try to figure out what's going on.


message 30: by Svemagie (new)

Svemagie | 5 comments Jess wrote: "... something has changed in the API..."
I pulled manually via json and they changed the API indeed. Sadly without notice (?)


message 31: by Svemagie (new)

Svemagie | 5 comments Um.. meant not json but xml


message 32: by Shane (new)

Shane Phillips | 24 comments thanks Sven. do you know what change? all methods or specific ones?


message 33: by Svemagie (last edited Jan 07, 2020 05:36AM) (new)

Svemagie | 5 comments Hi Shane (and all) - can answer not very precise, because I'm just starting dev. I'm on python, learning with and reusing code from qs_ledger (quantified self python tool).

It seems that a query on /user/show/ gave back a reviews_count, now that query breaks. Haven't looked into it before it broke, so can't tell what's the different return value now and then.

> user_info_url = get_user_info_url(api_key, user_id)
>user_info_data = urllib.request.urlopen(user_info_url).read()
>user_info_dict = xmltodict.parse(user_info_data)
>books_total = int(user_info_dict["GoodreadsResponse"]["user"]["reviews_count"]["#text"])
>user_name = user_info_dict["GoodreadsResponse"]["user"]["user_name"]
>user_info = {
> "user_name" : user_name,
> "books_total" : books_total
> }
>return(user_info)

Then I tried rewrite my code using the wrapper above, but to no success.

So I believe the API changed between 29/12/2019 (last correct query) and 2/2/2020.


message 34: by Shane (last edited Jan 08, 2020 10:03AM) (new)

Shane Phillips | 24 comments I am using "ElementTree" to extract the Goodreads API. However, I cannot find out how to get my list of shelves that I have a book on.
note: added a space so goodreads would not process xml.
-------sample----------
< spoilers_state>none
< shelves>
< shelf exclusive="false" id="153934772" name="full-list" review_shelf_id="2562838816" sortable="true"/>
< shelf exclusive="true" id="269364260" name="s-harry_bosch" review_shelf_id="2562838881" sortable="true"/>
< /shelves>
< recommended_for/>
---------end sample

I am not sure how to get the shelf name and shelf_id out of these tags.

any help?

API method:
https://www.goodreads.com/review/list...

thanks


message 35: by frando (new)

frando (broadishreads) | 11 comments I apologize for my lack of response on this. This is a side project that I just have not had the bandwidth to keep up to date. I am open to looking at PRs and I hope that I will have some time in the late winter/early spring to release an updated version that accounts for some of the reported errors.


message 36: by Svemagie (new)

Svemagie | 5 comments Jess, no hurry! Enjoy Winter and thanks for your work.


message 37: by Svemagie (new)

Svemagie | 5 comments my "review_counts" problem was simply the privacy settings on my profile page. odd, as I'm authenticated, but ok, it works again.


message 38: by Shane (new)

Shane Phillips | 24 comments Jess wrote: "I apologize for my lack of response on this. This is a side project that I just have not had the bandwidth to keep up to date. I am open to looking at PRs and I hope that I will have some time in t..."

No worries.


message 39: by Suraj (new)

Suraj R | 1 comments Anybody who's using python right now to authenticate, fetch user's book list, user ratings, average ratings, etc can check out my implementation. I've used a combo of the Goodreads python wrapper by Sefa and get requests to the Goodreads API itself.

https://github.com/suraj-ranganath/bookpal


message 40: by wout (new)

wout | 3 comments What is the best way right now to access the Goodreads API through Python? Would I need to look into @Suraj's solution or is there something that is already a bit more standardized so that I can incorporate it directly. The goal is eventually to build a Discord-bot that can call-up book info for our next bookclub.


message 41: by Shane (new)

Shane Phillips | 24 comments I use elementtree library. I wrote program (my first python) to download all my books to a sqlite db.

# Shane Phillips, 2019
# load goodreads.com book data to database via API
import requests, sqlite3, time, sys, getopt
from xml.etree import ElementTree
import os.path
from os import path
from datetime import datetime

I would be glad to share it. Nothing special about it. message me directly and I can email to you.


message 42: by wout (new)

wout | 3 comments Thanks, I'll take any help I can get, because I am rather new to this and getting a few early pointers do make the difference!


back to top