Mireille's blog
Goofy things, code, data, and stuff worth keeping or sharing.
home / blog / datacode
Exporting your Good Reads list to Markdown

I decided to have my book list available in markdown, instead of using Good Reads, an Amazon service.
To do this, I exported my data from Good Reads as a csv file. Here is the link and a how to.

Once I had the csv, I wrote a script that:

  • creates a directory for each book and a markdown file;
  • fetches the book info from google books;
  • inserts some metadata about the books like pageCount, categories and tags;
  • inserts the book description as the content of the post;
  • fetches the cover (as a thumbnail);
  • if I had a review of the book on Good Reads, the script adds it after the book’s description.

Here is the code:

#!/bin/bash
loc=/path/goes/here/content/books/
# time to pause between fetches
# be nice to the google api and/or respect general API call limit
time_between_fetches = 3
INPUT=good-reads.csv
while IFS=, read -r nisbn myrating dateread dateadded bookshelf myreview
do
isbn=$nisbn
if [[ ! -z "$isbn" ]]
then
echo "================="
echo getting book with isbn number : "$isbn"
# fetch the json file from google books api
curl "https://www.googleapis.com/books/v1/volumes?q=isbn:$isbn" -o $loc/temp.json
# save into a temp json file
totalItems=`cat "$loc/temp.json" | jq '.totalItems'`
if [[ "$totalItems" -gt 0 ]]
then
# read properties from temp json file
# use jq -r to remove quotes
title=`cat "$loc/temp.json" | jq '.items[0].volumeInfo.title'`
description=`cat "$loc/temp.json" | jq '.items[0].volumeInfo.description'`
dat=`date -u`
publishedDate=`cat "$loc/temp.json" | jq '.items[0].volumeInfo.publishedDate'`
authors=`cat "$loc/temp.json" | jq '.items[0].volumeInfo.authors'`
pageCount=`cat "$loc/temp.json" | jq '.items[0].volumeInfo.pageCount'`
tags=`cat "$loc/temp.json" | jq '.items[0].volumeInfo.categories'`
categories="['books']"
imageLinks=`cat "$loc/temp.json" | jq '.items[0].volumeInfo.imageLinks.thumbnail'`
# read the book title and create a friendly directory name
# replace space with dash, lowercase
# remove double quotes, single quote, comma, question mark, and :
book=${title// /-}
book=$(echo $book | tr -d '"')
book=$(echo $book | tr -d "'")
book=$(echo $book | tr -d ',')
book=$(echo $book | tr -d '?')
book=$(echo $book | tr -d ';')
book=$(echo $book | tr -d ':')
book=$(echo $book | tr '[:upper:]' '[:lower:]')
# path of the book
book="$loc/$book"
# make the directory
mkdir -p "$book"
# save the details to book-name/index.md
echo -en "--- \n" > $book/index.md
echo -en "title: $title \n" >> $book/index.md
echo -en "date: $dateadded \n" >> $book/index.md
echo -en "datefetched: $dat \n" >> $book/index.md
echo -en "myrating: $myrating \n" >> $book/index.md
echo -en "bookshelf: $bookshelf \n" >> $book/index.md
echo -en "authors: $authors \n" >> $book/index.md
echo -en "pageCount: $pageCount \n" >> $book/index.md
echo -en "categories: $categories \n" >> $book/index.md
echo -en "tags: $tags \n" >> $book/index.md
echo -en "--- \n \n" >> $book/index.md
echo -en "$description" >> $book/index.md
chrlen=${#myreview}
echo "$chrlen"
if [[ ${#myreview} -gt 1 ]]
then
echo -en "\n --- \n" >> $book/index.md
echo -en "## My Review \n" >> $book/index.md
echo -en "$myreview \n" >> $book/index.md
echo -en "--- \n" >> $book/index.md
fi
# get the large image by replacing the zoom value
cover="${imageLinks/zoom=1/zoom=0}"
cover=`echo "$imageLinks" | tr -d '"'
# echo "$cover"`
curl "$cover" -o $book/cover.jpg
echo "taking a break"
sleep $time_between_fetches
fi
fi
done < $INPUT
# clean up the temp json file.
rm "$loc/temp.json"
view raw good-reads-to-markdown hosted with ❤ by GitHub

What you need

You need to have jq installed. Instructions are here.
If you are on macOS, it is brew install jq.
For ubuntu, it is sudo apt-get install jq.

How to run:

  • The file name is hardcoded to good-reads.csv. (I know that it should be an argument ;p). so rename your file to that and put it in the same directory as the script.
  • You need to manually drop some columns. Only keep what you see on line 3. Also drop the first line (column headers)

If I ever need to run the script again, i’ll improve on those last two points.