Book recommendation system

Book recommendation system, using the content based filtering method and MapReduce.

Description

The goal of this project is to provide book recommendations to specific users. For this we have a dataset containing book ratings of a set of users, and a dataset containing the main characteristics of a set of books. To tailor personalized recommendations to the users we used the content based filtering method.

In summary, the program works with following 4 steps:

1. Convert book descriptions to a set of the top n TFIDF-scoring words.
2. Construct item profiles for all books.
3. Construct user profiles for a set of users, based on read books and ratings.
4. Recommend a set of books to a subset of users using cosine similarity.

Built With

This project was build using the following technologies:

• Python
• Pandas
• NumPy
• mrjob
• NLTK
• spaCy
• ProgressBar

Make sure to have Python (version 3.x) and the above libraries installed. If not, run the following from a terminal:

pip install pandas
pip install numpy
pip install mrjob
pip install nltk
pip install spacy
pip install progressbar

Data

We used the Goodreads Datasets from UCSD. Below are the links to the dataset to run the project. Save the datasets to a folder called data.

Mandatory

To prevent having to run the data preprocessing step, we have exported our preprocessed data set:

• Dataset of books with all of their characteristics: books_metadata.csv
• Dataset of all user ratings for the book they read: user_book_ratings.csv

Optional

To run the preprocessing steps, it will require downloading the original dataset. Below are the links to access the original dataset:

• List of books with their title and other partial metadata: books_titles.json
• Every rating of every user for different books: goodreads_users_book.csv
• Mapping between book ids in the json and csv files: book_id_map.csv
• Books full metadata (this data does not come from Goodreads, it needs to be mapped to the other dataset to be used): Best_Books_Ever.csv

Run the project

To run the project, follow the steps below after downloading the required dataset and saving it to a directory called data.

1. (Optional) If you have downloaded the optional dataset, then you can run the DataPreprocessing.ipynb file which preprocesses the dataset (make sure the dataset is saved to a directory called data).

2. • Before running this step, ensure that you have a directory nammed data containing books_metadata.csv user_book_ratings.csv (either download them directly or by running step 0).
   • Run the tfidf.ipynb file to extract the most significant words in books description.
3. • Before running this step, ensure that you have a tfidf_scores file in the current directory.
   • Run the ItemAndUserProfiles.ipynb file to create and store item and user profiles.
4. • Before running this step, ensure that you have an item_profiles.txt and user_profiles.txt file in the current directory.
   • (Optional) Run the first cell of Recommendation.ipynb to get a random user
   • (Optional) In the second cell, set the id of the user you want to get recommendation for: recommendation_user_id variable
   • Run all cells of Recommendation.ipynb to get recommendations for a specific user.

Sources

• Mining of Massive Datasets. Book by Jure Leskovec, Anand Rajaraman and Jeffrey D. Ullman, 2019
• Content-based recommendation algorithms on the Hadoop MapReduce framework. Paper by T. D. Pessemier, K. Vanhecke, S. Dooms, and L. Martens