Docs -> Contains Project Report and Presentation Slides
SVM_Rating -> Final SVM training and testing models
Dataset -> Contains the Data used in this project
Examples -> Keras Sentimental Analysis Implementation
Models -> This folder contains various models implemented for the project.
SVM_KNN_NB.ipynb - Implementation of our approach
models.py -> Paper Implementation
Kindly go through the report documents for a better understanding.
Topic of project: Predicting Movie Ratings Based on Reviews
Literature Survey: http://aclweb.org/anthology/Y07-1050 : In this paper they have conducted various experiments on different classifiers like SVM, Maximum Entropy and Scoring which resulted in poor performance. Then they have collaborated all the three models which significantly increased the performance.
Data Extraction:
• Download the IMDB dataset from Kaggle (https://www.kaggle.com/orgesleka/imdbmovies/version/1)
• Perform Data Cleaning (missing attributes/ misplaced data)
• From the CSV file we can extract the movie title id for each given movie.
• Frame a url https://www.imdb.com/title/tt0032976/reviews.
For example, title id of movie Rebecca is tt0032976.
• Using web scrapping, we can retrieve the User Reviews section from the above mentioned url.
Problem Categorization: It is a classification problem. + Regression problem
Success Metric: Accuracy, Recall, Precision, F1 Score
Feature Extraction: Sentence to Vector/ Paragraph to Vector
Model Selection/ Choice: Based on empirical experiments
Validation and Testing: We can tune the parameters based on the validation set and calculate accuracy on the test data
Expectations
- Need to implement the research paper (SVM, Maximum Entropy,Scoring)
- Need to implement using Deep Learning Techniques
- Need to extend it to Indian Movies
- Categorize the rating (Direction, Music, Action etc..)
Scrapping.py This code read the imdb.csv dataset. It extracts the comments using webscrapping ( using BeautifulSoup, urllib) from the imdb reviews page for each movie. Stores all the user comments for a given movie into a txt file. For 14K movies in the imdb data set, it creates 14K text files. The sample output is uploaded tt0012349.txt