This project implements a full-fledged sentiment analysis system for Amazon product reviews. Reviews are ethically scraped, processed through a robust data mining and preprocessing pipeline, and analyzed using both rule-based NLP techniques and machine learning models.
The objective is to classify customer opinions into Positive, Neutral, and Negative sentiments and derive actionable insights regarding customer satisfaction.
Case Study: Philips Vacuum Cleaner (Amazon Reviews)
- Ethically scrape Amazon product reviews
- Perform data mining and text preprocessing
- Apply multiple sentiment analysis techniques
- Compare rule-based and ML-based approaches
- Visualize sentiment trends and insights
Amazon Reviews
↓
Ethical Web Scraping
↓
Data Cleaning & Preprocessing
↓
Sentiment Analysis
(Rule-Based & ML-Based)
↓
Visualization & Insights
- Python
- Selenium – Dynamic content scraping
- BeautifulSoup – HTML parsing
- Pandas, NumPy – Data processing
- TextBlob – Rule-based sentiment analysis
- VADER – Lexicon-based sentiment analysis
- Scikit-learn – Machine learning models
- TF-IDF Vectorization
- Logistic Regression
- Matplotlib, Seaborn, WordCloud – Visualization
- Reviews scraped ethically from Amazon product pages
- Product identified using ASIN
- Pagination handled programmatically
- Duplicate reviews removed
- Final dataset stored in a structured format
- Review ID
- Review Title
- Rating
- Review Text
- Author
- Date
- Lowercasing and whitespace normalization
- Removal of special characters and noise
- Stopword removal (negations preserved)
- Lemmatization for ML-based models
- Cleaned data stored separately for analysis
- Uses predefined sentiment lexicons
- Computes polarity and subjectivity scores
- Serves as a fast baseline model
Limitations
- No domain learning
- Weak handling of sarcasm and context
- Designed for informal, opinion-rich text
- Handles negation, capitalization, and intensity
- Uses compound sentiment scoring
Advantages
- More accurate than basic lexicon methods
- Well-suited for customer reviews
- Supervised sentiment classification
- TF-IDF converts text into numerical features
- Hyperparameter tuning improves performance
Advantages
- Learns sentiment patterns from data
- Better generalization and accuracy
Limitations
- Requires labeled data
- Limited semantic understanding
- Sentiment distribution bar charts
- Pie charts for sentiment proportions
- Word clouds for key opinion terms
- Comparative analysis across methods
- Majority of reviews are positive, indicating strong customer satisfaction
- VADER outperforms basic rule-based methods
- Logistic Regression with TF-IDF delivers the best overall performance
- Negative sentiment is minimal compared to positive and neutral reviews
- Python 3.8 or higher
- Google Chrome (for Selenium)
- ChromeDriver (matching Chrome version)
- Clone the repository
git clone https://github.com/AayushCharde/Amazon-Sentiment-analysis.git
cd Amazon-Sentiment-analysis