In this project, we are going to classify whether a given news is reliable or unreliable using the given data set
Data Source : https://www.kaggle.com/c/fake-news
Train DataSet
-
id: unique id for a news article
-
title: the title of a news article
-
author: author of the news article
-
text: the text of the article; could be incomplete
-
label: a label that marks the article as potentially unreliable
- 1: unreliable - 0: reliable
Test Dataset
- id: unique id for a news article
- title: the title of a news article
- author: author of the news article
- text: the text of the article; could be incomplete
- Classifications
- Logistic regression
- Decision tree
- KNeighbours
- Linear Discriminant
Given a title, author and text determine whether the news is reliable or unreliable (0 for unreliable and 1 for reliable).
With the help of given attributes we are going to combine all of the three important attributes i.e title, author and text into a single column so as to preserve all the important data and to make our data cleaning and preprocessing task much easier
- Merged all three attributes title, author and text into one new column total
- Filled Null Values
- Removed duplicate entries
- Text Cleaning - Removed Url, special characters, stopwords, and performed Snow Ball Steaming.
- Cleaned both test and train data
- Used Tf-Idf to convert text into vector and also used bigram
- Used different Machine Learning Models to find the one with the best accuracy
- Finally used Logistic Regression to predict our test data
- Saved the output in a CSV file
- Submitted on Kaggle with accuracy of 97%
No model can be perfect, so with a score of 97 % accuracy, I am pretty much surprised. But we also use some more turning to see if we can further increase our model accuracy.
