This project focuses on predicting passenger survival using machine learning models. The Titanic dataset was used for this analysis.
To ensure accurate model training, missing values in the dataset were carefully handled through deletion or imputation based on their proportion and significance. After analysis of the data, it was decided to:
- delete row: Embarked (0.2% missing), Fare(0.2% missing)
- delete column: Cabin (77.1-78.2% missing), Ticket (not important)
- imputate with KNN: Age (19.9-20.6% missing)
Imputation was done using KNN algorithm from Impute module.
To enhance the prediction accuracy, several new features were engineered from the existing data:
- FamilySize: Created by combining the SibSp and Parch columns.
- IsAlone: A binary feature indicating if a passenger is traveling alone.
- Title: Extracted from the Name column to capture social status.
These new features were added to the dataset using the functions defined in src/NewFeatures.jl.
A logistic regression model was implemented to provide a baseline for comparison with more complex models. Two methods for logistic regression were used, Newton and Gradient descent. In the graph, it can be seen that Newton performs slightly better.
Two decision trees models were chosen for the project:
- Random forest: constructs multiple trees independetly on a randomly selected subset of training data and features
- Gradient boosting trees: builds trees sequentially by training each new tree on residual errors made by previous trees
Both build and combine decision trees using Gini impurity as the criterion. According to this article, Gini impurity has a shorter training time compared to entropy, making it a more efficient choice for this project.
To install the survival prediction package, use the following command in the Julia REPL in project environemt:
add https://github.com/B0B36JUL-FinalProjects-2024/Project_sidlodo1.gitExample usage in examples/examples.jl:
include("src/Utils.jl")
using .Utils
# prepare data
path = joinpath(@__DIR__, "../data/train.csv") |> normpath
df = Utils.load_csv(path)
X_trn, y_trn, X_tst, y_tst = Utils.process_and_split_data(df; test_ratio=0.2)
include("SurvivalPrediction.jl")
using .SurvivalPrediction
SP = SurvivalPrediction
# run models
model_LR = SP.LR.LogRegModel(n_iters=100)
method_grad = SP.LR.GradientDescentMethod(0.01)
pred = SP.get_prediction(model_LR, method_grad, X_trn, y_trn, X_tst)
accuracy = Utils.classify_predictions(pred, y_tst)
model_RF = SP.RF.RandomForestModel()
SP.report_classification(model_RF, X_trn, y_trn, X_tst, y_tst)