A machine learning study for predicting student dropout and academic success using classification models, feature selection, hyperparameter optimization, and ensemble learning.
The project was developed as part of CENG 484 — Data Mining at İzmir Institute of Technology.
The study evaluates a broad range of machine learning models on the Predict Students' Dropout and Academic Success dataset from the UCI Machine Learning Repository.
Student dropout prediction is an important educational data mining problem.
Identifying students who may be at risk of dropping out can potentially support earlier intervention and better allocation of academic support resources.
This project explores the problem as a binary classification task:
Graduate → 1
Dropout → 0
Students with the Enrolled target label are excluded from the binary classification experiments.
The project includes:
- exploratory data analysis
- Φk correlation analysis
- feature selection
- outlier analysis
- data preprocessing
- classification with multiple machine learning algorithms
- GridSearchCV hyperparameter optimization
- Optuna hyperparameter optimization
- stacking ensemble models
- confusion matrix analysis
- ROC-AUC evaluation
- permutation feature importance
- 10-fold stratified cross-validation
The project uses the Predict Students' Dropout and Academic Success dataset from the UCI Machine Learning Repository.
Dataset page:
https://archive.ics.uci.edu/dataset/697/predict+students+dropout+and+academic+success
The original dataset contains:
4,424 records
36 input attributes
1 target variable
The original target contains three classes:
Graduate
Dropout
Enrolled
For this study, only Graduate and Dropout observations are retained.
After removing the Enrolled observations, the binary dataset contains:
3,630 records
The overall workflow can be summarized as:
UCI Student Dataset
│
▼
Binary Target Selection
Graduate / Dropout
│
▼
Exploratory Data Analysis
│
▼
Φk Correlation Analysis
│
▼
Feature Selection
│
▼
Outlier Analysis
│
▼
Train / Test Split
│
▼
Machine Learning Models
│
├── Standard Models
├── GridSearchCV
├── Stacking
└── Optuna
│
▼
Evaluation
Feature relationships with the target were investigated using Φk correlation.
Features with:
Φk ≥ 0.4
were selected for the modeling stage.
A total of 11 features were retained.
The strongest relationships with the target included academic performance variables such as:
- Curricular units 2nd semester approved
- Curricular units 2nd semester grade
- Curricular units 1st semester approved
- Curricular units 1st semester grade
- Tuition fees up to date
- Scholarship holder
- Curricular unit evaluations
- Age at enrollment
- Application mode
- Debtor status
The complete correlation structure is visualized below.
Several exploratory analyses were performed before model training.
These include:
- feature histograms
- boxplots
- bivariate analysis
- target relationships
- correlation analysis
- outlier detection
The project applies the 1.5 × IQR rule to selected numerical features.
After the IQR-based filtering step:
Before filtering: 3,630 samples
After filtering: 1,606 samples
Removed: 2,024 samples
Because this is a substantial reduction in the dataset, its effect on class balance and model evaluation should be considered when interpreting the results.
The notebook evaluates a broad collection of classification algorithms.
- Neural Network / MLP
- Logistic Regression
- K-Nearest Neighbors
- Support Vector Machine
- Decision Tree
- Random Forest
- AdaBoost
- Gradient Boosting
- XGBoost
- Bagging Classifier
GridSearchCV is used for several models, including:
- XGBoost
- Random Forest
- Decision Tree
- Bagging Classifier
Additional experiments use Optuna for hyperparameter optimization.
Two stacking strategies are evaluated.
The first stacking model combines boosting-based learners:
AdaBoost
│
Tuned AdaBoost
│
Gradient Boosting
│
▼
Logistic Regression
The second stacking architecture combines multiple model families:
Random Forest
│
Gradient Boosting
│
XGBoost
│
AdaBoost
│
▼
Logistic Regression
The objective is to combine complementary decision boundaries from different base learners.
The strongest models in the main train/test experiment include:
| Model | Accuracy | Precision | Recall | F1 Score | AUC ROC |
|---|---|---|---|---|---|
| Random Forest | 92.53% | 93.01% | 99.07% | 95.95% | 76.20% |
| Combo Stacking Classifier | 92.53% | 93.01% | 99.07% | 95.95% | 76.32% |
| Tuned Random Forest | 92.32% | 93.00% | 98.84% | 95.83% | 75.20% |
| Tuned Bagging Classifier | 91.91% | 92.59% | 98.84% | 95.61% | 76.73% |
| AdaBoost | 91.70% | 92.58% | 98.60% | 95.50% | 77.74% |
| XGBoost | 91.49% | 92.94% | 97.91% | 95.36% | 75.27% |
The complete comparison is visualized below.
Selected models were also evaluated using stratified 10-fold cross-validation after Optuna-based tuning.
| Model | Accuracy | Precision | Recall | F1 Score | AUC ROC |
|---|---|---|---|---|---|
| Random Forest | 92.09% | 92.79% | 98.81% | 95.71% | 75.06% |
| Stacking Classifier | 92.03% | 92.51% | 99.09% | 95.69% | 78.46% |
| Bagging Classifier | 91.97% | 92.79% | 98.67% | 95.64% | 77.98% |
| Combo Stacking Classifier | 91.72% | 92.43% | 98.81% | 95.51% | 75.42% |
| XGBoost | 91.16% | 92.78% | 97.70% | 95.17% | 74.04% |
| Decision Tree | 90.97% | 92.76% | 97.49% | 95.06% | 72.06% |
The results show that Random Forest provides the highest cross-validated accuracy among these models, while the Stacking Classifier achieves the strongest AUC ROC.
Overall accuracy does not tell the complete story for this dataset.
After preprocessing, the dataset is strongly imbalanced toward the Graduate class.
For example, the Random Forest confusion matrix shows:
Predicted
Dropout Graduate
Actual Dropout 20 32
Actual Graduate 4 426
This corresponds to approximately:
Dropout Recall: 38%
Graduate Recall: 99%
Therefore, the high overall accuracy should not be interpreted as equally strong performance for both classes.
The model performs very well at identifying graduates but has substantially more difficulty identifying students who drop out.
This is an important limitation of the current modeling pipeline.
Feature importance was investigated using both built-in model importance values and permutation importance.
Across multiple models, academic progress variables consistently appear among the most informative features.
For Random Forest, the strongest features include:
- Curricular units 2nd semester approved
- Curricular units 2nd semester grade
- Curricular units 1st semester grade
- Curricular units 1st semester approved
- Curricular units 1st semester evaluations
- Curricular units 2nd semester evaluations
- Age at enrollment
Similar feature importance analyses were also performed for XGBoost and the stacking models.
Two optimization strategies are explored.
Grid search is used to systematically evaluate predefined parameter combinations for several classifiers.
Optuna is used to perform more flexible hyperparameter optimization for selected models.
Models optimized with Optuna include:
- Random Forest
- XGBoost
- Decision Tree
- Bagging Classifier
- Stacking Classifier
- Combo Stacking Classifier
The optimized models are subsequently evaluated using stratified 10-fold cross-validation.
Models are evaluated using:
Accuracy
Precision
Recall
F1 Score
ROC-AUC
Confusion Matrix
Because the final dataset is imbalanced, the confusion matrix and class-specific performance are particularly important when interpreting the results.
student-dropout-prediction-ml/
│
├── CENG484_Project_300201123.ipynb
│
├── figures/
│ ├── all_models_comparison.png
│ ├── bivariate_analysis.png
│ ├── boxplots.png
│ ├── confusion_matrix_rf.png
│ ├── confusion_matrix_stacking.png
│ ├── feature_importance_combo_stacking.png
│ ├── feature_importance_rf.png
│ ├── feature_importance_stacking.png
│ ├── feature_importance_xgboost.png
│ ├── histograms.png
│ ├── phik_correlation_matrix.png
│ └── ...
│
└── README.md
Clone the repository:
git clone https://github.com/behicekadioglu/student-dropout-prediction-ml.git
cd student-dropout-prediction-mlDownload the dataset from the UCI Machine Learning Repository:
https://archive.ics.uci.edu/dataset/697/predict+students+dropout+and+academic+success
Then open:
CENG484_Project_300201123.ipynb
using Jupyter Notebook, JupyterLab, VS Code, or another compatible environment.
The notebook uses libraries including:
numpy
pandas
matplotlib
seaborn
scikit-learn
xgboost
phik
optuna
A typical installation can be performed with:
pip install numpy pandas matplotlib seaborn scikit-learn xgboost phik optuna- Scikit-learn
- XGBoost
- Ensemble Learning
- Stacking
- Hyperparameter Optimization
- Pandas
- NumPy
- Φk Correlation
- Exploratory Data Analysis
- Matplotlib
- Seaborn
- GridSearchCV
- Optuna
The experiments highlight several observations:
- Random Forest achieved the highest evaluated train/test accuracy at 92.53%, tied with the Combo Stacking Classifier.
- Random Forest also achieved the strongest accuracy among the evaluated 10-fold cross-validation models at 92.09%.
- The Stacking Classifier achieved the strongest cross-validated AUC ROC at 78.46%.
- Academic progress variables were consistently among the strongest predictors.
- Overall accuracy is high, but performance on the minority
Dropoutclass remains substantially weaker. - Class imbalance should therefore be considered when interpreting the reported results.
Several limitations are important when interpreting this project.
The processed dataset contains substantially more graduates than dropouts.
As a result, overall accuracy and standard recall values can hide weaker performance on the dropout class.
The IQR-based filtering process removes a large portion of the binary dataset.
This changes both dataset size and class distribution and may influence model performance.
Features are selected using their Φk relationship with the target before the modeling stage.
For a stricter evaluation setup, supervised feature selection should ideally be performed independently within the training folds of the evaluation pipeline.
Machine learning predictions in educational contexts should not be treated as deterministic judgments about individual students.
Any practical use would require additional validation, fairness analysis, and appropriate human oversight.
This project was developed as part of CENG 484 — Data Mining coursework at İzmir Institute of Technology.
The goal was to investigate a real-world classification problem through:
- exploratory data analysis
- feature selection
- model comparison
- hyperparameter optimization
- ensemble learning
- and performance evaluation
The repository is presented as an academic machine learning study rather than a production student-risk prediction system.
Behice Kadıoğlu
Computer Engineering
İzmir Institute of Technology
GitHub: @behicekadioglu
The dataset used in this project is available through the UCI Machine Learning Repository under the title:
Predict Students' Dropout and Academic Success
Please refer to the original dataset source for its authorship, citation information, and usage terms.