Stellar Classification

A solution to Kaggle's Playground Series S6E6 competition: classifying sky objects observed by SDSS17 as GALAXY, QSO (quasar), or STAR from photometric and spectroscopic features. The competition data is synthetic, generated from the real SDSS17 stellar classification dataset.

Project structure

stellar-class/
├── data/
│   ├── raw/            # original train/test/sample_submission CSVs (gitignored)
│   └── processed/      # engineered feature sets (gitignored)
├── notebooks/
│   ├── 01_eda.ipynb                  # exploratory data analysis
│   ├── 02_feature_engineering.ipynb  # color indices, redshift transform, KNN density features
│   ├── 03_baseline_model.ipynb       # baseline LightGBM with stratified CV
│   ├── 04_tuning.ipynb               # Optuna hyperparameter search
│   ├── 05_ensemble.ipynb             # final tuned LightGBM ensemble
│   └── 06_advanced_experiments.ipynb # multi-model stacking and leakage experiments
├── requirements.txt
└── README.md

Approach

1. Feature engineering — SDSS photometric color indices (u_g, g_r, r_i, i_z), a log1p transform of redshift, and KNN-based spatial density features computed from sky coordinates.
2. Baseline modeling — a stratified 5-fold LightGBM baseline to establish a performance floor.
3. Hyperparameter tuning — Optuna search over LightGBM parameters, optimizing multiclass log loss.
4. Ensembling — a tuned LightGBM ensemble trained across 5 stratified folds, blended for the final submission.
5. Advanced experiments — additional LightGBM/XGBoost/CatBoost models combined via out-of-fold prediction stacking with a logistic regression meta-learner, alongside a deliberate investigation into target leakage (testing whether sky-position columns and other near-target features artificially inflate performance).

Results

The final tuned LightGBM ensemble (05_ensemble.ipynb) achieves:

• OOF accuracy: 0.954
• CV log loss: 0.122 (± 0.001 across folds)

Setup

pip install -r requirements.txt

Place the competition's train.csv, test.csv, and sample_submission.csv in data/raw/, then run the notebooks in order (01 → 06).