Bahn Delay Predictor: Will My Train Be Late?

A working, honestly-evaluated train-delay classifier built as a ProoV Deep Project. It predicts whether a German train will arrive more than 5 minutes late, using only signals knowable before the train departs, and reports the honest numbers, including where it still falls short.

About this ProoV project

This is a real, company-grade challenge from ProoV, an independent virtual work-experience platform. Read the full brief on projectstudy.in, and the detailed version in docs/BRIEF.md.

Not affiliated / illustrative. This is an independent, ProoV-owned educational case study. It uses real German open data (Deutsche Bahn punctuality data + Deutscher Wetterdienst weather observations) under nominative fair use, and it is not affiliated with, endorsed by, or sponsored by Deutsche Bahn AG or the Deutscher Wetterdienst. IBM Bob is a recommended AI coding tool; this project is not affiliated with IBM.

Results (real run, time-ordered split)

Metric	Baseline ("always on-time")	This model
Accuracy	89.9%	67.8%
Recall on "late"	0.00	0.65
Precision on "late"	0.00	0.18

The baseline never predicts a delay, so it can never catch one: recall 0 by construction. This model trades a lot of accuracy for the ability to actually catch roughly 6.5 in 10 late trains, which is the point of the exercise. See models/RESULTS.md for the full classification report and MODEL_CARD.md for what these numbers do and don't mean. Accuracy alone is a trap on this data and is never headlined here; see the threshold discussion in docs/FINDINGS_AND_DECISIONS.md.

What's actually in this repo

  LOAD + CLEAN ──▶ FEATURES ──▶ TRAIN ──▶ EVALUATE
  drops             leakage-     time-      beats-baseline
  cancellations,     free        ordered    check, recall/precision
  honest on-time     signals     split      on "late" headlined
  rate (90.0%)

• src/load_clean.py loads the delay data, drops cancelled trains (4.7% of rows) with a printed count report instead of silently counting them as on-time, and defines late_gt_5 = delay_in_min > 5 (Deutsche Bahn counts under 6 minutes as on time, so a 5-minute-late train is on time and a 6-minute-late train is late).
• src/features.py builds a leakage-free feature matrix: hour_of_day, day_of_week, train_type, station, temp_c/precip_mm/wind_kmh, and is_public_holiday (via the MIT-licensed holidays library). LEAKY_COLUMNS = {"actual_arrival", "delay_in_min"} is asserted absent from the feature matrix, twice: once here, once again in src/evaluate.py before any number is trusted.
• src/train.py sorts by date and cuts a time-ordered wall at 80% of the unique dates (2025-09-01 through 2025-11-12 train, 2025-11-13 through 2025-12-01 test, never a random split, which would leak the future into training), then fits a RandomForestClassifier with class_weight="balanced" (a deliberate choice: see docs/FINDINGS_AND_DECISIONS.md section 1 for why the unweighted default catches zero late trains).
• src/evaluate.py reports accuracy next to the baseline, recall and precision on "late", the full classification_report, and writes all of it to models/RESULTS.md.
• tests/ covers the target-rule edge case (5 minutes is on-time, 6 is late), cancellation handling, the leakage guard, and that the split is genuinely time-ordered, not shuffled.

Data attribution

• Data source: Deutsche Bahn AG, licensed under CC BY 4.0 (processed/aggregated data; not endorsed by Deutsche Bahn AG)
• Source: Deutscher Wetterdienst (weather observations; CC BY 4.0 / GeoNutzV)
• Public holidays via the MIT-licensed holidays Python library.

Not affiliated with Deutsche Bahn AG. This is an independent educational case study using real, license-clean open data under nominative fair use. It is not affiliated with, endorsed by, or sponsored by Deutsche Bahn AG or the Deutscher Wetterdienst.

Team

Name	GitHub handle	Role on this build	What I contributed
Jonas Weber	@jonasweber	everything, solo run	Built the full pipeline end to end: load_clean.py's cancellation handling and target definition, features.py's leakage-free feature set and holiday flag, train.py's time-ordered split and classifier, evaluate.py's honest metrics, the test suite, and the model card and brief. Caught and overrode two mistakes my AI agent made: a proposed shuffled train_test_split, and actual_arrival_time slipping into the feature matrix.

Getting started

This is a Python project. From the repo root:

# 1. Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

# 2. Install dependencies
pip install -r requirements.txt

# 3. Data: BAHN_DATA points the pipeline at any CSV pull; it defaults to
#    data/bahn_delays.csv if unset. See data/SOURCES.md for where the real
#    Deutsche Bahn + DWD data comes from.
export BAHN_DATA=data/bahn_delays.csv   # or your own real pull

# 4. Run the pipeline module by module:
python -m src.load_clean   # LOAD + CLEAN: drops cancellations, honest on-time rate (90.0%)
python -m src.features     # FEATURES: leakage-free feature matrix
python -m src.train        # TRAIN: RandomForest on the time-ordered split
python -m src.evaluate     # EVALUATE: recall/precision on "late", writes models/RESULTS.md

# 5. Run the tests
pytest

All four stages run clean from a fresh venv and all 12 tests pass.

Where the data lives

File	What it is
data/bahn_delays.csv	The real slice this repo was trained and evaluated on (see data/SOURCES.md for provenance and license). Not committed to this fixture copy of the repo to keep the working tree light; see NOTES.md.
data/SOURCES.md	Where the real dataset comes from (Deutsche Bahn open punctuality data on Hugging Face, CC BY 4.0, plus Deutscher Wetterdienst weather, CC BY 4.0) and how to cut a workable slice.
data/sample_slice_schema.md	The exact column schema load_clean.py expects.
data/processed/	The cleaned + feature-ready slices this repo actually trained on (cleaned.csv, features.csv).

Trained model artifact: models/late_predictor.joblib (a RandomForestClassifier, ~small enough to commit). Full evaluation output: models/RESULTS.md. Written brief: docs/FINDINGS_AND_DECISIONS.md.

AI agent usage

I directed Claude Code to build most of this repo module by module, reading README.md, docs/BRIEF.md, and data/SOURCES.md first. It made two mistakes I had to catch and override: it first proposed a shuffled sklearn.train_test_split for time_ordered_split(), and in an early draft of features.py it included actual_arrival_time as a feature because it correlated almost perfectly with the target. Both are documented with what I changed in docs/FINDINGS_AND_DECISIONS.md, section 3.

---

Built as a ProoV Deep Project · case study on German rail punctuality · real open data under nominative fair use · not affiliated with Deutsche Bahn AG or IBM.