Semi-Markov xT 360

A temporal expected-threat framework combining semi-Markov modeling, survival analysis, and StatsBomb 360 contextual data for football action valuation.

This repository studies whether time-to-threat and defensive context can improve temporal expected threat (xT). Traditional xT is mostly spatial: it values where the ball moves. This project keeps the semi-Markov xT structure from the reference paper, replaces the parametric sojourn-time component with XGBoost AFT survival modeling, and tests StatsBomb 360 pressure features when they are available. The model is evaluated on held-out international tournaments and on a Bundesliga 2023/24 split with real 360 data.

Contents

• Key Result
• Why This Project Matters
• Method Overview
• Project Architecture
• Quick Start
• Experimental Protocol
• Results
• Limitations
• Documentation

Key Result

Main held-out tournament benchmark:

Train: FIFA World Cup 2018 + UEFA Euro 2020 + FIFA World Cup 2022
Test : UEFA Euro 2024
Horizon: 10 seconds
Grid: 16 x 12

Model	AUC	Brier Score	LogLoss
Paper-style semi-Markov AFT, no 360	0.8437	0.005677	0.035611
Contextual semi-Markov xT, XGBoost AFT + 360	0.8586	0.005539	0.029966

The contextual model improved ranking performance while reducing probabilistic prediction error on the held-out Euro 2024 evaluation set. A controlled 2 x 2 ablation shows that the improvement is mainly driven by replacing the paper-style AFT survival component with XGBoost AFT. The 360 features were tested because the reference paper highlights missing off-ball/defensive positioning as a limitation and suggests conditioning future models on defensive geometry; in this benchmark, however, their incremental impact is marginal.

Across horizons T = {10, 20, 30, 60, 120, 300} seconds, the XGBoost AFT variants remain better than the paper-style AFT model: AUC gains stay positive at every horizon, and Brier Score improves consistently. The ablation notebook now also computes bootstrap confidence intervals for this multi-horizon comparison.

A second club-football check on Bundesliga 2023/24 with real StatsBomb 360 data showed the same direction of improvement and a substantially shorter training time:

Model	AUC	Brier Score	LogLoss	Training Time
Paper-style semi-Markov AFT, no 360	0.8468	0.008627	0.045436	655.9 s
Contextual semi-Markov xT, XGBoost AFT + 360	0.8577	0.008301	0.044095	68.4 s

Training-speed ratio on the Bundesliga split:

655.9 / 68.4 = 9.6x faster

For the complete experimental summary, see RESULTS.md.

Why This Project Matters

• Purely spatial xT ignores how long a team remains in a state before threat materializes.
• Semi-Markov modeling adds a temporal layer through sojourn-time estimation.
• StatsBomb 360 adds defensive context such as nearest-defender distance and local opponent density, which directly tests a future-work direction suggested by the reference paper.
• Brier Score and LogLoss matter because xT is used as a probability-like quantity and is often aggregated across actions, players, and teams.
• Faster survival modeling makes repeated experiments more realistic on larger football datasets.

Method Overview

The pipeline is intentionally close to the reference semi-Markov xT structure:

1. ingest StatsBomb event data and optional 360 freeze-frame context;
2. clean events and build possession sequences;
3. map actions into a 16 x 12 spatial grid and action states;
4. estimate transition and sojourn-time behavior;
5. train either paper-style AFT or XGBoost AFT survival models;
6. build temporal xT surfaces at a fixed horizon;
7. evaluate on held-out sequences using AUC, Brier Score, LogLoss, calibration plots, and bootstrap confidence intervals.

Detailed experimental notes are in EXPERIMENTS.md and RESULTS.md.

Project Architecture

flowchart LR
    a[StatsBomb event data] --> b[Event cleaning]
    a --> c[StatsBomb 360 context]
    b --> d[Possession sequences]
    c --> e[Defensive pressure features]
    d --> f[Spatial state construction]
    e --> g[XGBoost AFT survival]
    f --> h[Semi-Markov xT model]
    g --> h
    h --> i[Temporal xT surface]
    i --> j[Held-out evaluation]
    j --> k[Calibration, ablation, player ranking]

Loading

Repository Structure

semi-markov-xT-360/
|-- Class_Foot_xt.py                  # Semi-Markov xT model and evaluation logic
|-- build_dataset.py                  # StatsBomb loading, cleaning, sequence construction
|-- model_train.py                    # CLI training entry point
|-- player_xt_ranking.py              # Event-level xT aggregation into player views
|-- utility_function.py               # Shared evaluation helpers
|-- ablation_study.ipynb              # Tournament ablation, calibration, bootstrap CI
|-- championship_test.ipynb           # Club-football checks including Bundesliga 360
|-- final_results.ipynb               # Clean notebook with final public tables and plots
|-- test_model.ipynb                  # Held-out testing and player-ranking workflow
|-- RESULTS.md                        # Final result tables, plots, limitations
|-- EXPERIMENTS.md                    # Modeling branches tested and removed
|-- docs/assets/                      # Versioned calibration plots used in reports
|-- football_model/
|   |-- aft/                          # AFT utilities
|   |-- distribution/                 # Parametric sojourn distribution fitting
|   `-- pipeline/                     # Events, sequences, features, survival ML
|-- sim_modules/                      # Reference/support modules
|-- requirements.txt
|-- LICENSE
`-- README.md

The following local artifacts are intentionally ignored by Git:

dati/       # cached StatsBomb event data
csv/        # legacy/local cached data
models/     # trained .joblib models
results/    # generated evaluation outputs
mlruns/     # local MLflow runs
.venv/      # local virtual environment

Quick Start

Clone the repository:

git clone https://github.com/FEDERICOLANCINI/semi-markov-xT-360.git
cd semi-markov-xT-360

Create a virtual environment.

Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

macOS/Linux:

python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Train the current tournament model:

python model_train.py --competitions "{43: [3, 106], 55: [43]}" --model_name xgb_aft_360 --no_internal_test --survival_backend xgboost_aft --use_360 true

Train the paper-style tournament baseline:

python model_train.py --competitions "{43: [3, 106], 55: [43]}" --model_name paper_aft_no360 --no_internal_test --survival_backend paper_aft --use_360 false

Train the Bundesliga 360 split model:

python model_train.py --competitions "{9: [281]}" --model_name bundesliga_2023_360_xgb_aft --survival_backend xgboost_aft --use_360 true

The project does not currently expose a single end-to-end make command. The reproducible workflow is script-first for training and notebook-based for reporting.

Experimental Protocol

Main comparison:

Baseline: paper-style semi-Markov AFT without 360
Proposed model: semi-Markov xT with XGBoost AFT and optional StatsBomb 360 context
Metrics: AUC, Brier Score, LogLoss
Horizon: 10 seconds

Primary held-out benchmark:

Train: FIFA World Cup 2018 + UEFA Euro 2020 + FIFA World Cup 2022
Test : UEFA Euro 2024

Club-football 360 check:

Dataset: Bundesliga 2023/24 with StatsBomb 360
Split: match-level 80/20, seed=42
Train matches: 27
Test matches: 7

Ablation and calibration workflows are in ablation_study.ipynb and championship_test.ipynb. The tournament ablation isolates survival_backend and use_360, showing that XGBoost AFT is the main performance driver while 360 features are marginal on the current open-data tournament test. For experiment design notes, see EXPERIMENTS.md.

Results

Tournament calibration:

Bundesliga 360 calibration:

The semi-Markov models are more conservative than the timed Markov baseline and show better probabilistic error in the reported held-out settings. On the Bundesliga split, bootstrap confidence intervals still overlap, so the improvement should be described as consistent but not statistically decisive.

For the complete experimental analysis, see RESULTS.md.

Technical Stack

Language: Python
Data: pandas, NumPy, statsbombpy
Modeling: semi-Markov xT, survival analysis, lifelines, XGBoost AFT
Evaluation: scikit-learn, bootstrap confidence intervals, calibration analysis
Visualization: matplotlib, seaborn
Experiment tracking: MLflow support in model_train.py

Reproducibility

The training script exposes the core configuration through command-line arguments:

--competitions
--delta_t
--horizon
--train_size
--seed
--model_name
--survival_backend
--use_360
--no_internal_test
--mlflow

Current defaults include:

delta_t = 0.1
horizon = 10.0
train_size = 0.8
seed = 42

Data handling is cache-first: football_model/pipeline/events.py checks dati/ before downloading from StatsBomb. Trained models are written to models/, generated metrics to results/, and optional MLflow artifacts to mlruns/. These folders are ignored to keep the public repository lightweight.

The repository is reproducible at the code/protocol level, but not fully artifact-reproducible without downloading StatsBomb Open Data and re-training the models locally.

Limitations

• The project depends on StatsBomb Open Data availability and schema stability.
• Open-data football samples are limited and sometimes team-centered.
• StatsBomb 360 coverage is not available for every competition, and in the controlled tournament ablation the 360 covariates do not materially improve the XGBoost AFT model.
• The model keeps a discretized 16 x 12 pitch representation.
• The semi-Markov structure still relies on modeling assumptions about state transitions and sojourn times.
• Bundesliga confidence intervals overlap on the current 7-match test split.
• Player rankings measure attacking xT only; they do not include defensive value, off-ball movement, role adjustment, or possession responsibility.
• A stronger industry validation would require broader proprietary event and tracking data.

Documentation

• Detailed results
• Experimental branches and ablation notes
• Final public notebook
• Tournament ablation notebook
• Club-football benchmark notebook
• Training script
• Player ranking utilities

License and Data

This repository includes a LICENSE.

The project uses StatsBomb Open Data through statsbombpy. StatsBomb data is subject to the terms and licensing conditions of the StatsBomb Open Data project. This repository does not redistribute trained models, cached event data, or generated heavy artifacts.