Documentation for the machine learning pipeline built during an internship at Institut Pasteur.
This pipeline tests machine learning models on the task f(x) = y, where x is a vector of numerical features and y is a continuous or binary variable.
- Regression and classification support
- Linear and non-linear models, including deep neural networks
- Hyperparameter optimization and cross-validation
- Logging and feature attribution (SHAP) support
- Fully configurable via a single YAML file
- Extendable to new datasets, models, and metrics
┌───────────┐ ┌──────────────┐ ┌───────────────┐ ┌──────────┐ ┌────────┐
│ 1. │──▶│ 2. │──▶│ 3. │──▶│ 4. │──▶│ 5. │
│ Data │ │ Data │ │ Model │ │ Logging │ │ SHAP │
│ curation │ │ loading │ │ optimization │ │ │ │ │
└───────────┘ └──────────────┘ └───────────────┘ └──────────┘ └────────┘
raw GWAS → trainable → X (matrix), → trained → per-
SNP × phenotype matrix on y (vector) model(s), feature
matrix disk curves + attribution
metrics
One config file drives all five stages — each stage is independently toggleable.
git clone git@github.com:LeonAckermann/TabXPlainer.git
cd TabXPlainer
python3 -m venv tabxplainer && source tabxplainer/bin/activate
pip install -r requirements.txt
# interactive wizard → writes experiments/<name>.yaml
python configure.py
# run it
python main.py --config experiments/<name>.yamlJoin per-phenotype GWAS files into one matrix, then LD-clump the target phenotype's own signal. One-time work per illness/phenotype panel — skip this section if the dataset already exists.
┌─────────────────┐ ┌───────────────┐ ┌────────────────┐ ┌───────────────┐ ┌───────────┐
│ per-phenotype │──▶│ join │──▶│ LD-clump │──▶│ sample │──▶│ trainable │
│ GWAS files │ │ phenotypes │ │ target signal │ │ (optional) │ │ matrix │
└─────────────────┘ └───────────────┘ └────────────────┘ └───────────────┘ └───────────┘
./imputed_GWAS/ on a shared independent, prune sparse /
22 files/phenotype key (chrom, significant SNPs redundant
+ info.csv pos) features
(which to include)
gwas_phenotype_construction:
run: true
input_path: "<dir of z_<phenotype>.txt>"
phenotype_clumping:
run: true
phenotypes: ["target_phenotype"]
ref: ".<dir of 1000G reference panel>"
p_clump: 1
r2: 0.05
clump_kb: 500
data:
sampling: true
p_clump: [0.001]data/
├── pipeline/
│ ├── input/gwas_pheno/
│ │ all_z_scores.txt, info.csv
│ ├── intermediate/
│ │ aligned_{phenotype}.txt
│ └── output/
│ clumped_{phenotype}.clumps
└── sampled/{distribution}/
sampled_{phenotype}_p{p}.txt
Orchestrated from main.py and implemented in dataloader/pipeline.py. Two curation paths exist, depending on the input files — the join/clump flow above is Path B (a multi-phenotype panel); Path A curates a single illness against its own MRI phenotype file:
| Path A | Path B | |
|---|---|---|
| Scope | One illness at a time | Many phenotypes in one joined matrix |
| Config keys | construct_gwas_mri, plink2 |
gwas_phenotype_construction, phenotype_clumping |
→ documentation/1_data_curation.md for the full diagrams and complete config examples, or straight to the per-path detail: Path A / Path B.
Loads whatever the curation stage (or an already-existing dataset) produced, resolves the target column, and applies any configured transforms to produce a clean X/y matrix ready for training.
┌──────────────┐ ┌─────────┐ ┌────────────┐ ┌──────┐
│ load table │ │ resolve │ │ optional │ │ X, y │
│ (path / dir) │──▶│ target │──▶│ transforms │──▶│ │
└──────────────┘ └─────────┘ └────────────┘ └──────┘
Optional transforms: whitening · residualization · PCA · invertible PCA · max_target_corr
data:
dir: "data/my_datasets/" # one experiment per file in the folder
target_regex: "^Z_scores_" # matched against each file's columns
whitening: {enabled: true}
residual: true
pca: 0.95
invert: true
max_target_corr: 0.8There are two versions of this stage, chosen automatically from the config:
┌─ data.path / data.dir set? ──yes──▶ generic path
data.* config ───────────────┤
└─ no ──▶ specialized GWAS phenotype-panel path
The simple case: point the pipeline straight at one prepared tabular file, or a directory of them (one experiment per file), and name the target column (exactly, or by regex). Every GWAS-specific knob (illness, p_clump, distribution, gwas_pheno_path, whitening, …) is automatically ignored on this path, and every other setting defaults to something sensible.
→ documentation/2_data_preprocessing_generic.md for the full flow diagram, every default value, and a caveat about the binary-classification target conversion (it assumes a Z-score-shaped target — see that doc before using binary_classification on a non-GWAS target).
The full-featured path, used whenever data.path/data.dir are not set: phenotype/illness auto-detection, on-the-fly SNP sampling (with feature-density and target-correlation pruning), significance-based row/column filtering, same-trait-category feature exclusion, Σ-based whitening, and an out-of-fold linear-regression residual target.
→ documentation/2_data_preprocessing_specialized.md for the full flow diagram, every config key, and a full example data: section.
┌──────────┐
│ model(s) │
│ selected │
└──────────┘
│
┌────────────┴─────────────┐
▼ ▼
┌────────────┐ ┌───────────────────┐ ┌────────────────────┐ ┌──────────────────────────┐
│ cross-fold │ │ nested cross-fold │ │ outer CV │ │ HPO search │
│ validation │ │ validation │──▶│ train / test split │──▶│ on train fold (inner CV) │
└────────────┘ └───────────────────┘ └────────────────────┘ └──────────────────────────┘
6 supported models: linear · lasso · ridge · xgboost · residual_dnn · tabpfn
model:
names: [xgboost, tabpfn]
hpo:
run: true
n_trials: 30 # per-model override: n_trials_by_model
outer_cv: 5
inner_cv: 5
sampler: tpe # tpe | random | cmaesOne generic dispatch point, build_model() in src/hpo.py, instantiates every model from a model_name + params dict. Every model goes through the same choice: evaluate with plain cross-fold validation, or with nested cross-fold validation that runs its own hyperparameter search inside each outer fold.
| Step | What happens | Config |
|---|---|---|
| 1. Select model(s) | Choose which model(s) to run in this job | model.name (single) or model.names (list) |
| 2. Choose evaluation mode | Per model: plain cross-fold validation, or nested cross-fold validation with hyperparameter optimization | hpo.run: false (or no hpo: block) → plain CV; hpo.run: true → nested CV + HPO |
| 3a. Cross-fold validation | Train once per outer fold, using the config-file (or default) hyperparameters | n_trials = 0 (automatic when HPO is off, or when a model has no default search space) |
| 3b. Nested cross-fold validation | Each outer fold runs its own inner-fold Optuna search first, then trains + evaluates with the best hyperparameters found for that fold | hpo.n_trials, hpo.inner_cv |
| 4. Aggregate | Per-fold metrics are averaged (mean/std) across outer folds | → 4. Logging |
| Model | model.name |
Status |
|---|---|---|
| Linear Regression | linear |
implemented |
| Lasso | lasso |
implemented |
| Ridge | ridge |
implemented |
| XGBoost | xgboost |
implemented |
| Residual DNN | residual_dnn |
implemented |
| TabPFNv3 | tabpfn |
implemented |
TabPFNv3is the model behind thetabpfnconfig name (the shipped checkpoint ismodel/tabpfn-v3-regressor-v3_*.ckpt). This table is the primary, curated model set — a couple of extra models still exist in the code (e.g.dnn,mdn) but aren't part of the current supported set; see documentation/3_adding_a_new_model.md for the full, current list.
Each fold trains once, with no inner hyperparameter search — the simple, cheap default for models with few or no real hyperparameters.
→ documentation/3_cross_fold_validation.md — diagram, config example, and when to use it.
Each outer fold runs its own inner-fold hyperparameter search before being scored, so reported metrics are never inflated by hyperparameters chosen using the same data they're evaluated on.
→ documentation/3_nested_cross_fold_validation.md — diagram, config example, and how the inner-fold Optuna search and parameter pinning work (§3.2.1).
→ documentation/3_model_hyperparameters.md — the tunable hyperparameters and default search range for every model above (§3.2.2).
model.names (list) loops over model families in the same job, alongside every other axis (task type, phenotype, noise level, …):
model:
names: [linear, xgboost, tabpfn]
overrides:
tabpfn:
finetune: true
learning_rate: 1.0e-5model.overrides.<name> scopes model-specific extra parameters (like TabPFN's finetune/learning_rate) to that model only, so they don't leak into another model's pinned hyperparameters when they share a key name (e.g. learning_rate also being a DNN hyperparameter). Falls back to the single model.name for backward compatibility.
→ documentation/3_adding_a_new_model.md — checklist for implementing a model, wiring it into build_model(), and registering its scaling/val-split/HPO-search-space needs.
Every experiment's results — not just the training curves — are logged, locally and (optionally) live.
┌────────────┐
│ experiment │
│ results │
└────────────┘
│
┌─────────────┴──────────────┐
▼ ▼
┌────────────────┐ ┌──────────────────┐
│ locally │ │ Weights & Biases │
│ (results.json) │ │ (live logging) │
└────────────────┘ └──────────────────┘
training_curves:
plots: true # local PNG plots
wandb:
enabled: true # + Weights & Biases,
project: ml-genetics4psychiatryresults/
└── {experiment}/
├── {experiment}_{timestamp}.json
└── training/
fold_1_curve.png
fold_2_curve.png
...
Computed per outer fold via src/evaluation.py, then aggregated (mean/std) across folds by aggregate_metrics:
- Regression:
r2,pearson_r/pearson_r2(+ p-value),spearman_rho/spearman_rho2(+ p-value). - Binary classification:
accuracy,precision,recall,f1,balanced_accuracy,roc_auc,pearson_r2, confusion-matrix counts (tn/fp/fn/tp).
{
"experiment": "...",
"timestamp": "...",
"config": { ... },
"selected_features": { ... },
"hpo": {
"fold_metrics": [ ... ],
"fold_best_params": [ ... ],
"mean_r2": ..., "std_r2": ...,
...
}
}
- config — full snapshot, for reproducibility
- fold_metrics — per-fold performance
- fold_best_params — per-fold hyperparameters
- mean / std — aggregated across folds
- training_curves — if
training_curves.enabled - shap_mean_values — if
shap.enabled
Every run writes this JSON to results/{experiment_name}/{experiment_name}_YYYYMMDD_HHMMSS.json. Plot and aggregate saved results with analysis/resultAnalysis.py:
python -m analysis.resultAnalysis --results results/linear_regression_scz/
python -m analysis.resultAnalysis --combined results/See figures/README.md for the broader figure-generation scripts built on top of saved results.
Optional, off by default (wandb.enabled: false). When on, src/log.py uploads one run per experiment, logged live as training happens — every epoch (or, for XGBoost, every boosting round) is pushed to W&B the instant it's computed, not batched after the fact. Each fold's selected hyperparameters (from HPO or pinned config) are logged the moment they're resolved, before that fold even starts training. Once every fold is done, two comparison tables are logged: fold hyperparameters side by side, and per-fold metrics + the mean/std aggregate — the same numbers the console and local result JSON already report. Currently populated for any model that produces a training curve — xgboost and residual_dnn in the primary model set (also dnn/mdn). Needs W&B auth already set up on the machine (wandb login, or a WANDB_API_KEY env var) — the config never carries a credential.
wandb:
enabled: true
project: ml-genetics4psychiatry # default
entity: your-team-or-username # optional, defaults to your W&B account default
mode: online # or "offline" to sync later with `wandb sync`
tags: [xgboost, adhd]→ documentation/4_wandb_logging.md for what gets logged and how folds with different curve lengths (e.g. early-stopped XGBoost) are charted correctly on one plot.
A Shapley value measures how much one feature pushed a single prediction away from the average — its contribution to the model's output.
┌─────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ final model │ │ explain per fold │ │ average across │
│ (per fold) │──▶│ (shapiq) │──▶│ folds │
└─────────────┘ └──────────────────┘ └─────────────────┘
shap:
enabled: true
interactions: true
# false → plain Shapley values (SV)
# true → pairwise interactions (k-SII)results/
└── {experiment}/
└── shap/
interaction_values_mean.json
interaction_values_mean_abs.json
fold_1/row_interactions.npz
... (if shap.store)
Beyond a model's aggregate accuracy, feature attribution shows which input features (genotype/MRI phenotypes) drive its predictions — needed to relate a model's performance back to biologically interpretable signal, and to compare that signal across models and folds.
Implemented in src/shap.py, gated by cfg["shap"]["enabled"]. Every model goes through shapiq.Explainer, which dispatches on the estimator: a TabularExplainer (marginal imputation) for the sklearn linear family, a TabPFNExplainer for TabPFN (using its native fast re-conditioning rather than marginal imputation).
Each explained test row yields one InteractionValues object; those are averaged first over a fold's rows, then over the outer folds — giving one mean (signed) and one mean-magnitude attribution per feature (and per feature pair, in interaction mode) for the whole experiment. Both are written under <results_dir>/shap/ and included in the result JSON under shap_mean_values. Per-row values can additionally be persisted with shap.store: true (independent of predictions.enabled) for later custom analysis — see src/shap.py for the full config schema.
All five steps, one config file — each block is independently optional.
data:
dir: "data/my_datasets/"
target_regex: "^Z_scores_"
whitening: {enabled: true}
model:
names: [xgboost, tabpfn]
hpo:
run: true
n_trials: 30
outer_cv: 5
wandb:
enabled: true
project: ml-genetics4psychiatry
shap:
enabled: true
interactions: true
experiment:
run: trueSee requirements.txt and documentation/ for the full pipeline documentation.