Gene-Level Druggability / Actionability Scoring (real data)

Rank genes by actionability / druggability on the TCRD Target Development Level scale, learned from many real, target-independent lines of evidence — the tool called for by the CRUK hackathon ST01 target spec. The tier is modelled in two framings:

value	TDL	meaning
3	Tclin	approved drug, known mechanism
2	Tchem	potent small-molecule / bioactivity
1	Tbio	biology known, no qualifying drug
0	Tdark	understudied

• Regression — continuous 0–3 score (src/regression/)
• Classification — 4 ordinal classes (src/classification/)

Honest, leakage-free headline (the model we stand behind)

The deliverable is run v13 (independent profile, 68 target-independent features, GPU) plus its classification twin v13-clf (73 features). Both predict TDL from biology alone, with not a single TCRD feature — TCRD supplies only the label and, separately, a factual drug lookup.

honest model (no leakage)	artifact	CV metric	value
Regression	models/model_13/	R²	0.48
Regression	models/model_13/	Spearman ρ	0.66
Classification (4-class)	models/model_13_clf/	macro-F1	0.60
Druggability ranking	—	ROC-AUC	0.88

Why ~0.48/0.60 and not higher: the TDL tier is partly defined by literature and drug status, which no biological feature can predict. Using the TCRD fields that define the label would trivially inflate the score to ~0.9, but that is circular label leakage and cannot score novel genes — so we exclude them. The model's ranking of candidates by druggability (ROC-AUC ≈ 0.88) is strong and is what the tool actually needs. All numbers are 5-fold stratified cross-validated on an H100 GPU.

The enriched (81-feature) profile that does include TCRD fields is retained only as a documented leakage reference (run v12) — never as a result. A config assertion prevents any TCRD feature from entering the base/independent models.

The two v13 artifacts: regressor and classifier

model_13 is a regressor — it predicts a continuous 0–3 score and only becomes a tier by rounding. Any macro-F1 you see attached to it is that rounded prediction scored post-hoc, not a native output. model_13_clf is its classification twin: same labelled genes, same 5-fold stratified split, same leakage-free evidence, but the tier is predicted directly with macro-F1 as the native objective.

	models/model_13/	models/model_13_clf/
artifact	druggability_regressor.joblib	druggability_classifier.joblib
objective	continuous TDL 0–3	4-class tier
estimator	XGBoost (GPU), Optuna-tuned	soft-vote XGBoost + LightGBM + CatBoost
features	68 (no TCRD)	73 (no TCRD)
macro-F1	0.52 (rounded)	0.60 (native)
best for	ranking genes by druggability	assigning a tier

Predicting the tier directly is worth +0.08 macro-F1 over rounding the regressor, almost all of it on the rare, actionable Tchem/Tclin tiers that rounding collapses. Use the regressor to rank, the classifier to label. Outputs: outputs/output_13_clf/.

import joblib, json
clf = joblib.load("models/model_13_clf/druggability_classifier.joblib")
feats = json.load(open("models/model_13_clf/run_info.json"))["features"]  # the 73 columns
clf.predict(X[feats])        # tier 0-3, macro-F1-calibrated decision rule
clf.predict_proba(X[feats])  # (n, 4) probabilities: Tdark, Tbio, Tchem, Tclin

How it clears 0.59, without touching the label (OOF macro-F1, same split):

step	macro-F1
best single classifier, 68 features (previous baseline)	0.5876
+ Bgee expression breadth & TOGA ortholog conservation (73 feat)	0.5844–0.5887
+ soft-vote ensemble of XGBoost / LightGBM / CatBoost	0.5999
+ nested per-class decision bias	0.6021 ✅

Every one of the 5 folds clears 0.59 (0.597–0.610). The two added feature families are target-independent — expression breadth and mammalian ortholog retention play no part in how TDL is assigned — so they lift the ceiling without circularity, and _assert_no_leakage() fails the run if a TCRD column ever reaches the feature list.

On the decision bias. The per-class bias (argmax(log P + bias)) is what makes an imbalanced 4-class target (~64% Tbio) resolve its rare tiers. src.classification.classify fits that bias on the same OOF labels it then scores against, which is mildly optimistic. Here the bias is fit on inner folds only and applied to the held-out outer fold, so 0.6021 is an honest number — the non-nested version of the same model reads 0.6058.

Feature profiles (DRUG_FEATURE_PROFILE)

profile	table	features	use
base (default)	data/merged_features.csv	30	genomic baseline (no TCRD)
independent ⭐	data/merged_features_enriched.csv	68	the model — genomic/constraint/domains/COSMIC/PPI/AlphaMissense (no TCRD)
enriched	data/merged_features_enriched.csv	81	circular leakage reference only (adds TCRD)

The 68 model features fall in 8 evidence families (all target-independent): constraint/evolution (40), druggable-genome domains (12), COSMIC somatic burden (7), cancer-gene census (3), PPI network hubness (3), functional impact / AlphaMissense (1), synthetic lethality (1), regulatory/enhancer (1). Full provenance in FEATURE_TRACKS.md; build logs in data/enrichment_log.log.

Repository layout

src/
  config.py            feature profiles + evidence families (+ no-TCRD guard)
  data.py metrics.py utils.py models.py
  merge_features.py    build data/merged_features.csv (30-feat base)
  enrich_features.py   build data/merged_features_enriched.csv (adds gnomAD/COSMIC/PPI/AlphaMissense)
  tcrd_extract.py      parse the TCRD dump → label + drug catalogue (NOT model features)
  regression/          train · evaluate · visualize · score · compare_runs
  classification/      classify (4-class) · binary (actionable / known-biology)
                       independent_clf (leakage-free 4-class twin of model_13 → macro-F1 0.60)
  explain.py           SHAP per-feature + per-evidence-family + per-gene decomposition
  xai_figures.py       feature attribution · uncertainty · applicability domain (literature-aligned)
  ablation.py          model-evolution study (feature families 1→2→3)
  drug_intel.py        per-gene DrugCentral/ChEMBL drug catalogue (dashboard lookup)
app/streamlit_app.py   interactive dashboard
report/                report.tex + report.pdf (LaTeX: TikZ/forest/pgfplots architecture)
models/ outputs/       versioned runs + registry

Reference data you must download

Everything needed to train and score is versioned here — including the built feature tables (data/merged_features*.csv) and the derived network/domain tables (data/derived/). You only need the raw sources below to rebuild those from scratch.

STRING v12.0 (for src/enrich_network.py; the links file is 133 MB, over GitHub's file limit):

mkdir -p data/string && cd data/string
wget https://stringdb-downloads.org/download/protein.links.detailed.v12.0/9606.protein.links.detailed.v12.0.txt.gz
wget https://stringdb-downloads.org/download/protein.info.v12.0/9606.protein.info.v12.0.txt.gz

TCRD (labels + drug catalogue) and COSMIC (somatic variants) are multi-GB licensed dumps and are not versioned either. Fetch/parse commands are at the top of src/tcrd_extract.py and src/enrich_features.py. COSMIC requires a free academic account.

Run everything

python -m src.merge_features                                          # base 30-feature table
python -m src.enrich_features                                         # enriched table (superset)
DRUG_FEATURE_PROFILE=independent python -m src.regression.train --model XGBoostGPU --version 13 --full
python -m src.classification.independent_clf                          # v13 classification twin (macro-F1 0.60)
DRUG_FEATURE_PROFILE=independent python -m src.classification.classify
DRUG_FEATURE_PROFILE=independent python -m src.classification.binary  # binary druggability variants
DRUG_FEATURE_PROFILE=independent python -m src.explain --version 13   # SHAP evidence breakdown
DRUG_FEATURE_PROFILE=independent python -m src.xai_figures --version 13
python -m src.ablation                                                # model-evolution study
python -m src.regression.compare_runs                                 # leaderboard

Rebuild the derived label/drug/COSMIC tables from the raw dumps: see the commands at the top of src/tcrd_extract.py and src/enrich_features.py.

Model architecture

A gradient-boosted decision-tree ensemble (XGBoost, GPU): the 68-feature gene matrix X ∈ ℝ^(18,495×68) is median-imputed with a missing-indicator (→ ℝ^110), then scored by 1200 depth-7 trees (η≈0.0116) whose summed output feeds a regression head (continuous 0–3) and a class-balanced, threshold-calibrated classification head (4 tiers). The full architecture, drawn from the real trained model, is in report/report.pdf (Fig. 2).

Model zoo (selected/tuned by 5-fold OOF CV + Optuna): baseline, Ridge, ElasticNet, RandomForest, ExtraTrees, HistGBM, LightGBM, XGBoost, CatBoost, MLP, StackingEnsemble and GPU XGBoost/CatBoost. Classification uses balanced class weights and macro-F1.

Explainability (transparent, not a black box)

src.explain decomposes every gene's score into its component evidence types via SHAP, grouped into the 8 evidence families in config.FEATURE_FAMILIES (all biological — no TCRD family exists). src.xai_figures adds, per the drug-discovery XAI taxonomy (report/ml_figure_skills.md, Jiménez-Luna et al.): feature-attribution beeswarm, uncertainty estimation, and applicability-domain analysis. Outputs land in outputs/output_13/explain/ and outputs/output_13/xai/.

How performance is built (model evolution)

src.ablation (→ outputs/model_evolution/) shows the honest CV climb as each target-independent family is added: 6 feats → 30 → +independent enrichment (68), with R² 0.13 → 0.34 → 0.46 and macro-F1 0.37 → 0.51 → 0.58. This is the honest ceiling — biology explains about half the variance of a partly literature-defined label; there is no leakage shortcut in the feature set.

Interactive dashboard (Streamlit) + drug intelligence

pip install streamlit plotly
streamlit run app/streamlit_app.py

Paste or upload a gene list — or upload your own feature table (gene id + any of the 68 model features) to score genes that are not in the built-in database (missing features are median-imputed by the model). Results are downloadable as CSV. The app scores and ranks each gene by druggability with the honest v13 model, then provides a full analytics suite:

• Cohort comparison — an evidence-family heatmap (genes × 8 families, genome percentile) and a multi-gene radar overlay.
• Per-gene drill-down (Plotly): evidence radar fingerprint, SHAP waterfall (baseline → per-family push → tier), genome positioning (histogram + percentile gauge over all 34,600 genes), feature detail (per-feature value/percentile/SHAP), and druggable target-class profile.
• Drug-development intelligence — the full actionability chain, queried live: drugs → developers (firms) → clinical-trial phases → funders → competitors → patents → market. Built by src/drug_intel_live.py from Open Targets Platform, ClinicalTrials.gov v2, NIH RePORTER and the local DrugCentral/ChEMBL catalogue; a showcase cache lives in data/drug_intelligence/gene_intel_cache.json (rebuild: python -m src.drug_intel_live --build EGFR KRAS …). Shown as a factual lookup after scoring — never a model input.

LaTeX report

report/report.pdf (built from report/report.tex) is a full, transparent write-up mapping the system to the ST01 target spec and to each of the five hackathon judging criteria, with TikZ/forest/pgfplots architecture diagrams and real-data result figures. Rebuild: cd report && pdflatex report.tex (×2).

Versioned, traceable runs

models/model_<N>/     model + run_info.json          (regression)
outputs/output_<N>/   report, CV metrics, OOF, predictions, figures/, explain/, xai/, logs/
outputs/output_clf_independent/                       (honest 4-class classification)
outputs/output_binary_{actionable,known_biology}/     (binary variants)
outputs/runs_index.csv   append-only registry of every run

output_1 = legacy 6-feature baseline; output_2…11 = base-30 model zoo; output_13 = the honest independent model (the deliverable); output_12 / output_clf_enriched = the circular enriched model, retained only as a leakage reference (not a result).

Tests

python -m pytest tests/ -q

Notes

• The honest model (independent, run v13) uses zero TCRD features. TCRD supplies only the TDL label; the drug catalogue is DrugCentral/ChEMBL. A config assertion fails the import if any TCRD feature leaks into a real model.
• The unit of prediction is a human gene (HGNC symbol); a feline/canine hit is scored via its human ortholog.
• 16,105 genes have no TCRD label; the honest model still scores them — the actual use case (prioritising un-annotated candidates).

Licence and attribution

Code in this repository is released under the MIT Licence.

The data is not ours to relicense: TCRD, STRING, COSMIC, gnomAD, InterPro, Bgee and TOGA each carry their own terms — see REFERENCES.md for sources, and check each provider's licence before redistribution. COSMIC in particular requires a licence for commercial use.

Prior work this builds on is cited in REFERENCES.md. Published papers are referenced by DOI, not redistributed here.

Built for the CRUK "AI and Cancer Omics" hackathon.