Live report: https://en970.github.io/particlenet-jet-tagging/
PDF (LaTeX): reports/particlenet-jet-tagging-report.pdf
Both versions classify jets (gluon, light-quark, top, W, Z) from the JetNet dataset (particle clouds of up to 30 constituents), comparing a Deep Sets baseline (no edges, permutation-invariant pooling only) against a Graph Neural Network.
v1 (src/jet_tagging_local.py): a static, fully-connected particle graph, 3 raw features per
particle (eta_rel, phi_rel, pt_rel), 20,000 jets.
| Model | Accuracy | Macro AUC |
|---|---|---|
| Deep Sets | 65.5% | 0.891 |
| mini-ParticleNet (static graph) | 66.7% | 0.898 |
v2 (src/jet_tagging_particlenet_v2.py): a dynamic k-nearest-neighbour graph rebuilt at
every layer in the learned feature space (the defining trait of real ParticleNet), 5 engineered
features (+log(pt_rel), +deltaR), 30,000 jets.
| Model | Accuracy | Macro AUC |
|---|---|---|
| Deep Sets (v2 features) | 70.5% | 0.913 |
| ParticleNet v2 (dynamic k-NN) | 69.7% | 0.911 |
Going from v1 to v2, accuracy improved by +5 points -- but essentially all of that gain came from the engineered features, not from the GNN's edges. The dynamic-graph GNN did not beat its own Deep Sets baseline at this scale. Three likely reasons, discussed in more depth in the full report:
- With only 30 particles per jet, k=16 nearest neighbours is close to a fully-connected graph already -- there is little "dynamic" structure left to exploit. Real ParticleNet's edge shows up more clearly on jets with 100-150 constituents.
- 15 epochs and an untuned learning rate is a small budget for a higher-capacity model; the GNN's validation curve is visibly noisier (less converged) than the Deep Sets curve.
- Deep Sets with good features is a genuinely strong, well-known baseline in the jet-tagging literature -- not a strawman.
The takeaway: a more sophisticated architecture (GNN > Deep Sets) is not an automatic win; the gain depends on data scale, feature quality, and training budget. This is a real, common outcome in ML research, and reporting it honestly is more useful than only reporting wins.
python3 src/jet_tagging_local.py # v1: static graph
python3 src/jet_tagging_particlenet_v2.py # v2: dynamic k-NN + engineered featuresData: JetNet (pip install jetnet), cached wherever DATA_NPZ in the scripts points (an
external drive is recommended for large caches).
index.html formal technical report (open locally or via GitHub Pages)
assets/ figures used by the report
src/ training/evaluation code (v1 and v2)
outputs/ result figures and summary JSON
reports/ combined LaTeX technical report (PDF)
An earlier version of this section pointed at "the published ParticleNet numbers (~90%+ accuracy)" as the target. That was a cross-benchmark comparison and it is wrong. ParticleNet's ~90%+ figures come from binary top-tagging and quark-gluon discrimination, not from five-class JetNet; on the nine-class JetClass dataset ParticleNet reaches 84.4%.
The correct same-benchmark reference point is 75.08% overall test accuracy on five-class JetNet (850,000 jets, 170k per class), reported by Bassa et al., Jet image tagging using deep learning: an ensemble model, JINST 20 P09032 (2025), arXiv:2508.10034. Note the representation differs -- that work classifies jet images, this one classifies particle clouds -- so it is a yardstick for the task and dataset, not a like-for-like architecture comparison.
Against that, the 70.5% here on a 30,000-jet subsample is closer to the published figure than "~90%+" made it look.
The headline comparison is not yet a measurement. Both runs used SEED = 0 with no repeats, so
the seed variance was never established, and the reported gap is 0.8 points. Between v1 and v2
three things changed at once -- 20k to 30k jets, 3 to 5 features, static to dynamic k-NN graph
-- and the sign of the result flipped with them: the graph model won by 0.0117 in v1 and lost by
0.0082 in v2. Parameter counts were never reported, and ParticleNetV2 carries roughly twice the
capacity of its Deep Sets baseline.
A rerun is planned that fixes exactly this: three dense architectures matched to 100k parameters +/- 5% with the count printed rather than assumed, the full ~880k-jet JetNet-30, five seeds per cell, warmup plus cosine schedule, and both feature arms crossed with architecture so that feature richness and architecture are no longer confounded.
Retry on JetNet150 (150 particles/jet) where dynamic k-NN graphs have real structure to exploit,
with more epochs and a learning-rate schedule -- or use weaver-core, the training framework
used for the published ParticleNet results.
- Qu & Gouskos (2020), ParticleNet: Jet Tagging via Particle Clouds, Phys. Rev. D 101, 056019 -- https://arxiv.org/abs/1902.08570
- Zaheer et al. (2017), Deep Sets, NeurIPS 2017 -- https://arxiv.org/abs/1703.06114
- JetNet -- https://github.com/jet-net/JetNet
- weaver-core -- https://github.com/hqucms/weaver-core
- PyTorch Geometric -- https://pytorch-geometric.readthedocs.io/
MIT (see LICENSE). JetNet is a public benchmark dataset derived from simulation.