Summary
test_gnn_metric_learning[gpu] runs Examples/Scripts/Python/gnn.py (metric-learning graph construction, torchscript filter.pt + ONNX gnn.onnx edge classifiers, BoostTrackBuilding) and the chain finds no tracks at all: the track_finder_tracks tree in ntuple_finding_gnn.root is empty and the performance writer reports Efficiency with particles = 0.
Evidence
From the gnn_gpu job of https://github.com/acts-project/acts/actions/runs/32143985906/job/95830219625 (Tesla T4):
TrackFinding INFO GNN timing info
TrackFinding INFO - preprocessing: 0.00729 +- 0.00146 [0.00583, 0.00875]
TrackFinding INFO - graph building: 142.718 +- 137.238 [5.48, 279.955]
TrackFinding INFO - classifier: 7.55664 +- 6.33034 [1.2263, 13.887]
TrackFinding INFO - classifier: 0 +- 0 [0, 0]
TrackFinding INFO - track building: 0 +- 0 [0, 0]
RootPatternR INFO Efficiency with tracks (nMatchedtracks/nAlltracks) = -nan
RootPatternR INFO Efficiency with particles (nMatchedParticles/nTrueParticles) = 0
The zero-padded classifier/track-building times are the NoEdgesError path in GnnPipeline::run (Plugins/Gnn/src/GnnPipeline.cpp): the first edge classifier ran (7.6 ms), the second one and track building never did. So the torchscript filter stage (cut = 0.01) removed every edge, and the pipeline returned 0 track candidates.
Why CI never caught this
The test asserted a reference hash of ntuple_finding_gnn.root (30f11c53…). Since the chain has produced this empty result all along, the reference hash is the empty-track output — the assertion held while the test verified nothing about track finding. It also explains why this test never showed the Turing/Ampere numerical split that test_gnn_module_map[gpu-onnx] does: with no tracks, the only content left in the ntuple is the CPU-side truth-particle tree.
The pipeline logs No edges left in GNN pipeline, return 0 track candidates at DEBUG only, so nothing in the CI log flagged it.
Possible causes (unverified)
- Feature scaling:
gnn.py passes featureScales = [1.0, 1.0, 1.0] for [R, Phi, Z], while the models may expect normalised inputs.
- A mismatch between the
torchscript_models (embed.pt / filter.pt) in CI model storage and the cuts used in the script (filter cut 0.01, gnn cut 0.5).
- Metric-learning graph construction parameters (
embeddingDim = 8, rVal = 1.6, knnVal = 100).
Suggested follow-ups
- Re-run the chain with
acts.logging.DEBUG to get node/edge counts per stage and confirm where the edges disappear.
- Consider logging
No edges left in GNN pipeline at WARNING. The test already sets ACTS_LOG_FAILURE_THRESHOLD=WARNING for the subprocess, so that alone would surface this class of failure immediately.
Test status
#5903 replaces the reference-hash comparison in the GNN GPU tests with a smoke check (the hashes are not stable across the heterogeneous GPU pool). The track-content assertion is disabled for this test and the counts are only printed, so the test keeps running end-to-end without pinning the broken output; it should be re-enabled once this is fixed.
Summary
test_gnn_metric_learning[gpu]runsExamples/Scripts/Python/gnn.py(metric-learning graph construction, torchscriptfilter.pt+ ONNXgnn.onnxedge classifiers,BoostTrackBuilding) and the chain finds no tracks at all: thetrack_finder_trackstree inntuple_finding_gnn.rootis empty and the performance writer reportsEfficiency with particles = 0.Evidence
From the
gnn_gpujob of https://github.com/acts-project/acts/actions/runs/32143985906/job/95830219625 (Tesla T4):The zero-padded classifier/track-building times are the
NoEdgesErrorpath inGnnPipeline::run(Plugins/Gnn/src/GnnPipeline.cpp): the first edge classifier ran (7.6 ms), the second one and track building never did. So the torchscript filter stage (cut = 0.01) removed every edge, and the pipeline returned 0 track candidates.Why CI never caught this
The test asserted a reference hash of
ntuple_finding_gnn.root(30f11c53…). Since the chain has produced this empty result all along, the reference hash is the empty-track output — the assertion held while the test verified nothing about track finding. It also explains why this test never showed the Turing/Ampere numerical split thattest_gnn_module_map[gpu-onnx]does: with no tracks, the only content left in the ntuple is the CPU-side truth-particle tree.The pipeline logs
No edges left in GNN pipeline, return 0 track candidatesat DEBUG only, so nothing in the CI log flagged it.Possible causes (unverified)
gnn.pypassesfeatureScales = [1.0, 1.0, 1.0]for[R, Phi, Z], while the models may expect normalised inputs.torchscript_models(embed.pt/filter.pt) in CI model storage and the cuts used in the script (filtercut0.01,gnncut0.5).embeddingDim = 8,rVal = 1.6,knnVal = 100).Suggested follow-ups
acts.logging.DEBUGto get node/edge counts per stage and confirm where the edges disappear.No edges left in GNN pipelineat WARNING. The test already setsACTS_LOG_FAILURE_THRESHOLD=WARNINGfor the subprocess, so that alone would surface this class of failure immediately.Test status
#5903 replaces the reference-hash comparison in the GNN GPU tests with a smoke check (the hashes are not stable across the heterogeneous GPU pool). The track-content assertion is disabled for this test and the counts are only printed, so the test keeps running end-to-end without pinning the broken output; it should be re-enabled once this is fixed.