Track Purity DNN for Phase-2 HLT by jchismar · Pull Request #251 · SegmentLinking/cmssw

jchismar · 2026-03-27T10:33:43Z

Implementation of a track purity DNN used for high purity selection for HLT tracks. Initial results were presented by Aashay at the tracking POG meeting on 15 Dec 2025. Since then, the model has been retrained with the latest version of LST, and a separate threshold has been implemented for displaced tracks (|dxy| > 0.5) in order to improve displaced track efficiency. This threshold is set at a target recall of 99.5% calculated on tracks with |dxy| > 0.5. For tracks with |dxy| $\le$ 0.5, the threshold is set at a target recall of 99.5% calculated on all tracks. Additionally, the number of input features has been reduced from 29 to 15 with no loss of performance. The DNN is configured to run in the HLTInitialStepSequence after the hltInitialStepTracks step when the trackTorchClassifier procModifier is used. The procModifier is used due to an existing issue with PyTorch GPU memory usage.

The model file is here: SegmentLinking/RecoTracker-FinalTrackSelectors#1

MTV performance on TT+PU=200 is shown below.

slava77 · 2026-03-27T13:00:15Z

@ariostas
is the HLT test for CI still a work in progress?

ariostas · 2026-03-27T13:04:46Z

Yeah, I'll have it done next week

slava77 · 2026-03-27T18:36:01Z

I added a fork in the LST org https://github.com/SegmentLinking/RecoTracker-FinalTrackSelectors

jchismar · 2026-04-06T10:01:05Z

run-ci: [hlt, checks]
procmodifiers: trackTorchClassifier

ariostas · 2026-04-06T16:42:42Z

Actually, I never tested procmodifiers, so let me see if they work before I finalize the HLT workflow.

ariostas · 2026-04-06T16:43:00Z

run-ci: [hlt, checks]
modifiers: ci_devel
procmodifiers: trackTorchClassifier

github-actions · 2026-04-06T19:19:11Z

There was a problem while building and running with HLT workflow. The logs can be found here.

ariostas · 2026-04-06T19:32:54Z

I guess it makes sense that it failed since the procmodifier doesn't exist in master. Let me add extra options to have different procmodifiers in the PR and target branch.

ariostas · 2026-04-07T18:39:25Z

run-ci: [hlt, checks]
modifiers: ci_devel
procmodifiers: {pr: trackTorchClassifier}

github-actions · 2026-04-07T20:07:09Z

The PR was built and ran successfully with HLT setup running on CPU (procModifiers = ). Here are some plots.

HLT General Plots

The full set of validation and comparison plots can be found here.

slava77

mostly naming/style or simpler syntax comments.
Also, unclear why a non-Alpaka torch classifier is needed

slava77 · 2026-04-22T15:26:49Z

+import FWCore.ParameterSet.Config as cms
+
+# This modifier sets the use of a deep neural network for high purity track selection  
+trackTorchClassifier = cms.Modifier()


slava77 · 2026-04-22T15:27:08Z

    firstValid = cms.vuint32(1), 
    iovIsRunNotTime = cms.bool(True)
 )
+


remove the diff

slava77 · 2026-04-22T15:27:21Z

+from Configuration.ProcessModifiers.trackTorchClassifier_cff import trackTorchClassifier
+trackTorchClassifier.toModify(hltInitialStepTrackCutClassifier,
+    src = cms.InputTag("hltInitialStepTrackTorchClassifierOutput")
+)


slava77 · 2026-04-22T15:29:35Z

+hltPhase2LegacyTracking.toModify(hltInitialStepTrackCutClassifier,
+    mva = dict(passThroughForAll=False, passThroughForDisplaced=False)
+)
+


why is this here? looks duplicate of 10 lines above

slava77 · 2026-04-22T15:30:10Z

+
+from Configuration.ProcessModifiers.trackTorchClassifier_cff import trackTorchClassifier
+trackTorchClassifier.toModify(hltInitialStepTrackCutClassifier,
+    src = cms.InputTag("hltInitialStepTrackTorchClassifierOutput")


Suggested change

src = cms.InputTag("hltInitialStepTrackTorchClassifierOutput")

src = "hltInitialStepTrackTorchClassifierOutput"

is a safer syntax

slava77 · 2026-04-22T15:57:22Z

+          features_token_(consumes(iConfig.getParameter<edm::InputTag>("features"))),
+          scores_token_{produces()},


may add Input/Put modifiers in the names for clarity

slava77 · 2026-04-22T15:58:28Z

+
+#include "DataFormats/SoATemplate/interface/SoALayout.h"
+
+GENERATE_SOA_LAYOUT(TrackFeaturesSoALayout,


Suggested change

GENERATE_SOA_LAYOUT(TrackFeaturesSoALayout,

GENERATE_SOA_LAYOUT(TrackTorchClassifierFeaturesSoALayout,

unless the idea is that this is going to be universal

slava77 · 2026-04-22T22:48:31Z

why is this file/TrackTorchClassifier needed if the Alpaka variant is present?

slava77 · 2026-04-22T22:51:37Z

+
+// This module consumes the HOST copy of the Alpaka device scores
+// The framework automatically creates host copies of device PortableCollections
+class TrackTorchClassifierAlpakaOutput : public edm::stream::EDProducer<> {


Suggested change

class TrackTorchClassifierAlpakaOutput : public edm::stream::EDProducer<> {

class TrackTorchClassifierFromSoA : public edm::stream::EDProducer<> {

FromSoA is a more common name modifier for modules reading a SoA and making a "legacy"/non-alpaka collection

slava77 · 2026-04-22T22:52:31Z

+    : tracks_token_(consumes<reco::TrackCollection>(iConfig.getParameter<edm::InputTag>("src"))),
+      scores_token_(consumes<PortableHostCollection<TrackScoresSoA>>(iConfig.getParameter<edm::InputTag>("scores"))),


Suggested change

: tracks_token_(consumes<reco::TrackCollection>(iConfig.getParameter<edm::InputTag>("src"))),

scores_token_(consumes<PortableHostCollection<TrackScoresSoA>>(iConfig.getParameter<edm::InputTag>("scores"))),

: tracks_token_(consumes(iConfig.getParameter<edm::InputTag>("src"))),

scores_token_(consumes(iConfig.getParameter<edm::InputTag>("scores"))),

IIRC types are not needed in the consumes call, it's already detectable from the type of the token.
Check other files

jchismar · 2026-05-01T11:19:03Z

run-ci: [hlt, checks]
modifiers: ci_devel
procmodifiers: {pr: trackTorchClassifier}

jchismar · 2026-05-01T11:30:12Z

run-ci: hlt
procmodifiers: {pr: trackTorchClassifier}

github-actions · 2026-05-01T13:40:16Z

The PR was built and ran successfully with HLT setup running on CPU (procModifiers = ). Here are some plots.

HLT General Plots

The full set of validation and comparison plots can be found here.

slava77

This PR looks almost good to go to CMSSW.
The step to do before doing it is to take out the model file from the commits in this PR and submit it in https://github.com/cms-data/RecoTracker-FinalTrackSelectors

slava77 · 2026-05-18T23:25:39Z

this file should be removed from the CMSSW PR.
The file will be passed via a PR to https://github.com/cms-data/RecoTracker-FinalTrackSelectors
please remove it from the commit history as well.

slava77 · 2026-05-27T12:27:13Z

@jchismar jchismar force-pushed the track-purity-dnn branch from d812d8f to 15d938f 2 hours ago

It looks like the model file is gone.
It would still be nice to squash the commits to get rid of the quite significant file renaming and reshuffling made in a follow up to the code review.

jchismar · 2026-05-27T13:09:19Z

@jchismar jchismar force-pushed the track-purity-dnn branch from d812d8f to 15d938f 2 hours ago

It looks like the model file is gone. It would still be nice to squash the commits to get rid of the quite significant file renaming and reshuffling made in a follow up to the code review.

Just squashed to 2 commits, let me know if you'd prefer a single commit or if this looks fine.

slava77 · 2026-05-27T13:56:07Z

It looks like the model file is gone. It would still be nice to squash the commits to get rid of the quite significant file renaming and reshuffling made in a follow up to the code review.

Just squashed to 2 commits, let me know if you'd prefer a single commit or if this looks fine.

currently the second commit (62883d2) still about a half redoing/renaming the code from the previous commit.
So, better squash back to one.
Do you have a reason to keep the first one separate?

slava77 · 2026-05-27T14:07:08Z

(in case needed; a diff of the May 1st version wrt the two last two commit variant; git isn't making it easy to find or see anymore: https://github.com/SegmentLinking/cmssw/compare/a229b532955f0693977515bf883ac674c3d4b77f..62883d201c58b60d03f1576d54b67d8925e1e002

slava77 · 2026-06-02T15:13:57Z

@jchismar

in CMSSW bot results the hltInitialStepTrackTorchClassifier takes 288.3 ms on the CPU backend.
I suggest to reproduce this locally; start with a single thread and then check/repeat on multi-threaded (x8, x32 ... ) to see if it's scaling up. Compared to the NGT DNN we have CPU-GPU transfers; I'm not sure if some of them are blocking.

Once the bad scaling is confirmed, the CPU time trace from a profiler would also be useful. e.g. with nsys profile --sample=cpu --demangle --output=filename cmsRun config.py

slava77 · 2026-06-25T19:29:28Z

run-ci: all

slava77 · 2026-06-25T19:30:02Z

[the last call to run - ci: all ]

mainly to check/confirm that CI is broken

github-actions · 2026-06-25T19:37:23Z

There was a problem while building and running in standalone mode. The logs can be found here.

github-actions · 2026-06-25T20:01:18Z

There was a problem while building and running with CMSSW. The logs can be found here.

Co-authored-by: Jade Chismar <jchismar@ucsd.edu>

jchismar mentioned this pull request Mar 30, 2026

Add model for phase-2 HLT track classification SegmentLinking/RecoTracker-FinalTrackSelectors#1

Open

slava77 reviewed Apr 22, 2026

View reviewed changes

slava77 reviewed May 18, 2026

View reviewed changes

jchismar force-pushed the track-purity-dnn branch 2 times, most recently from d812d8f to 15d938f Compare May 27, 2026 09:58

jchismar force-pushed the track-purity-dnn branch from 15d938f to 62883d2 Compare May 27, 2026 13:07

jchismar force-pushed the track-purity-dnn branch from 62883d2 to 4986d26 Compare May 28, 2026 09:34

jchismar force-pushed the track-purity-dnn branch from 93e8c78 to faccda8 Compare June 22, 2026 19:47

Add pytorch model for phase-2 hlt track classification

22195b2

Co-authored-by: Jade Chismar <jchismar@ucsd.edu>

jchismar force-pushed the track-purity-dnn branch from 1c6ef29 to 22195b2 Compare July 9, 2026 17:59

github-actions Bot merged commit 8b60706 into SegmentLinking:master Jul 15, 2026

	src = cms.InputTag("hltInitialStepTrackTorchClassifierOutput")
	src = "hltInitialStepTrackTorchClassifierOutput"

		features_token_(consumes(iConfig.getParameter<edm::InputTag>("features"))),
		scores_token_{produces()},


		#include "DataFormats/SoATemplate/interface/SoALayout.h"

		GENERATE_SOA_LAYOUT(TrackFeaturesSoALayout,

	GENERATE_SOA_LAYOUT(TrackFeaturesSoALayout,
	GENERATE_SOA_LAYOUT(TrackTorchClassifierFeaturesSoALayout,

	class TrackTorchClassifierAlpakaOutput : public edm::stream::EDProducer<> {
	class TrackTorchClassifierFromSoA : public edm::stream::EDProducer<> {

		: tracks_token_(consumes<reco::TrackCollection>(iConfig.getParameter<edm::InputTag>("src"))),
		scores_token_(consumes<PortableHostCollection<TrackScoresSoA>>(iConfig.getParameter<edm::InputTag>("scores"))),

Uh oh!

Conversation

jchismar commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

slava77 commented Mar 27, 2026

Uh oh!

ariostas commented Mar 27, 2026

Uh oh!

slava77 commented Mar 27, 2026

Uh oh!

jchismar commented Apr 6, 2026

Uh oh!

ariostas commented Apr 6, 2026

Uh oh!

ariostas commented Apr 6, 2026

Uh oh!

github-actions Bot commented Apr 6, 2026

Uh oh!

ariostas commented Apr 6, 2026

Uh oh!

ariostas commented Apr 7, 2026

Uh oh!

github-actions Bot commented Apr 7, 2026

Uh oh!

slava77 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jchismar commented May 1, 2026

Uh oh!

jchismar commented May 1, 2026

Uh oh!

github-actions Bot commented May 1, 2026

Uh oh!

slava77 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

slava77 commented May 27, 2026

Uh oh!

jchismar commented May 27, 2026

Uh oh!

slava77 commented May 27, 2026

Uh oh!

slava77 commented May 27, 2026

Uh oh!

slava77 commented Jun 2, 2026

Uh oh!

slava77 commented Jun 25, 2026

Uh oh!

slava77 commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 25, 2026

Uh oh!

github-actions Bot commented Jun 25, 2026

Uh oh!

Reviewers

Assignees

jchismar commented Mar 27, 2026 •

edited

Loading

slava77 commented Jun 25, 2026 •

edited

Loading