Skip to content

[df] Add initial RNTupleProcessor data source - #23477

Merged
enirolf merged 8 commits into
root-project:masterfrom
enirolf:ntuple-ds-processor-pt1
Oct 14, 2026
Merged

enirolf merged 8 commits into
root-project:masterfrom
enirolf:ntuple-ds-processor-pt1

Conversation

@enirolf

@enirolf enirolf commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

An initial (single-thread only) implementation for an RDataSource based on the RNTupleProcessor.

The implementation is adopted from RNTupleDS, and adapted for the RNTupleProcessor where needed.

Dataframes based on this data soruce can be created with the ROOT::Experimental::RDF::FromRNTupleProcessor factory, which takes an already-exisiting processor as its argument.

Almost all tests (all taken from datasource_ntuple) pass, except for the more involved array-based ones. These are commented out for now and will be addressed in the future.

@enirolf
enirolf requested review from hahnjo and pcanal September 24, 2026 07:54
@enirolf enirolf self-assigned this Sep 24, 2026
@enirolf
enirolf force-pushed the ntuple-ds-processor-pt1 branch 2 times, most recently from 3316612 to c3a5911 Compare September 24, 2026 08:30

@silverweed silverweed left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few minor comments

Comment thread tree/ntuple/src/RNTupleProcessor.cxx Outdated
Comment thread tree/ntuple/inc/ROOT/RNTupleProcessorEntry.hxx Outdated
Comment thread tree/ntuple/src/RNTupleProcessorEntry.cxx Outdated
Comment thread tree/dataframe/inc/ROOT/RNTupleProcessorDS.hxx Outdated
Comment thread tree/dataframe/src/RNTupleProcessorDS.cxx Outdated
Comment thread tree/dataframe/src/RNTupleProcessorDS.cxx Outdated
Comment thread tree/dataframe/src/RNTupleProcessorDS.cxx Outdated
Comment thread tree/dataframe/src/RNTupleProcessorDS.cxx Outdated
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Test Results

    24 files      24 suites   4d 1h 19m 27s ⏱️
 3 884 tests  3 883 ✅ 0 💤 1 ❌
83 159 runs  83 158 ✅ 0 💤 1 ❌

For more details on these failures, see this check.

Results for commit 4c09dc9.

♻️ This comment has been updated with latest results.

@enirolf
enirolf force-pushed the ntuple-ds-processor-pt1 branch 2 times, most recently from f65c7e0 to f69246e Compare September 28, 2026 09:46
@enirolf
enirolf force-pushed the ntuple-ds-processor-pt1 branch from f69246e to 46963eb Compare October 7, 2026 14:20

@jblomer jblomer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! In principle LGTM.

Comment thread tree/ntuple/inc/ROOT/RNTupleProcessor.hxx Outdated
Comment thread tree/ntuple/inc/ROOT/RNTupleProcessor.hxx Outdated
Comment thread tree/ntuple/src/RNTupleProcessor.cxx Outdated
Comment thread tree/dataframe/src/RNTupleDSFields.hxx
Comment thread tree/dataframe/inc/ROOT/RNTupleProcessorDS.hxx Outdated
Comment thread tree/dataframe/src/RNTupleProcessorDS.cxx Outdated
Comment thread tree/dataframe/src/RNTupleProcessorDS.cxx Outdated
Comment thread tree/dataframe/test/datasource_ntuple_processor.cxx Outdated
Comment thread tree/dataframe/test/datasource_ntuple_processor.cxx
Comment thread tree/ntuple/inc/ROOT/RNTupleProcessor.hxx Outdated
@enirolf
enirolf force-pushed the ntuple-ds-processor-pt1 branch from 46963eb to 291963d Compare October 8, 2026 14:31
@enirolf
enirolf requested review from jblomer and silverweed October 8, 2026 14:32
@enirolf
enirolf force-pushed the ntuple-ds-processor-pt1 branch from 291963d to fa27a98 Compare October 9, 2026 09:14

@vepadulano vepadulano left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I left a couple of minor comments for consideration.

Comment thread tree/dataframe/inc/ROOT/RNTupleProcessorDS.hxx Outdated
Comment thread tree/dataframe/inc/ROOT/RNTupleProcessorDS.hxx Outdated
Comment thread tree/dataframe/src/RNTupleDSFields.hxx Outdated
@enirolf enirolf added the clean build Ask CI to do non-incremental build on PR label Oct 13, 2026
In preparation for the addition of RNTupleProcessorDS.
@enirolf
enirolf force-pushed the ntuple-ds-processor-pt1 branch from fa27a98 to 0d63702 Compare October 13, 2026 08:56
In the experimental namespace. For now, it is single-threaded only.
In principle, the full type system (as tested for the regular RNTuple
DS) works, with the exception of arrays that are part of classes. This
will be addressed in a future commit.
@enirolf
enirolf force-pushed the ntuple-ds-processor-pt1 branch from 0d63702 to 4c09dc9 Compare October 13, 2026 14:17
@enirolf
enirolf merged commit 69badd2 into root-project:master Oct 14, 2026
60 of 63 checks passed
@enirolf
enirolf deleted the ntuple-ds-processor-pt1 branch October 14, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

clean build Ask CI to do non-incremental build on PR in:RDataFrame in:RNTuple

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants