Repository navigation
[df] Rework RNTuple MT scheduling with a split work-stealing approach - #23327
Open
vepadulano wants to merge 1 commit into
Open
vepadulano wants to merge 1 commit into
vepadulano wants to merge 1 commit into
Conversation
Test Results 23 files 23 suites 3d 21h 34m 30s ⏱️ For more details on these failures, see this check. Results for commit 5c79602. ♻️ This comment has been updated with latest results. |
hageboeck
reviewed
Sep 11, 2026
hageboeck
left a comment
hageboeck
left a comment
Member
There was a problem hiding this comment.
🚀
Here's a first pass with a few suggestions and a few questions.
jblomer
reviewed
Oct 4, 2026
jblomer
left a comment
jblomer
left a comment
Contributor
There was a problem hiding this comment.
Thank you for this work!
I think it may be easier to follow the code if the lambdas became class methods. I would also make documentation easier. Generally, I think a high-level documentation of the scheduling approach would be helpful.
Regarding the use of TBB: would it be possible/easier to simply use a TTaskGroup?
vepadulano
force-pushed
the
df-rntuple-imt-work-stealing
branch
from
October 19, 2026 08:54
ece30c1 to
06ccada
Compare
vepadulano
marked this pull request as ready for review
October 19, 2026 08:58
Member
Author
|
HI @hageboeck @jblomer , I have addressed your PR reviews and updated the changes. Documentation has been added throughout, and I've also memorialised the strategy in a separate markdown document as requested by @jblomer . Let me know your thoughts, thanks! |
vepadulano
force-pushed
the
df-rntuple-imt-work-stealing
branch
from
October 19, 2026 09:27
06ccada to
9acbfde
Compare
silverweed
reviewed
Oct 19, 2026
vepadulano
force-pushed
the
df-rntuple-imt-work-stealing
branch
2 times, most recently
from
October 19, 2026 18:06
da362e3 to
5c79602
Compare
pcanal
reviewed
Oct 19, 2026
pcanal
reviewed
Oct 19, 2026
pcanal
reviewed
Oct 19, 2026
pcanal
reviewed
Oct 19, 2026
| a. A `TThreadExecutor` is initialised with the number of threads given by `ROOT::GetThreadPoolSize()`. | ||
| b. An `RSlotStack` is initialised with the same number of slots. | ||
| c. Optionally, if the user has specified a global entry range to restrict processing to, a list of global entry | ||
| offsets per file is populated by getting the number of entries in each file upfront. |
Member
There was a problem hiding this comment.
my usual complaint about slow starts when using a large number of files :)
Member
Author
There was a problem hiding this comment.
Yes, in fact this is not the default, but only happens if the user wants to restrict the processing, we must know how the global entry range given by the user applies to the actual global entry offsets in the input dataset, right?
pcanal
reviewed
Oct 19, 2026
The MT scheduling of RNTupleDS is changed. The data source adopts now a similar pattern as TTreeDS, implementing a ProcessMT(ROOT::Detail::RDF::RLoopManager &lm) method. This allows the data source to manage the creation and scheduling of multithreaded tasks, inside of which the loop manager is invoked to apply the computation graph to the range of entries in the task. A brief summary of the scheduling approach follows: 1. One task is launched in parallel per input file 2. At the beginning of the file-based task, a page source is created and the list of clusters is computed a. If the user has not specified a global entry range, it is the actual list of clusters of the page source. This happens in ScheduleTasksForFile. b. If the user has specified a global entry range, before launching the file-based tasks the data source had computed a full list of global entry offsets across all input files. The list of clusters of the page source is then adjusted according to the position of the global entry range w.r.t. the global offsets of this file. This happens in ScheduleTasksForFileWithGlobalOffset. 3. The task goes on and calls an internal function to invoke TBB's parallel_for with two arguments, one respecting the Range and the other respecting the Body requirements. a. The Range argument is of type RPageSourceSplittableRange, a struct representing a task with a single page source and an assigned list of clusters to process. In the beginning, the full list of clusters as computed in step 2 is assigned to the Range. b. While processing the task, TBB may call the Range object's SplitWork method, which will split the cluster range in roughly two equal parts, then create a new Range with a clone of the page source and the split cluster range. c. The Body argument calls into ParallelForBody with the page source and the assigned list of clusters of the Range argument, finally applying the computation graph to it by calling RLoopManager::RNTupleThreadTask. This approach delegates to the scheduling engine (TBB in this case) the responsibility to optimise the task processing and split tasks that may be running for too long (e.g. some files may contain many more entries than others). The strategy is also documented in a markdown file for posterity.
vepadulano
force-pushed
the
df-rntuple-imt-work-stealing
branch
from
October 19, 2026 22:57
5c79602 to
167b27a
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Alternative to #22631 , implementing native IMT work-stealing for the RNTuple data source