Using joined RDFs: FilterAvailable error on defined column that uses an aliased column and FilterAvailable error on an aliased column. #22759

Activity

1. modregominguez

   added

   bug

   on Jul 6, 2026
2. couet

   assigned

   vepadulano

   on Jul 6, 2026
3. ferdymercury

   added

   in:RDataFrame

   on Jul 6, 2026
4. vepadulano commented on Jul 6, 2026

   

   vepadulano

   on Jul 6, 2026

   Member

   More actions

   Dear @modregominguez,

   Thank you for the report and the concise reproducer! This indeed looks like a bug, I will take a look at it.

   👍React with 👍1Reacted by jminguezCern
5. MSK-005 commented on Oct 4, 2026

   

   MSK-005

   on Oct 4, 2026

   More actions

   Hello, @vepadulano! I'm Moazzam Shahzad, and I'm new here.

   For the TTree case, let's focus on what happens when the following line is executed:
   auto kept2 = base.FilterAvailable("matched_field").Count();

   When .Count() is executed, the lazy graph evaluation starts. We know RFilterWithMissingValues::InitSlot() calls RColumnRegister::GetReaderUnchecked(), which checks if a column is a variation or a Define. However, this function does not check if a column is an alias or not, and so it returns nullptr. This nullptr propagates upwards; it prevents reading the original column's values when the columns are later fetched for calculation, and thus, the system reports 0 entries, as found in the reproducer code. Besides RColumnRegister::GetReaderUnchecked(), I also found the same problem (no alias check) in RColumnRegister::GetReader(). Thus, the fix appears to be to add a simple check to see if a column is an alias or not (and resolve it if it is) inside RColumnRegister::GetReader() and RColumnRegister::GetReaderUnchecked().

   For the RNTuple case, that is a separate branch that is actively being worked on, and it's your choice whether you want me to work on that fix or if the currently active developer should fix it.

   If my diagnosis for the TTree case seems reasonable to you, please allow me to start work on the fix, and let me know how you want to proceed with the RNTuple case. Thank you.
6. vepadulano commented on Oct 6, 2026

   

   vepadulano

   on Oct 6, 2026

   Member

   More actions

   Dear @MSK-005 ,

   Thanks for taking interest in the ROOT project and in this issue!

   You are hitting certain things right with your description above, although I believe by reading it that there are some things which are not clear yet. Feel free to work on this and propose a PR, which at the very least should include the fix for TTree and a few unit tests. I don't think for RNTuple we currently have a way to expose the case of missing entries.
7. MSK-005 commented on Oct 13, 2026

   

   MSK-005

   on Oct 13, 2026

   Last edited by MSK-005

   More actions

   Dear @vepadulano,

   I need your input for the Define bug.

   FilterAvailable() on a Define column that has inputs with missing entries results in an exception instead of being filtered out. The problem is that FilterAvailable() on a Define triggers RFilterWithMissingValues::CheckFilters(), which calls RColumnReaderBase::TryGet(), which is supposed to return the evaluated value from the column. TryGet() calls RDefineReader::GetImpl(), which always runs Update() before returning fValuePtr. fValuePtr points to the Define's value storage and is never null, and GetValueChecked() (called from UpdateHelper) throws if an input is missing. So GetImpl() either returns a valid pointer or throws, and the == nullptr check in CheckFilters() can't work for Defines.

   I'd like RDefine to check, in a new function called from Update(), whether its inputs are available for the entry. If one isn't, GetImpl() returns nullptr instead of evaluating. The alternative is an availability method on the reader interface called from CheckFilters(). Either way, each input gets a second TryGet() call per entry. Which would you prefer?