Problem
For partial WindowGroupLimitExec, where the ranking column is not included in the output, WindowExec still creates a regular window processor and materializes a complete ranking array for all input rows.
It then builds a Boolean selection from the ranking array, filters both the ranking array and the input batch, and finally discards the filtered ranking array. This introduces unnecessary allocation and filtering for row_number, rank, and dense_rank.
The selected rows also commonly form contiguous ranges, but the current implementation always uses Arrow filtering.
Proposed solution
Add a specialized processor for partial WindowGroupLimitExec that tracks partition, ordering, and ranking state across input batches and directly produces selected row ranges.
Use zero-copy RecordBatch::slice when the selected rows form one contiguous range. For multiple ranges, build the Boolean selection in blocks and use Arrow filtering.
Keep the existing execution path unchanged when window columns are required in the output.
Problem
For partial
WindowGroupLimitExec, where the ranking column is not included in the output,WindowExecstill creates a regular window processor and materializes a complete ranking array for all input rows.It then builds a Boolean selection from the ranking array, filters both the ranking array and the input batch, and finally discards the filtered ranking array. This introduces unnecessary allocation and filtering for
row_number,rank, anddense_rank.The selected rows also commonly form contiguous ranges, but the current implementation always uses Arrow filtering.
Proposed solution
Add a specialized processor for partial
WindowGroupLimitExecthat tracks partition, ordering, and ranking state across input batches and directly produces selected row ranges.Use zero-copy
RecordBatch::slicewhen the selected rows form one contiguous range. For multiple ranges, build the Boolean selection in blocks and use Arrow filtering.Keep the existing execution path unchanged when window columns are required in the output.