Skip to content

[AURON #2507] Optimize partial WindowGroupLimit execution - #2508

Open
lyne7-sc wants to merge 4 commits into
apache:masterfrom
lyne7-sc:perf/window-group-limit
Open

[AURON #2507] Optimize partial WindowGroupLimit execution#2508
lyne7-sc wants to merge 4 commits into
apache:masterfrom
lyne7-sc:perf/window-group-limit

Conversation

@lyne7-sc

@lyne7-sc lyne7-sc commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #2507

Rationale for this change

Partial WindowGroupLimitExec currently materializes and filters a ranking column that is not included in its output. It also always uses Arrow filtering even when the selected rows form a contiguous range.

What changes are included in this PR?

  • Add a specialized partial group-limit processor for row_number, rank, and dense_rank.
  • Track partition, ordering, and ranking state across input batches.
  • Collect adjacent selected rows as ranges instead of building the selection bitmap row by row.
  • Use zero-copy RecordBatch::slice for a single contiguous range.
  • Build the Boolean selection in blocks for multiple ranges.
  • Add correctness tests and operator-level benchmarks for contiguous and multi-range selections.

Are there any user-facing changes?

No. This is an internal performance optimization.

How was this patch tested?

Passed unit tests.

Benchmarks

The benchmark measures WindowGroupLimit execution over 100,000 input rows. Input sorting is excluded from the measurement.

Benchmark Master This PR Improvement
window_group_limit_row_number 17.558 ms 13.538 ms 22.9%
window_group_limit_rank 32.092 ms 27.360 ms 14.7%
window_group_limit_dense_rank 32.683 ms 28.133 ms 13.9%
window_group_limit_row_number_single_partition 17.612 ms 13.175 ms 25.2%
window_group_limit_row_number_all_selected 17.436 ms 14.519 ms 16.7%

Was this patch authored or co-authored using generative AI tooling?

  • Yes
  • No

If yes, include: Generated-by: GPT-5

ASF guidance: https://www.apache.org/legal/generative-tooling.html

@github-actions github-actions Bot added the native label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Performance] Optimize partial WindowGroupLimit execution

1 participant