Repository navigation

Flink source split strategy - #40433

Draft

jto wants to merge 11 commits into

apache:masterfrom

jto:flink-source-split-strategy

Draft

Flink source split strategy#40433
jto wants to merge 11 commits into
apache:masterfrom
jto:flink-source-split-strategy

Commits on Oct 2, 2026
- [Flink] Re-split bounded sources for even work distribution
  
  jto
  and
  claude
  committed
  a19889c
- [Flink] Simplify re-splitting of oversized bounded source splits
  
  jto
  and
  claude
  committed
  238aa06
- [Flink] Relax rounding of bounded source split count to parallelism
  
  jto
  and
  claude
  committed
  4f385a1
Commits on Oct 5, 2026
- [Flink] Order bounded source splits for the actual reader count
  
  jto
  and
  claude
  committed
  10a7fa0
- [Flink] Balance bounded source split count by per-reader imbalance
  
  jto
  and
  claude
  committed
  49d6f69
- [Flink] Assign lazy source splits in constant time
  
  jto
  and
  claude
  committed
  1a0da0d
- [Flink] Do not re-split bounded source splits below 64 MiB
  
  jto
  and
  claude
  committed
  82f68c3
- [Flink] Aim for two bounded source splits per reader
  
  jto
  and
  claude
  committed
  29dd135
- [Flink] Log the effect of each bounded source splitting step
  
  jto
  and
  claude
  committed
  75e0d8d
Commits on Oct 6, 2026
- [Flink] Balance bounded source split count by bytes per reader
  
  jto
  and
  claude
  committed
  b1a4988
- [Flink] Log bounded source split sizes in human readable units
  
  jto
  and
  claude
  committed
  b9bb0c5