[Flink] Reshuffle batch inputs round-robin from deterministic start channels

[jto]

Addresses #40607.

Flink's rebalance() starts each sender's round-robin at a random channel. When senders emit only a few elements each, as when a Reshuffle redistributes file names or splittable DoFn restrictions, the resulting assignment is close to random: some receivers get no elements while others get several times the average.

What changes

In batch, Reshuffle is translated with a new DeterministicRebalance: sender i of M sends its j-th element to channel (floor(i * N / M) + j) mod N for N receivers, so already balanced inputs stay balanced. Elements are tagged with their sender's index and parallelism, since a Flink Partitioner is not told which sender it runs in. Streaming pipelines keep using rebalance().

Testing

New DeterministicRebalanceTest: balance with few elements per sender, a single sender, fewer and more senders than receivers, sender key round trips, and an end-to-end batch Reshuffle keeping all elements.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.16%. Comparing base (76f6e6d) to head (ece4054).
⚠️ Report is 181 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #40612      +/-   ##
============================================
+ Coverage     56.16%   65.16%   +8.99%     
- Complexity     2288    13994   +11706     
============================================
  Files          1124     1730     +606     
  Lines        178336   108497   -69839     
  Branches       1489    11307    +9818     
============================================
- Hits         100170    70706   -29464     
+ Misses        75645    33578   -42067     
- Partials       2521     4213    +1692     

Flag	Coverage Δ
go	?
java	65.16% <100.00%> (-10.62%)	⬇️
python	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Assigning reviewers:

R: @shunping added as fallback since no labels match configuration
Backup reviewer: Abacn (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[github-actions]

📋 Review Summary

This PR replaces Flink's random-start DataStream#rebalance() with a deterministic round-robin rebalance (DeterministicRebalance) for batch Reshuffle transforms on the DataStream translator path, preventing Poisson-distributed load skew when senders emit only a few elements each. The partitioning logic and tests are well-designed and compatible across Flink 1.x and 2.0.

🔍 General Feedback

• The CHANGES.md entry was inadvertently placed inside the commented-out template block at the top of the file rather than under the active # [2.78.0] - Unreleased section.
• Follow-up consideration: FlinkStreamingPortablePipelineTranslator#translateReshuffle (in both runners/flink/src/... and runners/flink/2.0/src/...) still unconditionally calls inputDataStream.rebalance(). Updating it when !context.isStreaming() would extend this improvement to portable batch pipelines running on DataStream (Flink 2.0 default and Flink 1.x with --useDataStreamForBatch=true).

---

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

[github-actions]

🟡 Medium: This entry was added inside the HTML-commented <!-- Template --> block (# [2.XX.X] - Unreleased, lines 19–57) instead of the active # [2.78.0] - Unreleased -> ## New Features / Improvements section (around line 71). Please move it to the 2.78.0 section so it is included in the release notes.

[shunping]

Could you help with the review here? @pkuzmickas