Skip to content

fix(binding-kafka): derive the merged produce window from the least budget offered - #2546

Merged
jfallows merged 2 commits into
developfrom
fix/2516-kafka-merged-produce-window-least-budget
Sep 4, 2026
Merged

fix(binding-kafka): derive the merged produce window from the least budget offered#2546
jfallows merged 2 commits into
developfrom
fix/2516-kafka-merged-produce-window-least-budget

Conversation

@jfallows

@jfallows jfallows commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #2516

doMergedInitialWindow aggregated the produce partitions by taking the largest unacknowledged byte count across them and, separately, the smallest maximum. A partition carrying traffic holds its acknowledge and grows its maximum, while idle partitions keep the maximum they were opened with, so those two reductions select different partitions: the unacknowledged bytes of the busiest one get paired with the maximum of an idle one. The merged window then shrinks by every byte written until it reaches zero and the stream stalls for good. With a single partition both reductions select the same stream, which is why this only appears on topics with more than one partition.

This is the primary defect behind the original bug report this was found in (published/forwarded 200/60 in a live 5-partition topic test before the fix, 600/600 after) — an MQTT client publishing small QoS 0 messages through an mqtt-kafka proxy to a multi-partition Kafka topic stopped being forwarded after roughly 60 messages, with Zilla closing the connection itself and no exception or event logged.

Fix reduces over the budget each partition actually offers instead (initialMax - initialNoAck), and adds the unacknowledged bytes back to express it as a maximum. For a single partition this is arithmetically identical to the previous computation. Also adds a merged-window trace under the existing produce debug property, since this aggregation was not observable from a running gateway.

Credit to community contributor @sfr-oc, who diagnosed and fixed this as part of the combined #2523. This PR cherry-picks that fix's commit standalone (unmodified, ccc296db), since #2523 bundles seven independently-scoped defects across three bindings into one PR — each deserves its own focused review, and this one is ready on its own.

Test coverage

Also cherry-picks the companion test commit (unmodified, 1a4495b7) from the same original PR, which added the actual regression coverage for this scenario (the fix commit alone had none): merged.produce.message.values.partition.idle opens two produce partitions, sends all traffic to one and leaves the other idle, and asserts a write past the idle partition's maximum still succeeds.

  • Paired merged/unmerged k3po scripts, matching this binding's existing convention for every other merged.produce.* scenario
  • Runtime IT: CacheMergedIT#shouldProduceMergedMessageValuesPartitionIdle

Checked the new scenario's naming against this binding's existing merged.produce.message.values.* family (.dynamic, .null, .producer.id, .partition.id) — .partition.idle fits the established vocabulary directly, no rename needed.

Verification

  • Reverted just the production fix and confirmed the IT fails for the right reason: TestTimedOutException after exactly 10 of the 11 expected writes succeed, matching "the merged window shrinks by every byte written until it reaches zero and the stream stalls for good" — then restored the fix and confirmed the test passes
  • Full reactor build compiles clean

🤖 Generated with Claude Code

https://claude.ai/code/session_015YVNaqKvEXZVzmg3HoGVnt


Generated by Claude Code

sfr-oc and others added 2 commits September 4, 2026 22:20
…udget offered

doMergedInitialWindow aggregated the produce partitions by taking the largest
unacknowledged byte count across them and, separately, the smallest maximum.
A partition carrying traffic holds its acknowledge and grows its maximum,
while idle partitions keep the maximum they were opened with, so those two
reductions select different partitions: the unacknowledged bytes of the
busiest one get paired with the maximum of an idle one. The merged window
then shrinks by every byte written until it reaches zero and the stream
stalls for good. With a single partition both reductions select the same
stream, which is why this only appears on topics with more than one
partition.

Reduce over the budget each partition actually offers instead, and add the
unacknowledged bytes back to express it as a maximum. For a single partition
this is arithmetically identical to the previous computation.

Adds a merged window trace under the existing produce debug property, since
this aggregation was not observable from a running gateway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tition

merged.produce.message.values.partition.idle opens two produce partitions,
sends all traffic to one and leaves the other idle. Reducing the maximum
across partitions independently of the unacknowledged bytes pins the merged
window to the idle partition's maximum while the busy partition's
unacknowledged bytes keep growing, so the stream stalls once that maximum is
reached; the scenario asserts a further write still succeeds beyond it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jfallows
jfallows merged commit 3c30158 into develop Sep 4, 2026
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Merged produce window collapses on topics with more than one partition

2 participants