feat(storage): report aborted transactions to consumers in postgres - #716
Open
OrHayat wants to merge 5 commits into
Open
feat(storage): report aborted transactions to consumers in postgres#716OrHayat wants to merge 5 commits into
OrHayat wants to merge 5 commits into
Conversation
OrHayat
changed the title
OrHayat
force-pushed
the
txn/pg-b-aborted-transactions
branch
from
August 24, 2026 12:00
221825e to
e4914ff
Compare
Replace the pg aborted_transactions stub with a real query: collect abort control markers in [offset, last_stable_offset) and return the aborted producer ranges, wired into the Fetch response so read_committed consumers can filter aborted data instead of reading it as committed.
A retried or repeated TxnOffsetCommit for the same transaction and consumer group violated the staging tables' unique constraints and failed the whole request; Kafka requires the latest staged offset for a partition to win within the transaction.
…oTxn txn_add_offsets was a no-op, so a transaction that only commits consumer offsets kept its txn_detail row's previous terminal status: EndTxn's guard treated the commit as already finalized and silently dropped the staged offsets, and the timeout sweep could never expire the transaction.
- check the producer identity against its full epoch history, so a sweep-fenced zombie's stale epoch gets ProducerFenced instead of staging rows no finalization will ever clean up - reject staging into a transaction that is not in BEGIN with InvalidTxnState: a delayed duplicate after EndTxn otherwise leaves rows the producer's next transaction applies as its own - report UnknownTopicOrPartition when the staged topic or partition does not exist instead of acking success while staging nothing - validation now runs before any insert, so rejected requests leave no partial staging state
v_txn_consumer_offset_tp and v_txn_produce_offset joined producer_epoch on pe.producer = pe.id, matching rows by coincidence of id sequences instead of by producer.
OrHayat
force-pushed
the
txn/pg-b-aborted-transactions
branch
from
August 31, 2026 16:25
e4914ff to
3406086
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
aborted_transactions was a stub returning an empty list, so a read_committed consumer was never told which offset ranges were aborted and client libraries delivered aborted data as if committed.
The first commit implements it: find each producer's abort markers, compute each aborted range's first offset, and wire the result into the Fetch response, kept cheap by a small partial index over control-batch records.