Skip to content

fix: prevent SMT corruption during HA leadership churn#158

Merged
jait91 merged 3 commits into
v0.1.0-legacyfrom
hotfix/v0.1.0-round-finalization-safety
Jun 17, 2026
Merged

fix: prevent SMT corruption during HA leadership churn#158
jait91 merged 3 commits into
v0.1.0-legacyfrom
hotfix/v0.1.0-round-finalization-safety

Conversation

@jait91

@jait91 jait91 commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Storage pressure could cause HA leadership churn (the aggregator instance repeatedly lost and reacquired
leadership) while old child-round goroutines were still waiting for parent proofs. A stale finalizer could then write the current round's pending SMT data under an older block, causing blocks, block_records, aggregator_records, and smt_nodes to diverge.

Fix by cancelling and waiting for in-flight round processing on Deactivate, refusing to overwrite a same-or-newer active round on reactivation, and validating round number, snapshot root, and duplicate-block contents before finalization continues.

Storage pressure could cause leadership churn while old child-round goroutines were still waiting for parent proofs. A stale finalizer could then write the current round's pending SMT data under an older block, causing blocks, block_records, aggregator_records, and smt_nodes to diverge.

Fix by cancelling and waiting for in-flight round processing on Deactivate,
refusing to overwrite a same-or-newer active round on reactivation, and validating round number, snapshot root, and duplicate-block contents before finalization continues.
@jait91 jait91 requested a review from MastaP June 5, 2026 06:07

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the block finalization and round management logic in the RoundManager to prevent stale finalizations, handle duplicate blocks safely, and manage activation/deactivation lifecycles properly. The reviewer feedback highlights critical concurrency issues in Activate where re-activation or partial failure can lead to resource leaks and inconsistent states. Additionally, several performance and safety improvements were suggested, such as using Cmp for big integer comparisons to avoid nil pointer dereferences and avoiding unnecessary slice allocations during block finalization.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread internal/round/round_manager.go
Comment thread internal/round/round_manager.go
Comment thread internal/round/batch_processor.go Outdated
Comment thread internal/round/batch_processor.go
Comment thread internal/round/batch_processor.go Outdated

@MastaP MastaP left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two findings from a focused review of the child-sharding HA path. Context: in child mode the child RoundManager has no BFT client (it talks to the parent via the root client), and finalization is driven by processRoundpollForParentProofFinalizeBlockWithRetry, which Deactivate's new roundWG.Wait() correctly covers — and cancel-during-poll is handled and tested. These two are additional to the existing automated-review comments: one High (a leadership stand-down landing in the finalize window is misclassified as fatal and crashes the node) and one Medium (the new duplicate-mismatch guard is untested because Test8 exits early at the snapshot-root guard).

Comment thread internal/round/round_manager.go
Comment thread internal/round/finalize_duplicate_test.go Outdated
@MastaP

MastaP commented Jun 16, 2026

Copy link
Copy Markdown
Member

@jait91

@jait91 jait91 merged commit ba4a77f into v0.1.0-legacy Jun 17, 2026
2 checks passed
@jait91 jait91 deleted the hotfix/v0.1.0-round-finalization-safety branch June 17, 2026 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants