fix(sql): anchor the best-chain walk on the highest non-orphaned block - #229
Merged
Merged
Conversation
Every "walk back from the tip" recursive CTE anchored on the table's global MAX(height). After a hard fork the abandoned chain's blocks stay in the archive ABOVE the new chain's tip, marked orphaned, so that anchor lands on a dead block. With the chain_status = 'pending' guard (blocks-service) it matched nothing and inBestChain: true silently degraded to canonical-only rows; without the guard (events, actions, action-state resolution, zkapps-with-pending-events) the walk followed the dead chain. Seen on mainnet at the Mesa upgrade, 2026-09-03: the old chain reached 548187 before stopping, the fork block was 548147, and for the whole first hour blocks(inBestChain: true) returned nothing above 548147 while networkState correctly reported a pending tip of 548164. The two disagreed because only networkState computes its maximum per chain_status. Downstream, the mina-explorer-api indexer discovers blocks only through that filter, so the explorer's recent-blocks list on mainnet showed the abandoned chain. The anchor is now (SELECT MAX(height) FROM blocks WHERE chain_status <> 'orphaned') — one constant in src/db/sql/best-chain.ts used by both query styles — which agrees with networkState.pendingMaxBlockHeight and is a no-op in normal operation, where the highest block is always the pending tip. Tests: an integration case plants the hard-fork shape (an orphaned block one height above the synthetic pending tip) and asserts inBestChain still reaches the tip, agrees with networkState, and reports the orphan under inBestChain: false; a unit test renders every events/actions query through a recording client and pins that none anchors on the global maximum. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SHHxXVk9RcCLHj7yM69m8J
dkijania
previously approved these changes
Sep 3, 2026
SanabriaRusso
force-pushed
the
fix/best-chain-anchor-after-hard-fork
branch
from
September 3, 2026 19:46
47ef2ac to
5676161
Compare
dkijania
approved these changes
Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every "walk back from the tip" recursive CTE anchored on the table's global
MAX(height). After a hard fork the abandoned chain's blocks stay in the archive above the new chain's tip, markedorphaned, so that anchor lands on a dead block:blocks-service.tsguards the anchor withchain_status = 'pending', so it matched nothing andinBestChain: truesilently degraded to canonical-only rows.events-actions/queries.ts(events, actions, action-state boundary, zkapps-with-pending-events) has no guard, so the walk followed the dead chain.The anchor is now
(SELECT MAX(height) FROM blocks WHERE chain_status <> 'orphaned'), one constant insrc/db/sql/best-chain.tsused by both query styles. It agrees withnetworkState.pendingMaxBlockHeight(which already computes its maximum perchain_status) and is a no-op in normal operation, where the highest block is always the pending tip.What it looked like on mainnet (Mesa upgrade, 2026-09-03)
The old chain reached height 548187 before stopping at 14:45Z; the fork block is 548147 (18:00Z); the new tip was 548164 an hour in.
Downstream,
mina-explorer-api's indexer discovers blocks only through that filter, so it ingested no Mesa-era block while its cursor read caught-up, and the explorer's recent-blocks list on mainnet showed the abandoned chain (o1-labs/mina-explorer-api companion PR handles that side).Tests
tests/integration/integration.test.ts, "Hard-fork shape"): plants anorphanedblock one height above the synthetic pending tip and assertsinBestChain: truestill reaches the tip, agrees withnetworkState, and reports the orphan underinBestChain: false. Fails onmain.tests/unit/best-chain-anchor.test.ts): renders every events/actions query through a recording stand-in for the postgres client and pins that none anchors on the global maximum. No DB needed.npm run lint,npm run test:unit(15 files, 0 failures),npm run test:integration(36 + 29 + 1 pass, 0 fail) — all green locally against Postgres 13.Rollout
Mainnet's
archive-node-apineeds a release and an image bump in gitops-infrastructure for the fix to take effect. Until then the symptom clears on its own only once the new chain outgrows the old one (548187), and only forblocks; the events/actions walk stays on the dead chain until the archive daemon relabels every old-chain blockorphaned.A runbook row was added under Common incidents for the relabelling window.
🤖 Generated with Claude Code
https://claude.ai/code/session_01SHHxXVk9RcCLHj7yM69m8J