Skip to content

refactor: implement atomic state file updates and improve error handl…#396

Open
A-one-tech wants to merge 1 commit into
tributary-protocol:mainfrom
A-one-tech:fix/atomic-cursor-state-write
Open

refactor: implement atomic state file updates and improve error handl…#396
A-one-tech wants to merge 1 commit into
tributary-protocol:mainfrom
A-one-tech:fix/atomic-cursor-state-write

Conversation

@A-one-tech

Copy link
Copy Markdown

#Closes
#257

fix(indexer): atomic cursor state writes and defensive load

What this changes

Makes the indexer's cursor persistence safe against process crashes, container stops, and disk-full events mid-write.

Two functions in indexer/index.mjs are changed:

  • saveCursor — previously called writeFileSync directly on state.json, which truncates the file before writing. A crash in the write window leaves an empty or partial file. Now the function writes to a sibling state.json.tmp and calls renameSync to atomically replace the target. On POSIX, rename(2) is guaranteed atomic: any reader sees either the previous valid file or the new one, never a partial state.

  • loadCursor — previously called JSON.parse without a try/catch. existsSync returns true for a zero-byte file, so a corrupt or empty state.json caused SyntaxError: Unexpected end of JSON input at module load, before any signal handlers were attached. Now the parse is wrapped in try/catch; on failure it logs a warning and returns null (same as "no state file"), so the indexer starts from the default window instead of crashing permanently.

A secondary refactor converts the module-level const STATE into a stateFile() getter that reads process.env.STATE at call time. This lets tests inject an isolated path without reloading the module, and it keeps the fix self-contained — no new config surface.

The polling entrypoint is wrapped in if (import.meta.filename === process.argv[1]) so index.mjs can be imported in test files without starting live network calls.

Why

saveCursor is called on every page of every poll. Under normal operation the write window is hit constantly, making the failure mode much more likely than it appears. A single badly-timed crash previously left the indexer unable to start until an operator manually deleted state.json — and deleting the file sent the cursor back to latestLedger - 100_000, causing a full re-index and duplicate events in events.ndjson.

Checklist

  • cargo fmt --all and cargo clippy --all-targets -- -D warnings pass (no Rust changes)
  • cargo test passes (no Rust changes)
  • New behavior has tests
  • Linked issue, if there is one

Test coverage added (indexer/index.test.mjs)

Test Verifies
loadCursor returns null when state file does not exist Safe startup with no prior state
loadCursor returns null and warns on empty state file Zero-byte file (crash mid-truncate) falls back gracefully
loadCursor returns null and warns on corrupt state file Partial JSON falls back gracefully
saveCursor writes atomically and loadCursor reads back the cursor Full round-trip; .tmp file is cleaned up
saveCursor overwrites previous cursor atomically Second write replaces first; no .tmp left behind

All 10 tests in the suite pass (npm test).

Files changed

File Change
indexer/index.mjs saveCursor atomic write, loadCursor try/catch, stateFile() getter, entrypoint guard
indexer/index.test.mjs 5 new hermetic tests for cursor persistence

@A-one-tech
A-one-tech requested a review from Spagero763 as a code owner July 25, 2026 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants