Skip to content

Commit 05de689

Browse files
AIQnetLabclaude
andcommitted
state: cut Merkle finalize from seconds to ~0.3s, drop no-op store tombstones
Merkle finalization was 99% of block-apply cost — ~250 us per dirty leaf, ~2 s for a loaded block — measured in isolation: applying 4000 transfers takes 25 ms, finalizing them took 2.0 s. The live 382 TPS single-transfer ceiling and the 3-5 s batch-block applies were both this one function. recompute_levels rewritten in three layers, root byte-identical at every step: - Pair grouping in one HashMap pass per level, replacing the per-level key vector + processed-set churn. Order is provably irrelevant: the root is a pure function and node keys are distinct. - Pair hashing runs in parallel past a 64-job threshold; light blocks keep the sequential path with no fork-join overhead. - LONE CLIMB: below ffd-1 a leaf provably has no neighbour, so its whole path segment is a tight hash+delete loop with no map traffic, parallel per leaf; entries join the level machinery at their meet depth via buckets. Climb regions of distinct leaves are disjoint (a shared ancestor implies a foreign leaf, contradicting ffd), so the emitted delete set is exactly the old one. Map traffic drops from O(k x 256) to O(k x meet-levels). Incremental finalize over 6.6k dirty leaves: 2.0 s -> 322 ms. node_del pushed a store tombstone for every never-existing row — a loaded block's climb emitted ~3M no-op deletes into the store per finalize. New node_cache_complete invariant (true after a full rebuild, false on eviction or store attach): while the cache provably mirrors the store, a cache miss proves store absence and the tombstone is skipped. The store has a single writer (the tree flush), so no side channel can invalidate the invariant; under a full rebuild the flush wipes the node set anyway. Watchdog: new apply_starved CRIT — apply op idle while verified-minus-applied grows. A live wedge had exactly this signature and the idle exemption kept the watchdog silent through it. Tests: 177 state (both incremental-vs-full equivalence tests, a new randomized insert+update one, and the store-backed eviction test all pass on the new path), 420 integration, zero warnings. Profiling benches added as #[ignore]. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent f8f01ab commit 05de689

2 files changed

Lines changed: 405 additions & 43 deletions

File tree

0 commit comments

Comments
 (0)