Simplex already performs the equivalent of header sync above the finalized tip, retaining the selected branch's commitments in memory. Its application calls expose only the immediate parent. Marshal and glue then rediscover ancestry by fetching block bodies backward, retaining them to replay forward.
Pass those commitments as a separate argument through Automaton::propose/verify to Application, including certify when verification happens there. Follow Simplex's selected parent links, including optimistic dependencies. Its consensus behavior and Context stay unchanged.
Marshal should expose one block acquisition path keyed by the full consensus commitment. Applications select the required ancestry range and consume it forward: read local bodies on demand, fetch missing bodies concurrently, and release consumed blocks. Bound in-flight and buffered bodies, deduplicate shared requests, and cancel unused demand. Marshal's finalized storage supplies the canonical prefix when an application trails consensus.
Migrate every caller directly, with no backward API compatibility or adapters. Delete:
- The backward ancestry API:
Ancestry, AncestorStream, BlockProvider, parent subscriptions, and boxed/prefixed stream machinery.
- Duplicate digest/commitment subscriptions,
DigestFallback, CommitmentFallback, and hint_notarized.
- Request-origin annotations, source-dependent delivery routing, and the notarized/finalized block client, including certificate-response parsing, batching,
PendingVerification, and verify_delivered.
- Glue's backward replay vectors,
WeakAncestry, fetch_ancestor, and candidate/parent consume-and-re-prefix plumbing. Provide candidate/parent access explicitly.
Migrate inline, deferred, and coding Marshal and their consumers. In glue, replace stateful's parent walk and replay vector and DKG's backward inclusion scan with forward processing. Move height-only finality discovery (hint_finalized and its callers) to existing consensus/probe paths and repair missing bodies by commitment.
Keep validation and persistence driven by block/consensus state. Preserve commitment, linkage, coding-root and application-state checks, shared execution, durable delivery, and DKG log ordering.
Keep serving legacy notarized/finalized requests for older peers; new clients issue only commitment requests. Wire/storage encodings, commitment derivation, and archive layouts remain unchanged.
Validation:
- Deterministic tests demonstrate forward processing before the entire range loads, concurrent fetching, bounded retained blocks, and cancellation with overlapping consumers. Cover optimistic dependencies, malformed bodies, certification without prior verification, application lag, sparse recovery, pruning, and epoch transitions.
- Existing wire/storage fixtures and legacy request-serving tests pass, alongside relevant consensus/glue tests, linting, and stability checks.
- Run serial before/after glue benchmarks across ancestry depths, block sizes, and local/missing bodies. Report retained block bytes, peak RSS, proposal/verification latency, requests, and disk reads. The expected gain is retaining commitment metadata plus a bounded block window instead of the entire replay path; measure the result and document the code/state removed.
Related: #2083, #3718, #3720, #1809.
Simplex already performs the equivalent of header sync above the finalized tip, retaining the selected branch's commitments in memory. Its application calls expose only the immediate parent. Marshal and glue then rediscover ancestry by fetching block bodies backward, retaining them to replay forward.
Pass those commitments as a separate argument through
Automaton::propose/verifytoApplication, includingcertifywhen verification happens there. Follow Simplex's selected parent links, including optimistic dependencies. Its consensus behavior andContextstay unchanged.Marshal should expose one block acquisition path keyed by the full consensus commitment. Applications select the required ancestry range and consume it forward: read local bodies on demand, fetch missing bodies concurrently, and release consumed blocks. Bound in-flight and buffered bodies, deduplicate shared requests, and cancel unused demand. Marshal's finalized storage supplies the canonical prefix when an application trails consensus.
Migrate every caller directly, with no backward API compatibility or adapters. Delete:
Ancestry,AncestorStream,BlockProvider, parent subscriptions, and boxed/prefixed stream machinery.DigestFallback,CommitmentFallback, andhint_notarized.PendingVerification, andverify_delivered.WeakAncestry,fetch_ancestor, and candidate/parent consume-and-re-prefix plumbing. Provide candidate/parent access explicitly.Migrate inline, deferred, and coding Marshal and their consumers. In glue, replace stateful's parent walk and replay vector and DKG's backward inclusion scan with forward processing. Move height-only finality discovery (
hint_finalizedand its callers) to existing consensus/probe paths and repair missing bodies by commitment.Keep validation and persistence driven by block/consensus state. Preserve commitment, linkage, coding-root and application-state checks, shared execution, durable delivery, and DKG log ordering.
Keep serving legacy notarized/finalized requests for older peers; new clients issue only commitment requests. Wire/storage encodings, commitment derivation, and archive layouts remain unchanged.
Validation:
Related: #2083, #3718, #3720, #1809.