Skip to content

Restore the digest-mismatch surface to a real integrity signal now that compaction rewrites are journaled #1981

Description

@macanderson

Problem

#1667 (PR #1979) closed the gap that made a digest mismatch a routine event: compaction now journals its replacement bytes, and stella_store::reconstruct::resolve_content resolves a compacted tool_result block by digest, so on a journal written by current Stella the bytes shown are the ones the step actually sent.

That was the stated precondition for restoring the alarm. #1667's definition of done says:

The mismatch surface then fires only when the digest genuinely cannot be resolved — i.e. it becomes a real integrity signal again, and can arguably go back to DANGER.

PR #1979 deliberately did not do this, because the reasoning is not as simple as reverting #1668's wording. A journal written before #1979 has no rewrites on its Compaction events, so every compacted block in it still resolves through the call_id fallback and still reports a mismatch — benignly, and forever. Styling that as DANGER would re-introduce exactly the false alarm #1668 removed, just narrowed to old journals.

So the real task is not "flip the colour back" — it is telling the two eras apart, then styling each honestly.

What needs deciding first

There is currently no journal-era signal. Candidate approaches, none yet chosen:

  1. Infer from the event. If any Compaction event in the execution carries a non-empty rewrites, the writer was post-fix(stella-core,stella-store): journal compaction rewrites so 'context sent' shows the bytes the model received (#1667) #1979. Cheap and needs no schema change, but wrong for an execution that compacted only in ways that produced no records, and for a turn where the only compaction is an overflow summary splice (which legitimately journals no rewrites — see driver.rs::apply_overflow_summary).
  2. A schema/era stamp. Record the writer's journal era on the execution row, so the reader can ask directly. Honest and unambiguous; costs a stella-store schema migration.
  3. Per-block. Distinguish "this block's digest is unresolved and its execution journaled rewrites" (real integrity signal) from "unresolved and the execution journaled none" (legacy). A refinement of (1) with the same blind spot.

Recommend (2) unless review disagrees — (1) and (3) both guess, and the whole point of this surface is not guessing.

Files

  • crates/stella-store/src/reconstruct.rsReconstruction::digest_mismatches, resolve_content, content_matches_digest, and the module docs (already rewritten by fix(stella-core,stella-store): journal compaction rewrites so 'context sent' shows the bytes the model received (#1667) #1979 to describe both eras)
  • crates/stella-cli/src/inspect.rsprint_reconstruction, the ! WARN line (:452); JSON field digest_mismatches (:902)
  • crates/stella-cli/src/trace.rs — the second surface (~:231), which must not drift from inspect.rs
  • crates/stella-tui/src/deck_render.rsrender_inspect_overlay mismatch line (~:784). God file at its ceiling — no new lines. The overlay clips rather than wraps, so the line cannot grow either.
  • crates/stella-tui/src/envelope.rsInspectView::digest_mismatches
  • crates/stella-observatory/src/sent_context.rsdigest_matches, and the verified / digest_mismatches JSON keys

Definition of done

  • A journal's era is determined by a signal that does not guess (see above), not inferred from absence.
  • A mismatch on a current journal renders as a genuine integrity signal (DANGER), on every surface: inspect, trace, the TUI overlay, and the observatory.
  • A mismatch on a legacy journal keeps reading as the benign WARN fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668 introduced, with wording that names compaction as the cause.
  • All four surfaces agree. fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668 had to touch all of them at once for exactly this reason; a fifth surface added since must be found and included.
  • Witness: a reconstruction over a legacy-era journal and one over a current-era journal with the same unresolvable block produce different severities. That test fails today because both produce WARN.

Constraints

  • crates/stella-tui/src/deck_render.rs is a god file at its exact ceiling (scripts/file-size-baseline.txt) and its overlay clips — fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668 used a code comment rather than a second rendered line for precisely this reason.
  • crates/stella-tui/tests/deck_snapshot.rs holds fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668's rendered-output witness (asserts "did not re-hash", forbids ["torn","altered","tamper"], requires "compaction rewrite"). It must be updated deliberately, not blessed — a golden blessed without reading the diff is a changelog, not a test.
  • Option (2) means a stella-store schema migration; see crates/stella-store/src/migrations.rs and the v18→v19 migration in migrations/token_unit.rs for the established shape.

Refs #1667, #1668, #1979

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:storestella-store — DuckDB persistence & telemetrytriageUntyped request — convert by adding bug / feature / epic

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions