Skip to content

fix(stella-store,stella-cli,stella-tui,stella-observatory): tell the two journal eras apart before styling a digest mismatch (#1981) - #2037

Merged
macanderson merged 9 commits into
mainfrom
fix/1981-journal-era-stamp
Aug 7, 2026
Merged

fix(stella-store,stella-cli,stella-tui,stella-observatory): tell the two journal eras apart before styling a digest mismatch (#1981)#2037
macanderson merged 9 commits into
mainfrom
fix/1981-journal-era-stamp

Conversation

@macanderson

@macanderson macanderson commented Aug 7, 2026

Copy link
Copy Markdown
Owner

What & why

A digest mismatch in a reconstructed context means two completely different things depending on who wrote the journal, and until now nothing recorded which.

Both readings are correct, for different journals. So the work was never "flip the colour back" — it was telling the two eras apart, then styling each honestly. Styling a legacy journal DANGER would have re-introduced the precise false alarm #1668 removed, narrowed to old journals.

Closes #1981

The era signal: approach (2), the stamp — as the issue recommended

executions.journal_era (schema v22), written by Store::begin_execution, read back by Store::reconstruct_call.

I looked hard for evidence against the recommendation and found none — the opposite. Approaches (1) and (2) infer the era from the absence of rewrites, and stella_core::driver::apply_overflow_summary compacts while legitimately journaling none: that execution is current-era and would be read as legacy, so a genuine integrity signal would be styled as housekeeping. That failure is silent and permanent. The column costs one additive ADD COLUMN; guessing costs the alarm. The reasoning is recorded on JournalEra itself so the next reader does not have to re-derive it.

Two design choices worth review:

  • Unknown reads as the oldest era. JournalEra::from_code maps anything but 1 to CompactionUnjournaled — a missing column, a missing row, a code written by a newer build after a downgrade. Every unknown can only under-alarm, never invent an alarm on someone's telemetry.
  • is_verified() stays era-blind. A mismatch happened or it did not; the era changes how loudly it is reported, never whether it is reported. Scripts gating on verified are unaffected.

One honest limitation, documented on the migration: rows written in the few days between PR #1979 and this stamp backfill to era 0 even though their journals do carry rewrites. That is conservative in the safe direction (under-report, never invent), and correcting it would need the very inference this column exists to avoid.

The witness

Primary — a real fail→pass, no API change in its call path: crates/stella-observatory/tests/journal_era.rs. Two executions differing in exactly one column drive the served /api/execution-context; the payloads must not agree.

Checked artisanally by stashing only crates/stella-observatory/src (the test file and respond's signature are untouched, so it compiles against main's code):

running 2 tests
test the_same_mismatch_reads_as_housekeeping_on_a_legacy_journal_and_an_alarm_on_a_current_one ... FAILED
test a_store_too_old_to_carry_the_era_column_stays_quiet ... FAILED

assertion `left == right` failed: a pre-#1667 journal mismatches on ordinary compaction …
  left: Null
  right: "compaction"

left: Null is the point: on main the two eras are indistinguishable — there is no severity to disagree about. Restored, both pass.

Supporting witnesses, one per surface:

  • crates/stella-store/src/reconstruct.rsthe_same_mismatch_is_housekeeping_on_a_legacy_journal_and_an_alarm_on_a_current_one: same block, same bytes, same rebuilt messages, different mismatch_severity(). The issue's definition of done, at the source.
  • crates/stella-tui/tests/deck_snapshot.rsfix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668's rendered-output witness, extended. The legacy half keeps its ["torn","altered","tamper"] prohibition and its "compaction rewrite" requirement; the new half renders the same mismatch in the other era through the real render_deck and asserts the two buffers differ. Updated by hand and read, never BLESS=1.
  • crates/stella-cli/tests/inspect_cli.rs — the two new keys on the shipped binary's own --format json.
  • crates/stella-store/src/migrations.rs — v22 backfills an existing row to era 0.
  • crates/stella-observatory/tests/schema_conformance.rs — the dashboard reads the real column the real writer set, so renaming it fails at cargo test rather than silently falling back to legacy for everyone.
  • crates/stella-tui/src/envelope.rs — both overlay lines fit an 80-column terminal. The doc comment claimed it; now it is checked. It caught a real 76-column line while I wrote it.

Surfaces touched, and how I hunted for the one I might miss

rg for digest_mismatch, digest_matches, digest_verified, re-hash, torn, was altered, mismatch across the whole workspace, docs/, and website/, then a check of every consumer of Store::reconstruct_call (there are no others — stella-serve, stella-pipeline and stella-fleet do not reconstruct).

The four the issue names:

  1. crates/stella-cli/src/inspect.rs — the banner (now a pure, tested digest_mismatch_line) and --format json, which gains journal_era + digest_mismatch_severity.
  2. crates/stella-cli/src/trace.rs — the severity rides in the recorded error string.
  3. crates/stella-tui/src/deck_render.rs + envelope.rs — the overlay.
  4. crates/stella-observatory/src/sent_context.rs — the /api/execution-context payload.

The issue warned a fifth might exist. Three did:

  1. crates/stella-observatory/src/assets/index.html — the dashboard's own verdict panel, which fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668 never reached. It read "the journal is torn or was altered" in the danger colour, for every era — the exact accusation the CLI stopped making, still being levelled at routine compaction on every dashboard in the field. It now styles from the served severity, and the per-message danger rail (sentContextMessageHtml) does too.
  2. website/content/docs/commands/inspect.mdx and website/content/docs/guides/what-a-run-cost.mdx — both still documented the pre-fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668 !! N block(s) did NOT re-hash → "The journal is torn or was altered" card.
  3. docs/spec/session-telemetry-receipts-spec.md §5.1 step 3, and crates/stella-observatory/README.md.

Every rendering surface now styles from one verdict — Reconstruction::mismatch_severity — rather than reasoning about the era itself. The three severity words and two era words are shared by convention across the observatory boundary (that crate links no store by design) and pinned by a test on each side, so a spelling that moves fails one of them.

The god-file constraint

crates/stella-tui/src/deck_render.rs was at its exact scripts/file-size-baseline.txt ceiling and its overlay clips rather than wraps, so neither the file nor the line could grow. The wording and the severity moved to InspectView::digest_mismatch_line in envelope.rs, and the renderer got shorter. scripts/file-size-baseline.txt is untouchedstella-store/src/lib.rs and src/tests.rs briefly went over their ceilings too and were trimmed back rather than re-baselined.

The gate

make gateexit code 0, checked with $?, not by grepping cargo output.

I inherited no pre-existing failures. The four breaks on main I was warned about (fmt trailing newline in stella-protocol/src/event/tests.rs, the driver.rs / pipeline/tests.rs size skew, the verification_hardening.rs dead code, the CompactionRewrite intra-doc link) had all landed via the in-flight unbreak PRs by the time I rebased onto 056b56d3; the branch is green on its own.

  • cargo fmt --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace
  • cargo doc -D warnings, file-size, god-files, wire-schema, left-behind, and the rest of GATE_STEPS
  • Docs updated (two website pages, the receipts spec, two crate READMEs, the dashboard page)

No stella-protocol type changed, so docs/wire/ needed no regeneration — check-wire-schema confirms it still matches.

Nothing left behind

Summary by Sourcery

Introduce journal era stamping to distinguish routine compaction-induced digest mismatches from real integrity failures and propagate a single mismatch severity verdict across store, CLI, TUI, observatory, schema, and docs.

Bug Fixes:

  • Correctly classify digest mismatches from pre-compaction-rewrite journals as benign housekeeping instead of integrity alarms in all rendering surfaces, including the observatory dashboard.

Enhancements:

  • Add JournalEra and MismatchSeverity to reconstruction, persist the era in the executions table with a v22 schema migration, and expose it via the store API.
  • Extend stella inspect JSON output and CLI banners with structured journal era and digest mismatch severity fields, and adjust trace logging to include severity.
  • Update the inspect overlay in the TUI to render digest mismatches differently per journal era while respecting layout constraints, and plumb era data through the deck command envelope.
  • Teach the observatory context reconstruction and dashboard UI to consume journal era and mismatch severity from the store and style verdicts and per-message rails accordingly.
  • Add targeted tests across store, CLI, TUI, and observatory to pin the two-era behavior and the shared naming of eras and severities.

Documentation:

  • Revise CLI and telemetry documentation to describe journal eras, the two readings of digest mismatches, and the new JSON fields, and update receipts spec to reflect era-dependent integrity interpretation.

Tests:

  • Add integration and unit tests validating journal era stamping, mismatch severity computation, JSON and dashboard payload shapes, and TUI overlay wording and width constraints.

A digest mismatch means two different things depending on who wrote the
journal, and until now nothing recorded which. Schema v22 adds
executions.journal_era, stamped by begin_execution, and
Reconstruction::mismatch_severity is the one place the two eras are told
apart.

Refs #1981
…ch by journal era (#1981)

All four rendering surfaces now read Reconstruction::mismatch_severity
instead of deciding for themselves: a mismatch on a pre-#1667 journal
keeps the benign wording #1668 introduced, and one on a journal that
records every compaction rewrite reads as the integrity signal it is.

Refs #1981
…h banner (#1981)

The website cards still carried the pre-#1668 wording ('the journal is
torn or was altered') for every era at once. They now name the two, and
the CLI's JSON verdict words are pinned by a test on each side of the
observatory boundary.

Refs #1981
Both variants were written to survive an 80-column terminal and the doc
comment said so; nothing checked it. Now it fails if a wording change
pushes the distinguishing half off the right edge.

Refs #1981, #2029
…s too

The Observatory renders the same digest-mismatch verdict the CLI does, so
its page says so rather than leaving the dashboard the one surface whose
documentation still implies a single meaning.

Closes #1981
The unit tests cover the wording and the store covers the stamp; this is
the one assertion made against `stella inspect --format json` as a user
runs it, so the two new keys cannot vanish from the shipped surface while
the library tests stay green.

Refs #1981
@vercel

vercel Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
stella-cli-docs Ignored Ignored Aug 7, 2026 4:37am

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@sourcery-ai

sourcery-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Introduce an explicit journal-era stamp in stella-store and thread it through CLI, TUI, and observatory so digest mismatches are interpreted and styled differently depending on whether the journal predates compaction rewrite journaling, while keeping verification logic era-neutral and updating docs, migrations, and tests accordingly.

File-Level Changes

Change Details Files
Add journal era and mismatch severity concepts to reconstruction and store schema, and base mismatch interpretation on the stamped era.
  • Introduce JournalEra and MismatchSeverity enums in reconstruct.rs and add journal_era field plus mismatch_severity() to Reconstruction.
  • Have Store::reconstruct_worker_step read executions.journal_era via a new Store::journal_era() helper, defaulting unknown/absent rows to the oldest era.
  • Stamp JournalEra::CURRENT into executions at Store::begin_execution and extend EXECUTIONS_DDL and MIGRATIONS with v21→v22 adding executions.journal_era (default 0) plus tests and schema version bump.
crates/stella-store/src/reconstruct.rs
crates/stella-store/src/lib.rs
crates/stella-store/src/migrations.rs
crates/stella-store/src/ddl.rs
crates/stella-store/src/tests.rs
Propagate journal era and mismatch severity into CLI outputs (text and JSON), and expose stable tags for other consumers.
  • Refactor inspect CLI to use a pure digest_mismatch_line(Reconstruction) that switches wording/severity marker based on MismatchSeverity from Reconstruction::mismatch_severity().
  • Extend inspect --format json reconstruction payload with journal_era and digest_mismatch_severity using stable string tags plus deck_journal_era() mapper for the TUI.
  • Update trace error formatting to include mismatch severity and add tests to pin JSON/tag behavior and machine-readability.
crates/stella-cli/src/inspect.rs
crates/stella-cli/src/trace.rs
crates/stella-cli/src/command_deck.rs
crates/stella-cli/tests/inspect_cli.rs
Extend TUI inspect overlay envelope and renderer to carry journal era and render two different mismatch banners within god-file and width constraints.
  • Add JournalEra enum and journal_era field to InspectView plus digest_mismatch_line() that returns both banner text and alarm flag, with tests ensuring both variants fit within 80-column terminals.
  • Change deck_render inspect overlay to delegate mismatch text and severity coloring to InspectView::digest_mismatch_line().
  • Thread JournalEra through TUI tests and re-export it from stella-tui::lib for use by the CLI deck command.
crates/stella-tui/src/envelope.rs
crates/stella-tui/src/deck_render.rs
crates/stella-tui/src/deck_render/tests.rs
crates/stella-tui/src/deck_ui/tests/help.rs
crates/stella-tui/tests/deck_snapshot.rs
crates/stella-tui/src/lib.rs
Teach observatory to read journal era from the database, compute mismatch severity, and drive dashboard styling and API payloads off that verdict.
  • Add a local JournalEra enum and journal_era() reader in sent_context.rs that defaults missing/unknown to the benign (unjournaled) era and extend reconstruct() to include journal_era and digest_mismatch_severity fields plus era-aware severity computation.
  • Update context_diff.rs to pass the looked-up journal era into reconstruct() for both target and baseline payloads.
  • Retune dashboard HTML/JS to style verdict panel and per-message danger rails based on digest_mismatch_severity, add tests (journal_era.rs, schema_conformance, execution_context tests) to pin both semantics and schema usage, and document coupling in the observatory README.
crates/stella-observatory/src/sent_context.rs
crates/stella-observatory/src/context_diff.rs
crates/stella-observatory/src/assets/index.html
crates/stella-observatory/src/tests.rs
crates/stella-observatory/tests/schema_conformance.rs
crates/stella-observatory/tests/journal_era.rs
crates/stella-observatory/README.md
Align documentation, website content, and specs with the two-era interpretation of digest mismatches and expose new JSON fields in examples.
  • Update inspect command and run-cost guide docs to describe the two mismatch readings, the new journal_era and digest_mismatch_severity JSON fields, and how scripts should consume them.
  • Clarify the receipts spec to explain era-dependent interpretation of mismatches using executions.journal_era.
  • Adjust dashboard docs to mention era-dependent mismatch reading and update store README schema version and migration commentary.
website/content/docs/commands/inspect.mdx
website/content/docs/guides/what-a-run-cost.mdx
docs/spec/session-telemetry-receipts-spec.md
website/content/docs/telemetry/dashboard.mdx
crates/stella-store/README.md

Assessment against linked issues

Issue Objective Addressed Explanation
#1981 Introduce a non-heuristic, explicitly recorded journal-era signal (preferably via schema change) and use it in reconstruction to distinguish legacy (pre-compaction-journaling) and current (compaction-journaling) journals.
#1981 Style digest mismatches differently for legacy vs current journals on all mismatch surfaces (CLI inspect, CLI trace, TUI INSPECT overlay, observatory/dashboard, and any other surfaces), with legacy staying a benign WARN naming compaction as the cause and current becoming a DANGER integrity signal, all driven from a single shared severity verdict.
#1981 Add tests/witnesses ensuring that the same unresolvable block on a legacy-era vs current-era journal yields different severities and that old journals/missing-era cases degrade safely; update TUI constraints and existing golden tests accordingly.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@macanderson
macanderson merged commit 7da7495 into main Aug 7, 2026
17 checks passed
@macanderson
macanderson deleted the fix/1981-journal-era-stamp branch August 7, 2026 04:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Restore the digest-mismatch surface to a real integrity signal now that compaction rewrites are journaled

1 participant