fix(stella-store,stella-cli,stella-tui,stella-observatory): tell the two journal eras apart before styling a digest mismatch (#1981) - #2037
Merged
Conversation
A digest mismatch means two different things depending on who wrote the journal, and until now nothing recorded which. Schema v22 adds executions.journal_era, stamped by begin_execution, and Reconstruction::mismatch_severity is the one place the two eras are told apart. Refs #1981
…ch by journal era (#1981) All four rendering surfaces now read Reconstruction::mismatch_severity instead of deciding for themselves: a mismatch on a pre-#1667 journal keeps the benign wording #1668 introduced, and one on a journal that records every compaction rewrite reads as the integrity signal it is. Refs #1981
…s too The Observatory renders the same digest-mismatch verdict the CLI does, so its page says so rather than leaving the dashboard the one surface whose documentation still implies a single meaning. Closes #1981
The unit tests cover the wording and the store covers the stamp; this is the one assertion made against `stella inspect --format json` as a user runs it, so the two new keys cannot vanish from the shipped surface while the library tests stay green. Refs #1981
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
There was a problem hiding this comment.
Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
Contributor
Reviewer's GuideIntroduce an explicit journal-era stamp in stella-store and thread it through CLI, TUI, and observatory so digest mismatches are interpreted and styled differently depending on whether the journal predates compaction rewrite journaling, while keeping verification logic era-neutral and updating docs, migrations, and tests accordingly. File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
A digest mismatch in a reconstructed context means two completely different things depending on who wrote the journal, and until now nothing recorded which.
call_idfallback to the pre-compaction bytes and mismatches — routinely, and forever. fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668 downgraded every mismatch surface to a benign WARN for exactly that reason.Both readings are correct, for different journals. So the work was never "flip the colour back" — it was telling the two eras apart, then styling each honestly. Styling a legacy journal DANGER would have re-introduced the precise false alarm #1668 removed, narrowed to old journals.
Closes #1981
The era signal: approach (2), the stamp — as the issue recommended
executions.journal_era(schema v22), written byStore::begin_execution, read back byStore::reconstruct_call.I looked hard for evidence against the recommendation and found none — the opposite. Approaches (1) and (2) infer the era from the absence of rewrites, and
stella_core::driver::apply_overflow_summarycompacts while legitimately journaling none: that execution is current-era and would be read as legacy, so a genuine integrity signal would be styled as housekeeping. That failure is silent and permanent. The column costs one additiveADD COLUMN; guessing costs the alarm. The reasoning is recorded onJournalEraitself so the next reader does not have to re-derive it.Two design choices worth review:
JournalEra::from_codemaps anything but1toCompactionUnjournaled— a missing column, a missing row, a code written by a newer build after a downgrade. Every unknown can only under-alarm, never invent an alarm on someone's telemetry.is_verified()stays era-blind. A mismatch happened or it did not; the era changes how loudly it is reported, never whether it is reported. Scripts gating onverifiedare unaffected.One honest limitation, documented on the migration: rows written in the few days between PR #1979 and this stamp backfill to era 0 even though their journals do carry rewrites. That is conservative in the safe direction (under-report, never invent), and correcting it would need the very inference this column exists to avoid.
The witness
Primary — a real fail→pass, no API change in its call path:
crates/stella-observatory/tests/journal_era.rs. Two executions differing in exactly one column drive the served/api/execution-context; the payloads must not agree.Checked artisanally by stashing only
crates/stella-observatory/src(the test file andrespond's signature are untouched, so it compiles againstmain's code):left: Nullis the point: onmainthe two eras are indistinguishable — there is no severity to disagree about. Restored, both pass.Supporting witnesses, one per surface:
crates/stella-store/src/reconstruct.rs—the_same_mismatch_is_housekeeping_on_a_legacy_journal_and_an_alarm_on_a_current_one: same block, same bytes, same rebuilt messages, differentmismatch_severity(). The issue's definition of done, at the source.crates/stella-tui/tests/deck_snapshot.rs— fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668's rendered-output witness, extended. The legacy half keeps its["torn","altered","tamper"]prohibition and its"compaction rewrite"requirement; the new half renders the same mismatch in the other era through the realrender_deckand asserts the two buffers differ. Updated by hand and read, neverBLESS=1.crates/stella-cli/tests/inspect_cli.rs— the two new keys on the shipped binary's own--format json.crates/stella-store/src/migrations.rs— v22 backfills an existing row to era 0.crates/stella-observatory/tests/schema_conformance.rs— the dashboard reads the real column the real writer set, so renaming it fails atcargo testrather than silently falling back to legacy for everyone.crates/stella-tui/src/envelope.rs— both overlay lines fit an 80-column terminal. The doc comment claimed it; now it is checked. It caught a real 76-column line while I wrote it.Surfaces touched, and how I hunted for the one I might miss
rgfordigest_mismatch,digest_matches,digest_verified,re-hash,torn,was altered,mismatchacross the whole workspace,docs/, andwebsite/, then a check of every consumer ofStore::reconstruct_call(there are no others —stella-serve,stella-pipelineandstella-fleetdo not reconstruct).The four the issue names:
crates/stella-cli/src/inspect.rs— the banner (now a pure, testeddigest_mismatch_line) and--format json, which gainsjournal_era+digest_mismatch_severity.crates/stella-cli/src/trace.rs— the severity rides in the recorded error string.crates/stella-tui/src/deck_render.rs+envelope.rs— the overlay.crates/stella-observatory/src/sent_context.rs— the/api/execution-contextpayload.The issue warned a fifth might exist. Three did:
crates/stella-observatory/src/assets/index.html— the dashboard's own verdict panel, which fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668 never reached. It read "the journal is torn or was altered" in the danger colour, for every era — the exact accusation the CLI stopped making, still being levelled at routine compaction on every dashboard in the field. It now styles from the served severity, and the per-message danger rail (sentContextMessageHtml) does too.website/content/docs/commands/inspect.mdxandwebsite/content/docs/guides/what-a-run-cost.mdx— both still documented the pre-fix(tui,cli,store): stop calling a compaction rewrite a torn journal #1668!! N block(s) did NOT re-hash → "The journal is torn or was altered"card.docs/spec/session-telemetry-receipts-spec.md§5.1 step 3, andcrates/stella-observatory/README.md.Every rendering surface now styles from one verdict —
Reconstruction::mismatch_severity— rather than reasoning about the era itself. The three severity words and two era words are shared by convention across the observatory boundary (that crate links no store by design) and pinned by a test on each side, so a spelling that moves fails one of them.The god-file constraint
crates/stella-tui/src/deck_render.rswas at its exactscripts/file-size-baseline.txtceiling and its overlay clips rather than wraps, so neither the file nor the line could grow. The wording and the severity moved toInspectView::digest_mismatch_lineinenvelope.rs, and the renderer got shorter.scripts/file-size-baseline.txtis untouched —stella-store/src/lib.rsandsrc/tests.rsbriefly went over their ceilings too and were trimmed back rather than re-baselined.The gate
make gate— exit code 0, checked with$?, not by grepping cargo output.I inherited no pre-existing failures. The four breaks on
mainI was warned about (fmttrailing newline instella-protocol/src/event/tests.rs, thedriver.rs/pipeline/tests.rssize skew, theverification_hardening.rsdead code, theCompactionRewriteintra-doc link) had all landed via the in-flight unbreak PRs by the time I rebased onto056b56d3; the branch is green on its own.cargo fmt --checkcargo clippy --workspace --all-targets -- -D warningscargo test --workspacecargo doc -D warnings,file-size,god-files,wire-schema,left-behind, and the rest ofGATE_STEPSNo
stella-protocoltype changed, sodocs/wire/needed no regeneration —check-wire-schemaconfirms it still matches.Nothing left behind
deck_render.rsceiling a fixer will hit.stella tracerecords the reconstruction verdict as prose. It now contains the severity word, but every other surface got a structured field and the trace record is the one most likely to be read by a machine.Summary by Sourcery
Introduce journal era stamping to distinguish routine compaction-induced digest mismatches from real integrity failures and propagate a single mismatch severity verdict across store, CLI, TUI, observatory, schema, and docs.
Bug Fixes:
Enhancements:
JournalEraandMismatchSeverityto reconstruction, persist the era in theexecutionstable with a v22 schema migration, and expose it via the store API.stella inspectJSON output and CLI banners with structured journal era and digest mismatch severity fields, and adjust trace logging to include severity.Documentation:
Tests: