Supersede the legacy promotion to retire a stale expected failure - #177
Merged
Merged
Conversation
`cpp-parity-function-like-macro-expanded-call` is annotated `expectedFailure`, but Bifrost navigates it correctly and the v0.3.1 legacy snapshot reports it `improved` — the runner's signal for a case that passes while annotated. The annotation cannot be edited away. Its document is content-addressed by the promotion manifest that binds it, and corrections to this tier are append-only. So the retirement is recorded as an overlay instead, which is what promotion already is. A superseding manifest entry may carry `retiredExpectedFailure`, holding the annotation's reason verbatim and binding the evidence that the navigation now succeeds. The validator requires that the historical document actually authors the annotation, that the reason matches it byte-for-byte, and that the manifest carries a `supersedes` link. Retirement is one-way: a superseding manifest may retire more expectations but may not restore one its predecessor retired, so a later manifest cannot reinstate an excuse for a case already held to a pass. Execution staging drops the annotation from its filtered, execution-only copy, so the case scores as an ordinary pass. `benchmarks/cases/cpp-lsp-parity.yaml` stays byte-identical and `legacy-promotion-v1-balanced-core` keeps validating against it. `legacy-promotion-v2-balanced-core` carries the same 110 reviewed cases and retires this one annotation. Its evidence is the frozen v0.3.1 container run, which is the reference-environment confirmation the issue asked for. Freeze scope and the reproduction contract move to it; the docs evidence map stays on v1 until a legacy release is published under v2. Closes #164 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #164.
cpp-parity-function-like-macro-expanded-callcarries anexpectedFailureannotation that Bifrost no longer earns. The v0.3.1 legacy snapshot reports it
improved— the runner's signal for a case that passes while annotated.The issue asked to remove the annotation from
benchmarks/cases/cpp-lsp-parity.yaml. That is not available: the file iscontent-addressed by the promotion manifest that binds it, and corrections to
this tier are append-only. So the retirement is recorded as an overlay, which
is what promotion already is.
What this adds
A case entry in a promotion manifest may carry
retiredExpectedFailure:The validator requires that
what was never there;
supersededReasonmatches it byte-for-byte, so the manifest records what waswithdrawn rather than deleting it, and cannot drift from the frozen text;
supersedes, because a retirement is a correction; andexpectations but may not restore one its predecessor retired. A later
manifest cannot reinstate an excuse for a case already held to an ordinary
pass.
Execution staging drops the annotation from its filtered, execution-only copy
of the corpus, so the case scores as an ordinary pass. The historical YAML
stays byte-identical, and
legacy-promotion-v1-balanced-corekeeps validatingagainst it — both manifests validate in CI.
The evidence
The retirement rests on the frozen v0.3.1 legacy snapshot rather than a fresh
native run, which is the reference-container confirmation #164 asked for:
container,platformScope: canonical_referenceimprovedinclude/parity.h:45:20— the authored targetScope moved, docs held back
resolve-freeze-scope.sh,run-freeze-candidates.sh, and the reproductioncontract now point at
legacy-v2, so the next legacy freeze runs under it.The docs evidence map deliberately stays on
legacy-v1: it describes themanifest the published v0.3.1 result was frozen under. It moves forward with
the legacy release that supersedes it, not ahead of it.
Verification
cargo test --locked— 247 passed (5 new), 0 failedpython3 -m unittest discover tests— 42 tests (2 new), OKvalidate benchmarks/cases— 61 filesvalidate-legacy-promotionon both v1 and v2validate-legacy-promotion-cohort— N=10, 158 rowsvalidate-reproduction-contract.sh— exit 0staged corpus still validates as 30 documents / 110 cases
🤖 Generated with Claude Code