Skip to content

Supersede the legacy promotion to retire a stale expected failure - #177

Merged
DavidBakerEffendi merged 1 commit into
mainfrom
dave/retire-cpp-macro-expected-failure
Aug 31, 2026
Merged

Supersede the legacy promotion to retire a stale expected failure#177
DavidBakerEffendi merged 1 commit into
mainfrom
dave/retire-cpp-macro-expected-failure

Conversation

@DavidBakerEffendi

Copy link
Copy Markdown
Collaborator

Closes #164.

cpp-parity-function-like-macro-expanded-call carries an expectedFailure
annotation that Bifrost no longer earns. The v0.3.1 legacy snapshot reports it
improved — the runner's signal for a case that passes while annotated.

The issue asked to remove the annotation from
benchmarks/cases/cpp-lsp-parity.yaml. That is not available: the file is
content-addressed by the promotion manifest that binds it, and corrections to
this tier are append-only. So the retirement is recorded as an overlay, which
is what promotion already is.

What this adds

A case entry in a promotion manifest may carry retiredExpectedFailure:

"retiredExpectedFailure": {
  "supersededReason": "<the annotation, verbatim>",
  "evidence": { "file": "...", "sha256": "..." }
}

The validator requires that

  • the historical document actually authors the annotation — you cannot retire
    what was never there;
  • supersededReason matches it byte-for-byte, so the manifest records what was
    withdrawn rather than deleting it, and cannot drift from the frozen text;
  • the manifest carries supersedes, because a retirement is a correction; and
  • retirement is one-way — a superseding manifest may retire more
    expectations but may not restore one its predecessor retired. A later
    manifest cannot reinstate an excuse for a case already held to an ordinary
    pass.

Execution staging drops the annotation from its filtered, execution-only copy
of the corpus, so the case scores as an ordinary pass. The historical YAML
stays byte-identical, and legacy-promotion-v1-balanced-core keeps validating
against it — both manifests validate in CI.

The evidence

The retirement rests on the frozen v0.3.1 legacy snapshot rather than a fresh
native run, which is the reference-container confirmation #164 asked for:

execution mode container, platformScope: canonical_reference
status improved
queries 2 of 2 successful, 2 of 2 exact-set
usage→declaration include/parity.h:45:20 — the authored target
false positives / negatives 0 / 0

Scope moved, docs held back

resolve-freeze-scope.sh, run-freeze-candidates.sh, and the reproduction
contract now point at legacy-v2, so the next legacy freeze runs under it.

The docs evidence map deliberately stays on legacy-v1: it describes the
manifest the published v0.3.1 result was frozen under. It moves forward with
the legacy release that supersedes it, not ahead of it.

Verification

  • cargo test --locked — 247 passed (5 new), 0 failed
  • python3 -m unittest discover tests — 42 tests (2 new), OK
  • validate benchmarks/cases — 61 files
  • validate-legacy-promotion on both v1 and v2
  • validate-legacy-promotion-cohort — N=10, 158 rows
  • validate-reproduction-contract.sh — exit 0
  • Staging under v1 vs v2 differs by exactly the two annotation lines, and the
    staged corpus still validates as 30 documents / 110 cases

🤖 Generated with Claude Code

`cpp-parity-function-like-macro-expanded-call` is annotated `expectedFailure`,
but Bifrost navigates it correctly and the v0.3.1 legacy snapshot reports it
`improved` — the runner's signal for a case that passes while annotated.

The annotation cannot be edited away. Its document is content-addressed by the
promotion manifest that binds it, and corrections to this tier are append-only.
So the retirement is recorded as an overlay instead, which is what promotion
already is.

A superseding manifest entry may carry `retiredExpectedFailure`, holding the
annotation's reason verbatim and binding the evidence that the navigation now
succeeds. The validator requires that the historical document actually authors
the annotation, that the reason matches it byte-for-byte, and that the manifest
carries a `supersedes` link. Retirement is one-way: a superseding manifest may
retire more expectations but may not restore one its predecessor retired, so a
later manifest cannot reinstate an excuse for a case already held to a pass.

Execution staging drops the annotation from its filtered, execution-only copy,
so the case scores as an ordinary pass. `benchmarks/cases/cpp-lsp-parity.yaml`
stays byte-identical and `legacy-promotion-v1-balanced-core` keeps validating
against it.

`legacy-promotion-v2-balanced-core` carries the same 110 reviewed cases and
retires this one annotation. Its evidence is the frozen v0.3.1 container run,
which is the reference-environment confirmation the issue asked for. Freeze
scope and the reproduction contract move to it; the docs evidence map stays on
v1 until a legacy release is published under v2.

Closes #164

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@DavidBakerEffendi
DavidBakerEffendi merged commit e7319f6 into main Aug 31, 2026
5 checks passed
@DavidBakerEffendi
DavidBakerEffendi deleted the dave/retire-cpp-macro-expected-failure branch August 31, 2026 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Retire the expected failure on cpp-parity-function-like-macro-expanded-call

1 participant