UNVERIFIED reports 'no flip' on turns where verify_done returned WITNESS CONFIRMED

[macanderson]

Problem

A run can end with

error: verification could not be performed: UNVERIFIED: verifier passed with no
deterministic corroboration (no flip, no green test)

on a turn where a fail→pass flip demonstrably happened and is sitting in the transcript — the worker called the verify_done tool and got back WITNESS CONFIRMED, with the previous-code failure tail printed.

Observed in a real run: the worker wrote regex.txt, authored test_regex.py, called verify_done, and received

WITNESS CONFIRMED - deterministic definition of done met:
- new code:      `python3 test_regex.py` exit 0 (PASS)
- previous code: HEAD 020fb5d9 + your test files -> exit 1 (FAIL)

The run then reported "no flip".

Why it happens (and why the behaviour is probably right)

stands_alone in crates/stella-pipeline/src/replay.rs is exactly two conjuncts:

!snapshot.flip_achieved && snapshot.touched_tests_passed != Some(true)

flip_achieved is set only by the pipeline's own flip oracle. verify_done is a tool the worker invokes, and the pipeline deliberately does not credit a worker-run witness — that is the tamper-exclusion rule in AGENTS.md § "The definition of done: witness tests". So refusing to count it is defensible.

crates/stella-pipeline/src/pipeline.rs (see the verifier_evidence_demand doc comment, ~line 417) already documents the structural half: with no --test-command, touched_tests_passed stays None by construction and the oracle never observes a candidate run, so the condition holds "no matter what the worker does with the turn".

What is actually wrong

The message, not the decision. "no flip, no green test" states a fact about the world that the transcript contradicts, and it is the last line a user or a benchmark harness reads. It should distinguish:

• no flip was observed and none was attempted; from
• no flip was observed by a channel this run can trust — a worker-run verify_done does not count, and here is why.

On Terminal-Bench-shaped runs (no --test-command) the second case is the common one, so the misleading wording is what most operators see most of the time.

Files

• crates/stella-pipeline/src/replay.rs — stands_alone, CalibrationReport::uncorroborated_verdicts
• crates/stella-pipeline/src/verify.rs — LadderInputs::verifier_pass_stands_alone
• crates/stella-pipeline/src/pipeline/witness_stage.rs — unverifiable, which formats the string
• crates/stella-pipeline/tests/fixtures/golden/verifier_escalation_without_a_test_command.jsonl — pins the current wording

Definition of done

Either the message names the distinction, or the run records that a worker-run verify_done reported WITNESS CONFIRMED and the message says it was seen and not credited. A reader must not be able to conclude "nothing was proven" when something was proven by a channel this run declines to trust.

If the decision itself should change — i.e. a worker-run verify_done counts when the witness files pass the same tamper exclusion the authored-witness stage applies — that is a maintainer's call and should be settled on this issue before any code moves. Do not change the semantics as a side effect of fixing the wording.

Constraints

• The golden fixture above embeds the current string; regenerate it with make record-golden and read the diff rather than blessing it.
• Any wording change must keep check-gate-parity and the replay-fixture manifests green.
• Related: PR fix(stella-pipeline): let the verifier read an untracked file's content, not just its name #2034 (verifier reads untracked content), Revisit asking for more evidence when only the judge approved the work #1295 (the uncorroborated-verdict rate), pipeline todo #1211 §1 (why the evidence demand is gated on a test command).

[macanderson]

Closing as fixed — the runtime defect this issue names no longer reproduces on origin/main.

PR #2191 (issue #2129) added LadderInputs::verify_done_flip (crates/stella-pipeline/src/verify.rs:445, fed from pipeline/evidence.rs:85) and wired it into both decision paths:

• verifier_pass_stands_alone — verify.rs:511 is now !self.flip_achieved && self.touched_tests_passed != Some(true) && !self.verify_done_flip
• the heuristic fallback — verify.rs:1040 and :1045

So a turn whose verify_done returned WITNESS CONFIRMED is no longer reported UNVERIFIED for want of a flip.

The residue is real and is tracked by #2194, not lost here. stella_protocol::LadderSnapshot has no verify_done_flip field, so replay.rs:366's stands_alone still reads only the two old conjuncts — meaning CalibrationReport::uncorroborated_verdicts now over-counts against a predicate it claims to mirror. I have added that consequence explicitly to #2194.