feat(stella-pipeline): enforce and record verifier independence for the verdict call

[macanderson]

What & why

The witness author has a hard independence gate; the verdict call had none — with one configured provider, the "independent code reviewer" is the same model that wrote the code, its PASS ends the run, and the only trace is a once-per-run prose caveat that scrolls away (the 46%-agreement measurement in verify.rs was made under exactly this condition). Two halves, per the issue:

• Opt-in refusal, before spend. PipelineConfig::require_independent_verifier mirrors require_independent_witness — same shared probe (witness_author_independence, worker-vs-verifier model-ref), same placement ahead of triage so a refused run costs $0, its own error variant (PipelineError::VerifierNotIndependent). Default off: a single-provider BYOK seat keeps working. Reachable via the new settings key pipeline_require_independent_verifier (settings.json and stella.toml), wired in apply_pipeline_tuning so every driver honours it. The probe's reason wording was made role-neutral ("no model independent of the worker resolves") since three framings now share it.
• The structured fact, unconditionally. Every model verdict's LadderSnapshot now carries verifier_independent: Option<bool> — Some(false) = self-graded — stamped at the call seam (Pipeline::verifier) where the actual resolution is in hand, threaded through verify::Verdict, and additive on the wire (absent on deterministic/waived/abstain rungs, on pre-existing snapshots, and when the worker itself is unresolvable). replay::verdict_provenance narrates it (grader=self-graded (worker's own model)), and docs/wire/ is regenerated.

Exemplar: the existing require_independent_witness gate (#1147) — same refusal shape, same before-spend contract — and diff_coverage's three-valued honesty for why the fact is Option<bool>, never a defaulted bool.

Two god-file limits would have been newly crossed by this diff, so two test extractions ride along (repo rule: split, not grow): replay/late_reconciliation_tests.rs out of replay.rs, and verify/tests/witness_strip.rs out of verify/tests.rs. The regenerated baseline tightens pipeline.rs (3642 → 3616) and grows only the two irreducible one-liners (the mod declaration in pipeline/tests.rs, the required field in event.rs's test constructor).

Closes #1795

The witness

• This PR includes a witness test (fails on main, passes here)

Flip-checked by restoring the old behavior in place and re-running — three behavioral witnesses fail, the two invariant guards pass on both sides, as designed:

• verifier_independence::requiring_an_independent_verifier_refuses_before_spending_anything — refusal with an empty provider script, $0, no stage opened
• verifier_independence::a_self_graded_verdict_states_the_fact_on_its_snapshot — Some(false) on the stored verdict
• verifier_independence::an_independent_verdict_states_that_too — Some(true), distinguishable from pre-fact snapshots
• verifier_independence::the_verifier_independence_requirement_is_opt_in / ..._lets_a_distinct_verifier_through — guards (pass on both)

Plus protocol round-trip + additive-parse tests in ladder.rs (invariant 4).

The gate

• cargo fmt --check
• cargo clippy --workspace --all-targets -- -D warnings — clean except the two pre-existing witness_stage.rs warnings fix(stella-pipeline): unbreak main — two clippy failures merged with #1813 #1859 unbreaks; none in files this PR touches
• cargo test — stella-pipeline (549), stella-protocol, stella-cli all green; rustdoc -D warnings clean; make guards green except gate-parity, which is red on main and owned by fix(gate): unbreak main — file-size baseline skew and a stale gate-parity count #1845
• Docs updated (field docs, settings-key docs, wire schema regenerated)
• CLA signed
• Closes #N above and as a commit trailer

CI note: until #1845 (gate-parity) and #1859 (witness_stage clippy) merge, the required job will be red on those pre-existing steps; I'll gh pr update-branch after they land.

Nothing left behind

• Filed: calibration: partition verifier false-positive rates by the new grader-independence fact #1865 — partition the calibration false-positive rates by the new grader-independence fact (the reading side of Turn on the independent checker and measure whether it helps #1284); trusted posture auto-sets require_independent_witness but not require_independent_verifier — equivalent today, drifts if the probes diverge #1866 — the trusted posture auto-sets the witness flag but not the verifier flag (equivalent today via the shared probe, drifts if the probes diverge; related agents.verifier overrides never reach the witness author, which runs the verifier's model with the worker's tuning #1785)

Ground-rule check

• No I/O added to stella-core; no new deps
• New cross-boundary field round-trips through serde (test included; additive, alias-free)

Anything reviewers should know?

• The refusal also fires when the verifier role is unresolvable (not only same-model): a caller requiring an independent verifier is also refusing "no verifier at all", where the verdict would degrade to the heuristic — the conservative read, stated in the error.
• The heuristic-fallback verdict leaves the fact absent on purpose: no model answered, so grader independence is not a fact about it (mirrors diff_coverage's unmeasured honesty).
• The uncorroborated-PASS relabel (Unverifiable) also leaves it absent — by the field's own contract the fact rides only the ModelVerdict rung; calibration: partition verifier false-positive rates by the new grader-independence fact #1865 covers reading it at scale.

Summary by Sourcery

Enforce optional independence between worker and verifier models for verdicts and record verifier independence as structured data on verdict snapshots, while keeping wire formats and configuration in sync and extracting oversized tests into dedicated modules.

New Features:

• Add a configurable pipeline option to require the VERDICT call to use a model independent of the worker, refusing runs before any spend when independence cannot be ensured.
• Record whether a verdict’s grader model is independent of the worker on LadderSnapshot and propagate this fact through verdicts, events, and replay provenance narration.

Enhancements:

• Update verifier routing and error handling to use role‑neutral wording for independence failures and introduce a dedicated error for non‑independent verifiers.
• Expose the new verifier-independence requirement through CLI settings (JSON and TOML) and apply it when constructing pipeline configurations.
• Extend replay provenance strings to describe whether a verdict was self‑graded or independently graded.

Documentation:

• Regenerate and extend wire documentation and TypeScript definitions to include the new verifier_independent field on LadderSnapshot.

Tests:

• Add end-to-end tests covering the verifier-independence requirement, verifier independence recording on snapshots, and the opt‑in nature of the new flag.
• Add serialization and wire-contract tests to ensure LadderSnapshot with verifier_independent remains backward compatible and round-trips correctly.
• Extract late reconciliation tests from replay.rs and witness-strip tests from verify/tests.rs into separate modules to stay within file size limits.

[sourcery-ai]

Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
stella-cli-docs	Ignored	Preview	Aug 6, 2026 11:04am

[sourcery-ai]

Reviewer's Guide

Adds an opt-in hard gate to require an independent verifier model for the verdict call, records verifier independence on LadderSnapshot for all model verdicts, and performs small refactors/tests splits to keep large files within size limits while updating wire formats and settings plumbing accordingly.

Sequence diagram for verdict grading with verifier independence gate

sequenceDiagram
    actor Operator
    participant AgentEngineConfig
    participant PipelineConfig
    participant Pipeline
    participant VerifierStage
    participant Verdict
    participant LadderSnapshot

    Operator->>AgentEngineConfig: set pipeline_require_independent_verifier
    AgentEngineConfig->>PipelineConfig: apply_pipeline_tuning(cfg, config)
    AgentEngineConfig-->>PipelineConfig: config.require_independent_verifier = pipeline_require_independent_verifier_on()

    Operator->>Pipeline: start_run(config)
    Pipeline->>Pipeline: witness_author_independence()
    alt require_independent_verifier && Unavailable
        Pipeline-->>Operator: Err(PipelineError::VerifierNotIndependent)
    else gate_allows_run
        Pipeline->>VerifierStage: verifier(request)
        VerifierStage->>VerifierStage: resolve_provider(Role::Verifier)
        VerifierStage->>VerifierStage: resolve_provider(Role::Worker)
        VerifierStage->>VerifierStage: parse_verifier_response(result.text)
        VerifierStage-->>Verdict: Verdict { verifier_independent: None }
        VerifierStage->>Verdict: set verifier_independent = Some(worker.model_ref != resolved.model_ref)
        VerifierStage-->>Pipeline: Verdict

        Pipeline->>LadderSnapshot: with_rung(verdict.rung())
        Pipeline->>LadderSnapshot: with_verifier_independence(verdict.verifier_independent)
        Pipeline-->>Operator: AgentEvent::Verdict { evidence.ladder = LadderSnapshot }
    end

Loading

File-Level Changes

Change	Details	Files
Introduce an opt-in pipeline gate that refuses runs when the verdict grader would not be independent of the worker, using the existing witness independence probe and new error wiring.	Extend PipelineConfig with a require_independent_verifier flag (default false) and enforce it in Pipeline::verifier using witness_author_independence before any paid calls are made. Add PipelineError::VerifierNotIndependent with user-facing error text, and reuse the neutral "no model independent of the worker resolves" wording in witness_author_independence and related tests. Wire the new setting through stella-cli AgentEngineConfig, TOML settings, and apply_pipeline_tuning so all drivers can opt into the verifier independence requirement.	crates/stella-pipeline/src/pipeline.rs crates/stella-pipeline/src/pipeline/run_error.rs crates/stella-cli/src/settings.rs crates/stella-cli/src/settings/toml_config.rs crates/stella-cli/src/agent/engine.rs crates/stella-pipeline/src/pipeline/tests.rs crates/stella-pipeline/src/pipeline/tests/witness_isolation.rs crates/stella-pipeline/src/pipeline/tests/terminal_outcomes.rs
Record verifier independence as structured data on verdicts and propagate it through pipeline, protocol, replay, and docs.	Extend verify::Verdict and LadderSnapshot with an Option verifier_independent field plus helper with_verifier_independence, and ensure JSON/wire round-trips remain additive-compatible. Stamp verifier_independent at the verdict call seam in verifier_stage by comparing resolved worker and verifier model refs, and propagate it into the LadderSnapshot stored in VerdictEvidence. Update replay::verdict_provenance to narrate grader independence/self-grading, initialize the field where snapshots are constructed in tests, and regenerate TypeScript wire definitions and JSON schemas for AgentEvent/ServeFrame.	crates/stella-pipeline/src/verify.rs crates/stella-pipeline/src/pipeline/verifier_stage.rs crates/stella-pipeline/src/pipeline/verify_probes.rs crates/stella-pipeline/src/pipeline.rs crates/stella-protocol/src/ladder.rs crates/stella-protocol/src/event.rs crates/stella-protocol/tests/wire_contract.rs crates/stella-pipeline/src/replay.rs crates/stella-pipeline/src/replay/tests.rs crates/stella-pipeline/src/reward/tests.rs crates/stella-cli/src/trace.rs docs/wire/agentevent.d.ts docs/wire/serveframe.d.ts docs/wire/agentevent.schema.json docs/wire/serveframe.schema.json
Keep large files under the size ratchet by extracting focused test modules for late reconciliation and witness diff stripping, and add new verifier-independence behavior tests.	Extract late reconciliation tests from replay.rs into replay/late_reconciliation_tests.rs and wire them as a child module to avoid crossing the file-size limit ratchet. Extract witness strip tests from verify/tests.rs into verify/tests/witness_strip.rs and re-export via the tests module tree to keep internals reachable. Add pipeline/tests/verifier_independence.rs with end-to-end scenarios covering refusal before spend, opt-in semantics, acceptance with independent verifier, and correct setting of verifier_independent in ladder snapshots for both self-graded and independent verdicts; update file-size baseline and any affected test helpers to include the new field.	crates/stella-pipeline/src/replay.rs crates/stella-pipeline/src/replay/late_reconciliation_tests.rs crates/stella-pipeline/src/verify/tests.rs crates/stella-pipeline/src/verify/tests/witness_strip.rs crates/stella-pipeline/src/pipeline/tests/verifier_independence.rs scripts/file-size-baseline.txt

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#1795	Add an opt-in configuration gate that refuses a run before spend when the verdict verifier would resolve to the same model as the worker (or be unresolvable), achieving parity with the independent witness gate and wiring it through CLI/settings.	✅
#1795	Record verifier independence as a structured fact on verdict evidence (LadderSnapshot) for every model-graded verdict, indicating whether the grader was the worker’s own model or an independent one, and expose this in replay/provenance and wire schemas.	✅

Possibly linked issues

• Nothing enforces verifier != worker for the verdict call — one configured provider means the worker grades itself with a prose warning #1795: The PR adds require_independent_verifier and verifier_independent snapshot field, directly implementing the issue’s requested fix.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.