fix(stella-pipeline,stella-cli): verification follow-ups round 2 — role tuning parity, policy reach, cache-stable witness prompt, scoped goal verifier - #1813
Merged
Conversation
added 3 commits
August 6, 2026 02:54
…gines; the tool policy reaches the witness author's reads The witness author rides the verifier's model, so it now takes agents.verifier's effort/reasoning/temperature/max_output_tokens/params too (prompt stays raw-call-scoped — reviewer prose must not steer a test author). And the candidate workspace hands the witness executor the same policy-wrapped read surface the worker gets, so a read_file switch governs both. Closes #1785 Closes #1784
…ides the byte-stable system message The hard-requirements block opened the volatile user message, re-billing the whole instruction set uncached on every author call, repair call, and tool round-trip — the one verification role running a multi-step tool loop paid full freight (#1434's failure shape, unapplied here). It now merges into WITNESS_SYSTEM_PROMPT (moved to witness.rs beside its prompt-builder peers, shrinking pipeline.rs), and witness_prompt carries only the per-call sections: probed runners, structure, recall, goal. The cache-minimum measurement half of #1786 stays open on the issue. Refs #1786
…prompt names The bare ReadOnlyTools wrap admitted every schema claiming read_only — ~25 tools including web_fetch/web_search (outbound HTTP from a role that reads worker-influenced content) and any MCP/custom tool self-declaring read-only. VerifierScopedTools narrows the stack to the prompt's six (read_file, grep, glob, explorations, ci_status, search_issues) before the read-only view applies, enforced at execution; a test pins the allowlist against VERIFIER_SYSTEM_PROMPT (now pub) so the two cannot drift. Sub-agents keep their existing ReadOnlyTools surface — research children legitimately use the web. Closes #1783
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Sorry @macanderson, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
Contributor
Reviewer's GuideThis PR tightens verification-role behavior across witness and goal-verifier flows: witness engines now honor verifier-specific tuning and tool policy, the witness author’s instructions are moved into a cache-stable system prompt, and the goal verifier’s tools are strictly scoped to the set named in its prompt, all backed by targeted tests. File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
…— glob joins its tool surface The prompt's repo listing truncates at 200 sorted paths, and an author that could not see a tests/ directory there had no legal move: read_file needs a path it can already name and create_witness_test requires the parent to exist. glob is names-only, root-confined by the tool itself (an escaping search path is refused), and its results pass the same credential exclusion as the read path. The system prompt now names the discovery pair. Closes #1792
macanderson
enabled auto-merge (squash)
August 6, 2026 10:06
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
| // pass the same credential exclusion the read path enforces. | ||
| "glob" => { | ||
| if let Some(raw_path) = input.get("path").and_then(serde_json::Value::as_str) | ||
| && normalized_candidate_path(raw_path).is_none() |
Contributor
There was a problem hiding this comment.
The witness author's glob path guard wrongly rejects the documented root values . and "", denying the one blind-discovery move the feature exists to enable.
Owner
Author
There was a problem hiding this comment.
Confirmed — traced rather than assumed. normalized_candidate_path(".") drops the CurDir component and returns None at (!parts.is_empty()).then(...); ("") returns None at the is_empty() guard. Both are documented root values: glob defaults its path to "." and its own comment says "./empty resolve to the root itself".
Root cause is that normalized_candidate_path's None conflates "escapes the root" with "is the root" — fine for read_file, wrong for a listing tool. names_candidate_root splits them, ruling out absolute/drive-qualified spellings first so / stays an escape.
Fix + witness (glob_accepts_the_root_spelled_out_as_well_as_omitted) pushed to wip/fix-1813-glob-root as 8fc4000a; details in the PR comment. Kept off this branch pending your call, since it is your PR.
added 2 commits
August 6, 2026 03:15
…ion prompt Verdict::reasoning is the model's whole reply with no length contract; on a FAIL it became the revision reason verbatim and rode into every subsequent turn. The airlock scrub already gates content — this adds the missing size ceiling (head-kept, char-boundary-safe, explicit truncation marker), applied at the one site verifier prose crosses to the worker. The structured-output and per-candidate-degradation halves of #1787 stay open on the issue. Refs #1787
…e taxonomy, runner probing, and the real stage list The strongest security property in the verification path — the three-tool enforced executor with credential exclusion, the single-artifact claim with replace-on-repair, and the graft + re-pin — was documented nowhere outside the source. The pipeline page now covers it, plus the degradable vs fail-closed witness outcomes a transcript actually shows (runner probes, the assertion-density screen, the one bounded repair), corrects the stage list (verdict, then reflection and context write-back), and notes the guidance call emitting under the verdict stage name for stream consumers. StageKind::Witness's wire doc no longer claims authoring runs before execution — stale since the demand-driven reorder — and docs/wire is regenerated to match. Closes #1796
Owner
Author
|
Two things on this PR — one is the open review thread, one is a gate failure nobody has flagged yet. 1. The
|
This was referenced Aug 6, 2026
macanderson
pushed a commit
that referenced
this pull request
Aug 6, 2026
…s witness stage 608a0aa (#1813) landed two clippy -D warnings failures in pipeline/witness_stage.rs, red on main and on every PR branched from it (the first masks the second because cargo bails at the first error): - clone_on_copy: `params.clone()` where GenerationParams became `Copy` in a parallel change — copy it out of the borrow instead. - field_reassign_with_default: the #1785 witness test built its worker config by mutating a `default()` — use struct-update syntax.
Owner
Author
|
The Vercel review finding on the witness |
This was referenced Aug 6, 2026
macanderson
added a commit
that referenced
this pull request
Aug 6, 2026
…lob` (#1843) ## What & why Refs #1792, #1813. The review bot flagged this on #1813 (`witness_tools.rs:200`) and it merged before the thread was answered, so the defect is on `main` now. The witness author's `glob` guard **refuses the tool's own default argument**. `crates/stella-tools/src/glob.rs` documents and implements the root two ways: ```rust "path": { "type": "string", "description": "Subdirectory to search (default: workspace root)" } ... let search_path = input.get("path").and_then(|v| v.as_str()).unwrap_or("."); // `.`/empty resolve to the root itself, so existing default-scoped calls keep working. ``` But `WitnessToolExecutor` gated `path` on `normalized_candidate_path`, which answers `None` for both: - `""` → the `is_empty()` guard - `"."` → the `CurDir` component is dropped, `parts` ends up empty, and `(!parts.is_empty()).then(...)` yields `None` So a model that spells the default out loud — which they routinely do — is told *"the path must stay within the candidate root"* about the root itself. That is the blind author's opening move, and the whole reason `glob` was offered to it in #1792. ## The root cause `normalized_candidate_path`'s `None` conflates two different answers: **"escapes the root"** and **"*is* the root"**. That is correct for `read_file`, which was what it was written for — a path naming no file is simply not readable. It is wrong for a listing tool, where the root is the most ordinary argument there is. `names_candidate_root` answers the second question separately. It rules out absolute and drive-qualified spellings **first**, so `/` — which would otherwise trim to the empty string and look like the root — stays an escape. ## The witness - [x] This PR includes a witness test (fails on `main`, passes here). `candidate_ws::witness_tools::tests::glob_accepts_the_root_spelled_out_as_well_as_omitted`, checked the artisanal way: ``` panicked at crates/stella-cli/src/candidate_ws/witness_tools.rs:655:17: `path: "."` names the root and must be allowed: Error { message: "`glob` is not available to the witness author: the path must stay within the candidate root" } test result: FAILED. 0 passed; 1 failed ``` It asserts both directions: `.`, `""`, `./` are allowed and actually search the root; `/`, `..`, `../..`, `/etc` are still refused. **Why the existing test did not catch it:** `glob_lets_the_author_discover_tests_without_leaking_credentials` only ever passes *no* `path` (exercising the tool's own default, which never reaches the guard) or an escaping one. The explicit-root case sat exactly in the gap between them. ## The gate - [x] `cargo fmt --check -p stella-cli` - [x] `cargo test -p stella-cli` — 1421 pass, 0 fail ## Nothing left behind - [x] There is nothing: everything I noticed is fixed in this PR. ## Summary by Sourcery Allow the witness `glob` tool to treat explicit root paths as valid while still denying paths that escape the candidate workspace. Bug Fixes: - Fix `glob` witness tool incorrectly rejecting explicit root paths such as "." or "" that should resolve to the workspace root. Enhancements: - Introduce a helper to detect when a path names the candidate root, distinguishing it from paths that escape the root. Tests: - Add a witness test verifying that `glob` accepts explicit root spellings and continues to refuse paths that leave the candidate root. Co-authored-by: Mac Anderson <macanderson.mail@gmail.com>
macanderson
added a commit
that referenced
this pull request
Aug 6, 2026
…rity count (#1845) ## What & why `main` fails `check-file-size` at `5650c889`, so **every open PR inherits the red**: ``` crates/stella-protocol/src/event.rs grew to 2964, over its ceiling of 2962 (+2) crates/stella-tui/src/deck_render.rs grew to 1531, over its ceiling of 1528 (+3) ``` **Parallel merge skew, not a real regression.** #1805 retightened the baseline to the then-current counts (retiring #1797's raises), and #1813 and #1828 were already in flight with CI green against the *old* ceilings. ## Why this is `make file-size-update` and not a split AGENTS.md allows a ceiling to move only when the growth is genuinely irreducible, and CLAUDE.md is explicit that raising one to dodge a real split is a defect. Both of these are the former: | File | +N | What grew | Why it cannot move | |---|---|---|---| | `event.rs` | +2 | #1813 expanding the doc comment on `StageKind::Witness` to say authoring runs *after* execution | A doc comment cannot be moved off its item — and this is a wire-contract type, so the doc **is** the contract | | `deck_render.rs` | +3 | #1828's `SessionPhase::Stopped => theme::TEXT_TERTIARY` arm, plus two lines saying why a deliberate stop is not painted red | A match arm cannot live in a sibling module; the match must be exhaustive | Neither is a split someone declined to do. #1828 in fact *did* split where one was possible — it moved the sessions projection into `command_deck/sessions_view.rs` precisely because `command_deck.rs` sat at its exact ceiling. ## The baseline gets tighter, not looser Regenerated rather than hand-edited, so the diff also carries what those same two PRs **shrank**: ``` command_deck.rs 4740 -> 4691 (-49, #1828's sessions_view.rs split) pipeline.rs 3642 -> 3580 (-62, #1813 moving WITNESS_SYSTEM_PROMPT to witness.rs) fleet_cmd.rs 1507 -> 1504 (-3) event.rs 2962 -> 2964 (+2) deck_render.rs 1528 -> 1531 (+3) ``` **Four ceilings down by 114 lines, two up by 5.** Net the ratchet is stricter than before this PR. ## The witness - [x] No witness test needed (gate/baseline reconciliation) — verified directly: ``` $ ./scripts/check-file-size.sh check-file-size: OK — 1078 Rust/Python/shell files, none over 1500 lines except 32 grandfathered (none grew). $ ./scripts/check-god-files.sh check-god-files: OK — 24 god file(s) across 8 crate(s), named identically in AGENTS.md and every crate README. ``` The god-file **set** is unchanged — only counts moved — so AGENTS.md's table and the crate READMEs need no edit, and `check-god-files` confirms that rather than my asserting it. ## Note on recurrence This is the fourth distinct god-file ceiling break on `main` today (#1761 `deck.rs`, then `driver.rs`/`registry.rs` in #1800, now these two). The first three were "PR merged with a red gate"; **this one is different** — both PRs were green when they ran, and the baseline moved underneath them. `enforce_admins` is now `true`, which closes the first mechanism but not this one: a retighten and an in-flight PR can still cross. Worth knowing before concluding the class is closed. ## Summary by Sourcery Update file-size baseline to reconcile ceilings with recent god-file splits and documentation additions so the size gate reflects current main. Bug Fixes: - Fix failing file-size gate on main by aligning baseline ceilings with current file lengths. Enhancements: - Tighten overall file-size ceilings by capturing recent net line reductions while permitting small, irreducible increases in a few files. Chores: - Regenerate file-size-baseline.txt to reflect the latest distribution of lines across tracked files. --------- Co-authored-by: Stella Test <test@stella.local>
macanderson
added a commit
that referenced
this pull request
Aug 6, 2026
…1813 (#1859) ## What this is An unbreak: 608a0aa (#1813) landed two `clippy -D warnings` failures in `crates/stella-pipeline/src/pipeline/witness_stage.rs`, so the fmt+clippy+test gate is red on main and on every PR branched from it. The first masks the second because cargo stops at the first error. - **clone_on_copy** (line 55): `apply_role_shaping` did `params.clone()` where `GenerationParams` became `Copy` in a parallel change — copy it out of the borrow instead. - **field_reassign_with_default** (line 648, tests): the #1785 witness test built its worker config by mutating a `default()` — struct-update syntax now. ## Verification Pure lint fixes, no behavior change, so no witness test — the witness is clippy itself: `cargo clippy -p stella-pipeline --all-targets -- -D warnings` fails on main at both sites and passes on this branch. `cargo test -p stella-pipeline shaping` (including #1785's `verifier_shaping_overlays_the_worker_engine_config`) passes. After this merges, open PRs (#1843, #1836, #1844) need `gh pr update-branch` to go green. ## Summary by Sourcery Fix clippy lint violations in stella-pipeline’s witness_stage to restore a clean build. Bug Fixes: - Avoid cloning a Copy GenerationParams value when applying role shaping overrides. - Construct test EngineConfig instances with struct update syntax instead of mutating a default value to satisfy clippy lints. Co-authored-by: Stella Test <test@stella.local>
macanderson
added a commit
that referenced
this pull request
Aug 6, 2026
…nd three parameter objects (#1880) ## What — #1809, Option A as triaged The candidate plane (`run` → `run_best_of_n` → `dispatch_isolated_candidates` → `run_candidate` → `verify_candidate` → `revise_turn` → `run_engine_turn`) threaded the same values positionally through every layer, and **nine** functions across `pipeline.rs` and `fanout_stage.rs` carried `#[allow(clippy::too_many_arguments)]` to keep it lawful (the issue counted seven; the sweep found two more in `fanout_stage.rs`). Each new stage input — #1701's `mutating_actions`, #1798's `opaque_actions` — meant widening half a dozen signatures and re-justifying allows. Three parameter objects, each grouping values that already always travel together (transport, not semantics — every field keeps its own documented meaning): - **`TaskFrame`** (new `pipeline/task_frame.rs`, per the `driver/settlement.rs` sibling-module pattern): goal, staged prefix, plan, assessment. Built once in `run` where its last field settles; `Copy`, so the fan-out closures each take their own. - **`Spend`** (in `pipeline/stage_budget.rs`, beside `FanOutBudget`): the budget guard + running total. Borrowed rather than owned so every mutation lands at `run`'s locals with no write-back on early returns — one missed return would be a silently vanished spend. The fan-out builds a per-candidate `Spend` over its claimed allowance, which is the shape that code already had. - **`ChangeSignals`** — no new type: the warrant's input struct replaces the three loose `u32` counters on `CandidateState`, so `run_engine_turn` accumulates directly into the type the warrant reads. The deleted `change_signals()` accessor existed to stop the counts being transposed en route (the #1701 recurrence); holding the struct makes that protection structural. `revise_turn` now takes `&mut CandidateState` — its only caller already held one and exploded seven fields out of it. ## Result - `rg too_many_arguments crates/stella-pipeline/src` → **zero allows** (one doc-comment mention). - Every function in the plane is at or under clippy's threshold; `cargo clippy -p stella-pipeline --all-targets` adds no warnings from this change. - `pipeline.rs` **shrinks by 88 lines** (3487 against its 3642 ceiling); the god-file and file-size guards pass for every touched file. ## Verification Pure refactor — no behavior change, so no witness test (per AGENTS.md). Evidence: all **566** existing `stella-pipeline` tests pass unchanged (544 lib + integration suites), `cargo fmt --check` clean, scoped `RUSTDOCFLAGS="-D warnings" cargo doc` clean. **Known-red main caveat:** clippy currently fails on main with two `witness_stage.rs` warnings from #1813's auto-merge, and the file-size gate on `event.rs`/`deck_render.rs` skew — peer unbreaks #1859 and #1845 cover those; this PR deliberately does not duplicate them, so its CI stays red on those two axes until they land. Exemplar for the parameter-object shape: `rustc`'s own `Session`/context threading and this repo's existing `WitnessAuthoring` bundle. Closes #1809 Refs #1798, #1808 ## Summary by Sourcery Introduce parameter objects for shared task context, budget, and change signals to simplify the candidate execution/verification pipeline and remove all too_many_arguments allowances. Bug Fixes: - Eliminate the risk of misordered change-signal counters by storing and passing them as a typed ChangeSignals struct rather than separate u32 fields. Enhancements: - Thread immutable task context through the candidate pipeline via a new TaskFrame struct instead of multiple positional parameters. - Bundle budget guard and running cost into a Spend helper struct to centralize turn-level spending and reporting. - Replace loose change counters on CandidateState with the existing ChangeSignals struct to align execution tallies with warrant inputs and prevent miswiring. Tests: - Rely on the existing stella-pipeline test suite to validate that the refactor preserves behavior without adding new tests. Co-authored-by: Stella Test <test@stella.local>
macanderson
added a commit
that referenced
this pull request
Aug 6, 2026
…the fallback, budget stops degrade, arming symptoms recorded (#1890) ## What this is Round 3 of the verification-role work (#1798, #1813 were rounds 1–2): three decision-gated issues from the audit backlog, decided with the reasoning recorded in code, each with a witness test. ## The decisions 1. **A verifier outage is not a refutation** (`Closes #1788`). `heuristic_fallback` now passes on an observed fail→pass flip, not only on green touched tests. The asymmetry it closes: `Unverifiable` abstains when the *evidence* is absent, but a missing *checker* drove a flip-verified candidate (diff over budget → ModelVerdict → provider down) to `VerificationFailed` — the checker's absence treated as the work's failure. With nothing deterministic positive the fallback still fails closed. This deliberately inverts the "even a flip doesn't rescue an unconfirmed suite" pin; the test now states why. 2. **The scaffolding's budget must not discard the work** (`Refs #1789` — the reclassification + de-panic half). A budget stop during witness authoring/repair was `rejected`, aborting a candidate whose worker change was already complete. It degrades now: the budget guard still gates every later paid call (no overspend is possible), and what degrading buys is the deterministically-resolvable endings — a warranted waiver, an abstention — that need no further spend. The stage's two `expect`s on workspaces became degradable aborts, per its own produce-vs-trust contract. The issue stays open for the end-to-end complete-without-further-spend scenario. 3. **A build-failure baseline is recorded, never refused** (`Closes #1790`). Resolved per the on-issue analysis: refusing `SymptomClass::BuildFailure` baselines (the issue's original framing) would reject the most common Rust witness shape — a missing-API test fails to compile on the old code *by design*. Instead the arming failure's class is recorded (`witness_baseline=build_failure` in the verifier's trusted evidence, plus a run warning), so a compile-armed flip is visible to the verdict and to anyone reading the evidence — the honest treatment for a shape that is legitimate for missing-API goals and identical to two-tree environment drift. 4. **The cache-minimum measurement is a stated fact** (`Closes #1786` — the split landed in #1813). The management-prompt module doc records the measured prefix sizes: no fixed instruction block clears Anthropic's 1024-token minimum alone (verdict ~520, witness author ~620); the raw calls cache only with an `agents.<role>.prompt` override padding the prefix, while the witness author's engine turn crosses the minimum within its first tool round-trip — which is exactly where #1813's split pays. ## Witness tests `heuristic_fallback_passes_only_on_confirmed_green_tests` (inverted pin + new no-evidence case), `a_build_failure_baseline_is_recorded_and_an_assertion_one_is_not`. Both fail on main. ## CI note Until #1859 (witness_stage clippy, merged with #1813's premature auto-merge) and #1845 (file-size baseline skew: `deck_render.rs` +3) land, the required job is red on those pre-existing steps — none are in hunks this PR touches. I'll update-branch after they merge. Coordination: my #1795 implementation was dropped from this round in favor of the more complete open #1867; my duplicate unbreak #1874 was closed in favor of #1859/#1845. ## Still tracked #1787 (structured verdict output, per-candidate degradation records), #1789 (the e2e half), #1793 (FlipHalt — gated on a loop-bench measurement), #1794 (semantic witness review — needs its design pass; recommendation on the issue is extending the mutation audit to flip-corroborated ModelVerdict passes). Closes #1788 Closes #1790 Closes #1786 Refs #1789 ## Summary by Sourcery Adjust verification pipeline behavior to treat verifier outages, budget stops, and build-failure witnesses as recorded, degradable conditions rather than hard rejections, and document cache minimum measurements for management prompts. Bug Fixes: - Allow heuristic fallback verdicts to pass when a confirmed fail→pass flip exists even if the verifier is unavailable. - Prevent witness authoring and repair stages from discarding completed work on budget stops or missing workspaces by degrading the run instead of rejecting or panicking. - Record build-failure baselines for witnesses in pipeline state and evidence summaries instead of refusing them, ensuring missing-API compile errors are treated as legitimate but weaker evidence. Enhancements: - Expose the authored witness baseline symptom class in candidate state and evidence output so verifiers can distinguish build failures from assertion failures. - Clarify management prompt documentation with measured token-prefix sizes and their implications for provider cache behavior. Tests: - Extend heuristic fallback tests to cover verifier outage behavior and no-positive-evidence failure cases. - Add a witness-stage unit test to assert that build-failure baselines are recorded while assertion-based baselines are not. Co-authored-by: Stella Test <test@stella.local>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Round 2 of the verification-role work (#1798 was round 1): four of the issues filed from that audit, each with its witness test.
The fixes
agents.verifierrequest shaping reaches the witness engines (Closes #1785). The witness author rides the verifier's model but ran with the worker's tuning, and no setting could change that.witness_engine_configoverlays the verifier row's effort/reasoning/temperature/max_output_tokens/params onto the authoring and repair engines via a pure, unit-tested merge (apply_role_shaping).RoleCallOverrides::promptdeliberately stays raw-call-scoped — operator prose written against the reviewer instructions must not steer a test author; the doc comments state the contract.The operator's tool policy reaches the witness author's reads (
Closes #1784). The candidate workspace handedWitnessToolExecutorthe raw registry, so"tools": {"read_file": "off"}governed every worker surface while the witness author kept reading. The executor now receives the samePolicyToolSetwrap the worker path gets; the witness test pins that a switched-offread_filedisappears from both schemas and dispatch.The witness author's fixed instruction block rides the byte-stable system message (
Refs #1786— the split half; the cache-minimum measurement stays open on the issue). The ~450-word hard-requirements block opened the volatile user message, re-billing the whole instruction set uncached on every author call, repair call, and tool round-trip — Management calls (triage/verdict/guidance) have no cacheable prefix — and cannot have one until the adapters carry cache-control #1434's exact failure shape, unapplied to the one verification role that runs a multi-step tool loop. It now merges intoWITNESS_SYSTEM_PROMPT(moved towitness.rsbeside its prompt-builder peers, shrinkingpipeline.rs);witness_promptcarries only the per-call sections. Tests split accordingly: the system constant carries the enforced contract, the user prompt carries goal/structure/recall/runners and must NOT repeat the fixed block.The goal verifier is offered only the six tools its prompt names (
Closes #1783).ReadOnlyToolsalone admitted ~25 read-only-claiming tools includingweb_fetch/web_search— outbound HTTP from a role that reads worker-influenced content is a prompt-injection egress channel — plus any MCP/custom tool self-declaringread_only: true.VerifierScopedToolsnarrows to the prompt's six before the read-only view applies, enforced at execution; a second test pins the allowlist againstVERIFIER_SYSTEM_PROMPT(nowpubfor exactly that guard) so prompt and surface cannot drift. Sub-agents keep their existing surface — research children legitimately use the web.Witness tests
verifier_shaping_overlays_the_worker_engine_config,the_tool_policy_reaches_the_witness_authors_reads,the_system_prompt_carries_the_hard_requirements(+ the reworked prompt-split assertions),the_goal_verifier_gets_only_the_tools_its_prompt_names,the_allowlist_matches_what_the_prompt_promises. Each fails on main.Local:
cargo test -p stella-pipeline/-p stella-cli(witness + verifier scopes) /-p stella-core goal, clippy-D warningson all three, rebased onto c8faa37 (post-#1805/#1808) — green.Fifth fix (added after opening)
Closes #1792). The prompt's repo listing truncates at 200 sorted paths, and an author that could not see atests/directory there had no legal move —read_fileneeds a path it can already name,create_witness_testrequires the parent to exist.globjoins the enforced surface: root-confined by the tool (escaping search paths refused, pinned by test), results filtered through the same credential exclusion as reads, and the system prompt names the discovery pair. Witness test:glob_lets_the_author_discover_tests_without_leaking_credentials.Sixth and seventh fixes (added after opening)
Forwarded verifier prose is bounded (
Refs #1787— the size half; structured output and per-candidate degradation stay open).Verdict::reasoningis the whole model reply and became the revision reason verbatim; it now passes a char-boundary-safe head truncation with an explicit marker at the one site it crosses to the worker, alongside the existing airlock scrub.The verification docs tell the truth (
Closes #1796). The witness author's enforced tool surface, the degradable-vs-fail-closed outcome taxonomy, runner probing and the density screen, the corrected stage list (verdict → reflect → context write-back), and the guidance-emits-verdict wire note are all documented on the pipeline page;StageKind::Witness's wire doc comment no longer claims authoring runs before execution, with docs/wire regenerated in the same commit.Still tracked from the audit
#1786 (measurement half), #1787, #1788, #1789, #1790, #1792, #1793, #1794, #1795, #1796.
Closes #1785
Closes #1784
Closes #1783
Refs #1786
Summary by Sourcery
Align verification-related behavior across witness and goal verifier roles, ensuring prompts, tool access, and engine tuning match their documented contracts and operator policies.
Bug Fixes:
Enhancements:
Tests:
Closes #1792
Refs #1787
Closes #1796