Skip to content

Silent custom-tool success is byte-identical output the harness cannot stamp — the #3176 stagnation hazard survives for unstampable tools (custom scripts, MCP), and #3187's detector-side fix was never filed #3303

Description

@macanderson

Problem

The #3176/#3188 hazard class — the engine's stagnation detector keys on repeated byte-identical tool output, so a tool whose success output is constant converts a legitimate work loop into a stagnation kill — is fixed only for tools whose output Stella authors. It survives structurally for the population whose output Stella merely relays:

1. Custom script tools. crates/stella-tools/src/custom.rs:624-628 (at e352cf486):

let stdout = String::from_utf8_lossy(&output.stdout);
if output.status.success() {
    ToolOutput::Ok {
        content: crate::exec::truncate_middle_capped(&stdout, MAX_OUTPUT_BYTES),
    }

The success output is the child's stdout verbatimtruncate_middle_capped returns s.to_string() unchanged under the 64 KiB cap (crates/stella-tools/src/exec.rs::truncate_middle_capped, first branch) — so a script that exits 0 printing nothing yields ToolOutput::Ok { content: "" } on every call. Distinct effects (different files written, different arguments) render byte-identical empty outputs unconditionally — no same-length coincidence needed, which makes this worse than the deleted write_file case. The detector's rung fires on exactly this: crates/stella-core/src/loop_detect.rs, "Consecutive calls to the SAME tool that all produced byte-identical output — whatever their arguments".

This is the shape Stella itself authors: crates/stella-tools/src/foundry_author.rs:177-186 (render_script) wraps a mined command in #!/bin/sh + set -eu + the command line, so any quietly-mutating mined shape (cp, mv, tee, sed -i, a build writing an artifact) becomes, after adoption, a registered tool whose every success is the empty string.

Nothing in run_custom can stamp a content digest the way PR #3187 did for edit_file — the content is the child's stdout by contract, and in the empty case there is no content to digest.

2. MCP tools. Third-party server outputs are the same relay shape one layer out; Stella cannot stamp them either. PR #3187's body named this population and deliberately left its defense untaken:

The issue's fix 2 (detector-side: exempt calls whose arguments differ from output-keyed stagnation) is deliberately not taken here … noted for the maintainer in case a detector-side follow-up is still wanted as defense-in-depth for third-party/MCP tools whose outputs this PR cannot stamp.

"Noted for the maintainer" is where it stopped: #3176 closed completed on fix 1 alone, and no successor issue exists (searched 2026-08-14; nearest are #2706 and #3292, neither is this).

Exposure today

At origin/main = e352cf486 there are no tracked custom tools (.stella/ holds only rules/ and skills/) and no user-global ~/.stella/tools/ on this machine (unverified under a redirected $STELLA_HOME). So no shipping configuration hits this today — the exposure is one manifest away, and the foundry's authorship path (#830) manufactures the vulnerable shape by design. save_state, the one stampable built-in instance, is tracked separately (#3297, PR #3299).

Candidate fixes — a maintainer's design call, not decided here

Witness

A fixture manifest + silent script (exit 0, no stdout): two calls with different inputs currently render byte-identical Ok { content: "" } and count toward rung-3 stagnation (assert via loop_detect over synthesized records, or a driver test). After the chosen fix, either the outputs differ (fix a) or the rung no longer counts them (fix b); identical-input repeats must still be caught either way.

Constraints

  • custom.rs's execution contract section is the developer API — a stamp changes it; update the module doc in the same PR.
  • crates/stella-core is I/O-free; any detector change is pure logic + property tests.
  • No god files involved (custom.rs, loop_detect.rs are not in scripts/file-size-baseline.txt's table).

Refs #3176, #3187, #3188 (closed moot-by-deletion — this is the surviving half of its sweep), #3297, #3292, #2706, #3051.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Polish — worth doing, not urgentarea:corestella-core — engine: step loop, budget, compaction, retryarea:toolsstella-tools — built-in tools incl. verify_donebugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions