Skip to content

Fix runner loop-control, consent, and redaction defects#11

Merged
ksimback merged 1 commit into
mainfrom
fix/runner-correctness-security
Jul 4, 2026
Merged

Fix runner loop-control, consent, and redaction defects#11
ksimback merged 1 commit into
mainfrom
fix/runner-correctness-security

Conversation

@ksimback

@ksimback ksimback commented Jul 4, 2026

Copy link
Copy Markdown
Owner

Remediation batch 1 of 3 from the 2026-07-04 audit (see ROADMAP.md in a follow-up PR). All findings verified against main before fixing; each fix has a regression test.

Correctness

  • fixed_passes gates could never complete cleanly with max_revisions >= 2: the synthetic fixed_passes reviewer pass marker had a constant no-progress signature, so the stall detector (threshold 2) failed the run on the second pass. Synthetic markers no longer feed the detector.
  • Judges emitting nested JSON could never pass: the fence regex \{.*?\} truncated at the first }, failing json.loads and silently degrading to revise. Parsing now takes the whole fenced block, with a raw_decode fallback tolerating trailing prose.
  • A no-progress stop now stops the run — previously it abandoned the current delivery and let the outer loop generate a fresh one, defeating action: stop.
  • Resume actually resumes: iteration restored from state.json, existing artifacts aren't regenerated, the wall-clock budget accrues across resumed runs instead of renewing, and stale stall counts reset at run start.
  • Failures are recorded: a RunnerError marks state.json failed instead of leaving it stuck at running; malformed specs get a friendly missing required field error instead of a KeyError traceback.
  • loop_control.human_checkpoints (after_plan) is now honored — it was silently ignored.

Security

  • Consent fails closed. A cross-vendor (non-local) member with no privacy.egress entry used to receive artifacts with no consent prompt at all. Now any non-local member requires explicit first-send consent; an egress entry can pre-grant with consent: granted.
  • Context files honor redaction config. gather_context used a hardcoded default glob list, so configured privacy redactions never applied to context inlining.
  • Redaction globs match at any depth — bare .env now covers config/.env. Defaults always apply; configured globs extend rather than replace them.
  • Workspace/context paths must stay inside the loop directory (a hand-edited resolved spec could previously write state anywhere).

Robustness

  • Subprocess I/O pinned to UTF-8 (errors=replace) — the locale codepage (cp1252) corrupted or crashed Windows runs on any non-ASCII model output.
  • Interactive prompts raise a clean error when stdin is closed (EOFError traceback before).
  • The example wrapper now passes its own loop.resolved.jsonrunpy made the default spec path resolve to templates/ (nonexistent), so running the example directly always errored.

Tests

12/12 locally on Windows / Python 3.13; 5 new regression tests: fixed_passes clean completion, nested judge JSON, consent fail-closed, nested-.env redaction (context + judge prompt), workspace escape rejection.

🤖 Generated with Claude Code

Correctness:
- fixed_passes: the synthetic 'reviewer pass' marker no longer feeds the
  no-progress detector, which used to fail any clean reviewer-only gate
  with max_revisions >= 2 before its passes completed.
- Judge verdict parsing extracts the full fenced block (with a
  raw_decode fallback for trailing prose) instead of a non-greedy regex
  that truncated nested JSON and forced every structured verdict to
  revise.
- A no-progress stop now ends the whole run instead of only abandoning
  the current delivery attempt.
- Gate-boundary resume: iteration restored from state.json, existing
  artifacts not regenerated, wall-clock budget accrues across resumed
  runs, stale stall counts reset at run start, and an already-passed
  run is not silently re-run.
- RunnerError failures now mark state.json failed instead of leaving
  status 'running'; malformed resolved specs produce friendly
  'missing required field' errors instead of KeyError tracebacks.
- honor loop_control.human_checkpoints (after_plan).

Security:
- Consent fails closed: any non-local council member requires explicit
  first-send consent even with no privacy.egress entry (an entry may
  pre-grant with consent: granted).
- Context gathering honors configured privacy redactions (plus
  defaults) instead of a hardcoded glob list.
- Redaction globs match at any path depth, so config/.env is covered by
  '.env'; default redactions always apply and configured ones extend
  rather than replace them.
- Workspace and context paths must stay inside the loop directory.

Robustness:
- Subprocess I/O pinned to UTF-8 (locale codepage corrupted or crashed
  runs on Windows).
- Interactive prompts raise a clean error when stdin is closed instead
  of an EOFError traceback.
- Example wrapper passes its own loop.resolved.json explicitly (runpy
  made the default resolve to templates/).

Five regression tests cover the fixed_passes completion, nested judge
JSON, consent fail-closed, nested .env redaction, and workspace escape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ksimback ksimback merged commit edbc6e8 into main Jul 4, 2026
9 checks passed
@ksimback ksimback deleted the fix/runner-correctness-security branch July 4, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant