Skip to content

feat(call-agent): optional cwd and permission-mode for sandboxed runs - #3

Closed
joepetjr wants to merge 1 commit into
mainfrom
feat/call-agent-cwd-and-permission-mode
Closed

joepetjr wants to merge 1 commit into
mainfrom
feat/call-agent-cwd-and-permission-mode

Conversation

@joepetjr

@joepetjr joepetjr commented Aug 3, 2026 •

Copy link
Copy Markdown
Contributor

Why

A consumer built an eval runner that replays real recorded prompts against this package. Some of those prompts are action-shaped — "ship it", "commit and merge" — so replay is only safe if each call runs confined to a throwaway fixture directory under a restrictive permission mode.

It could not do that. call_agent took only (agent_name, prompt, schema, budget_usd), built a fixed argv, and never passed a working directory or a permission mode to the claude subprocess. With no way to confine a replayed "ship it", the consumer refused to run replay mode at all, and its nightly evals were blocked on this package. This unblocks them.

Summary

Two optional keyword-only arguments on call_agent, both defaulting to None:

  • cwd: str | Path | None — runs the subprocess in that directory. Validated to be an existing directory before anything is dispatched; an unusable path raises AgentError. AgentError is deliberate rather than a bare ValueError: inside fan_out it already converts to that unit's {"_error": ...} row, so one bad path is a visible coverage gap instead of a dead batch.
  • permission_mode: str | None — appended as --permission-mode <mode>.

Both are threaded through fan_out and through the campaign entry point deliver.run, on the same optional-and-defaulted contract. Inside a campaign they apply to the worker, advisor, and grader alike — otherwise an escalation would step outside the sandbox its worker ran in, which is exactly the hole the consumer was worried about.

agent_options is the small helper that builds those kwargs and drops whatever is unset; it is what makes the omitted path byte-identical, and it is re-exported for anyone writing their own call=.

Flag name is verified, not guessed

claude --help on the CLI in this environment:

  --permission-mode <mode>              Permission mode to use for the session
                                        (choices: "acceptEdits", "auto",
                                        "bypassPermissions", "manual",
                                        "dontAsk", "plan")

The runner does not whitelist those values. It passes the string through, matching how Tiers already treats model-tier names as caller-supplied vocabulary — so a mode the CLI adds later works without a release here.

Backward compatibility

Omitting both leaves behavior unchanged, and that is pinned by tests rather than asserted:

  • no --permission-mode in the argv;
  • no cwd keyword on the subprocess.run call at all — not even cwd=None. The pre-existing test whose stub is def fake_run(cmd, capture_output, text, timeout) would TypeError on an unconditional cwd=, so it enforces this by construction;
  • injected call= functions with the four-positional signature are invoked exactly as before — the existing fan_out and deliver tests all use such stubs and were not touched.

Docs land in the same change (README section + docstrings), per CONTRIBUTING.

One judgement call to flag: CONTRIBUTING says changelog entries go under ## Unreleased, but the brief called for a minor bump. I bumped 0.1.0 -> 0.2.0 in pyproject.toml and __version__ and filed the entry under ## 0.2.0 so the changelog heading and the declared version agree. Say the word and I will move it back under ## Unreleased and revert the bump.

Test plan

TDD: all eight new tests were written first and run before any implementation. Six went red (TypeError: call_agent() got an unexpected keyword argument 'permission_mode', etc.); the other two are the backward-compatibility pins, which pass on main by design and would only break if this change altered the omitted path. Then implemented to green (65 -> 73 tests). No test spawns a real claude process or touches the network — subprocess.run is monkeypatched and campaign calls are injected via call=, per the CONTRIBUTING rule.

Covered: default behavior unchanged with both omitted; cwd reaches the subprocess call; a nonexistent cwd and a path that is a file both raise before any subprocess starts; permission_mode appears in the built argv; both together; fan_out forwards both and passes nothing when unset; deliver.run confines worker + advisor + grader; deliver.run without the options still drives a four-argument call=.

$ python -m pytest -q
........................................................................ [ 98%]
.                                                                        [100%]
73 passed in 0.27s

ruff check output is byte-identical to main's baseline (same 9 pre-existing findings, same 3 files ruff format would touch) — this branch adds none and no line exceeds the configured 100 columns. Diff is 209 insertions / 13 deletions.

@joepetjr

joepetjr commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #5 — same change, cherry-picked onto current main.

The merge wrapper refused this PR on a genuine stale base: main moved ahead with the installer PR (#2), which also edits CHANGELOG.md and README.md, so CI had tested neither combination. Rebasing in place requires a force-push, which is blocked in this environment, so the commit was re-pushed on a fresh branch.

Two things changed while resolving, both toward this repo's stated convention:

  • the changelog entry moved under ## Unreleased alongside the installer entry, per CONTRIBUTING;
  • the 0.1.0 → 0.2.0 version bump is reverted, since the installer PR didn't bump either — this repo bumps at release time, not per PR.

73 tests still pass, and the consuming runner's gate was re-probed against the rebased branch: (True, 'pinnable via cwd + permission_mode').

Branch deleted to close.

@joepetjr joepetjr closed this Aug 3, 2026
@joepetjr
joepetjr deleted the feat/call-agent-cwd-and-permission-mode branch August 3, 2026 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant