Repository navigation
supervisor: Start the supervisor as an interactive session - #947
Conversation
The operator wants to talk to the supervisor while it investigates, with whatever it decides going through the executor and its caps, not back as a comment as session.yml's research run does (cgwalters-forge#884, steps 1, 2, 3 and 5). A `session` dispatch input (with a `pool` choice, as session.yml has) makes the agent job call the wrapper with `session: true` and session.json's idle timeout. Everything else is as for a prompt: supervisor.toml, the same handback, and the executor applying the returned `supervisor-actions` block once the session ends. A session counts as an explicit request (it sweeps and skips the cooldown) and goes in `supervisor-agent`, so health passes keep running beside it. Since the wrapper posts the endpoint on its item, which here is the requests issue, a small job also posts it on cgwalters-forge#728 in the same form, as github-actions[bot]. supervisor.toml gets hosted.toml's `[session]` operators and origins, which the wrapper needs for a session. The concurrency and health-gate test that bb8363a deleted is back, covering `session: true`, as a `just test-supervisor-workflow` recipe. Note that this is Node in the Justfile: ci's shrink job refuses .cjs growth, and there is precedent for Python test recipes there (356f7b4), but this is the first Node one, so please confirm that is acceptable. Step 4: I kept session.yml. Once the chat pane can start and attach to supervisor.yml sessions, I think it becomes redundant, because a supervisor session is the same profile with its requests executed. The one thing session.yml still offers is a read-only chat that cannot spend executor caps. I propose deleting it as a follow-up. Verified: `just test` passes, and the new test fails against the old workflow. `test-supervisor-config` was skipped because agent-board was not built. The shared lint passes (actionlint, runner labels, zizmor offline only, since there was no network for its online audits). The session-settings step was run against each pool and against a mismatched repository. No session was started. One review pass, no blockers. It left these open: - The chat pane lists only session.yml and reads only the newest 10 runs, which a busy supervisor.yml pushes a session out of. Its Start does not pass `session`. - The session-settings step trusts session.json and pools.json with plain jq, where `supervise session` validates them. - No test covers the `if` of the session step and the endpoint job. - The endpoint is posted even if the wrapper then refuses the run. - `pool` is ignored without `session`. - A mistral session lacks hosted.toml's context window. ## Caller settings - Repository: cgwalters-forge/review - File: `src/github/chat-config.json` - Key: `sources` - Value: add `{ "repo": "cgwalters-forge/tracker", "workflow": "supervisor.yml", "issue": 728, "publisher": "github-actions[bot]" }` Starting a session from the pane also needs it to dispatch with `session: true`, which is a code change there. Generated-by: https://github.com/cgwalters/#llms Generated-by: https://github.com/cgwalters/#llms
There was a problem hiding this comment.
VERDICT: APPROVE: no blocker; the session reuses the existing gates, caps and trusted settings files
Reviewed head: 36135bc278270b4a9b44879fde5a30a288321007 (merge-base 5203807).
Worst merged outcome and what bounds it. Before this change, only the operator's GitHub comments on the locked requests issue and write-access dispatch prompts could produce a supervisor-actions block that the executor applies with CGWALTERS_BOT_PAT. Now a prompt sent over the tailnet to a supervisor session can do the same. The session's prompts are limited to [session] operators and origins. Those are byte-identical to hosted.toml's, and config.test.cjs now asserts that. The executor is unchanged. It runs only at the pinned AGENT_BOARD_REV, in supervisor-execute and the agent-apply environment, and at that revision every write except the log is capped at zero unless config.json sets a cap. The session step and the endpoint job keep the default-branch gate on health, SUPERVISOR preview/apply and act == 'true'. A dispatcher picks only a pool name from a closed choice. The repository, issue and idle timeout come from the checked-out default-branch session.json, and the step fails closed unless .repository == $GITHUB_REPOSITORY. If that step fails, health fails, so agent and session-endpoint are skipped. The endpoint job gets only issues: write and github.token, and it writes to a repository already checked to be this one. Per docs/bring-up.md, nothing enforces the pool allowance, so overriding the trigger's pool with the session's pool bypasses no budget gate. No dependency manifests changed, and there is no CODEOWNERS or merge.toml.
Follow-ups
- important:
.github/agentic-job/supervisor.toml:24(origins) and.github/workflows/supervisor.yml:250-262. The originhttps://cgwalters-forge.github.iocovers every Pages site of the org, not just the review app. That includescgwalters-forge/cgwalters-forge.github.io, whichdispatch.tomlandfix.tomllet agents open PRs against. Under session.yml, a page there that ran script in the operator's browser while a session was live could at most steer a read-only run. Now it can steer a run whosesupervisor-actionsthe executor applies with the bot token. The endpoint is also posted publicly on #728. This is bounded by the executor's caps and by needing merged Pages content plus a live session, so it is not a blocker. Before raising any executor cap, consider serving the chat pane from an origin of its own, or adding a per-session secret that the pane gets out of band. - important: the session gets no session-specific rules. session.yml prepends
profiles/session.md, which makes the agent answer "ready" first, wait for prompts, write its output only when asked and rewrite it later. Here the agent gets the ordinarysupervise composetask, so its first turn runs a full pass and writes itssupervisor-actionscomment before the operator says anything. Directions given later in the chat take effect only if the agent happens to rewrite that output, and nothing tells it that tailnet prompts are operator instructions or that the result is handed back only when the session ends. As a result, the chat may not affect what the executor applies, which is the purpose of #884. Add a short supervisor-session preamble in compose (or via--rules) whensessionis set. - important:
.github/workflows/supervisor.yml:226-235. The session settings are read with plainjqand not validated the waysupervise sessionvalidates them:idleMinuteshas no 1–60 range check, and the pool fields are not checked for type. The files are trusted, so this is defence in depth. Callingsupervise session(or a validating subcommand) would keep the two workflows consistent. (The PR description already lists this.) - important:
.github/workflows/supervisor.yml:250-262.session-endpointposts the endpoint wheneverhealthsays session, even if the wrapper then refuses or fails the run. It also hardcodes the wrapper's tailnet hostname scheme,agentic-job-RUN_ID-ATTEMPT. If only the agent job is re-run, the attempt number changes but this job does not run again, so the posted address is stale. If the wrapper renames its host, the address silently breaks. Prefer an output from the wrapper, or post after the agent job reports the endpoint. - important: holding
supervisor-agentfor a whole session (idle timeout per turn, up to the wrapper's job limit) leaves room for only one pending run behind it. While a session lasts, a newer operator comment or the7 */2sweep replaces the earlier pending one. Those comments still reach the session throughwatch-item, but any comment that was only pending is lost as a run of its own. The header comment says this, but neither the README paragraph nor the test does. - nit:
Justfile:7-61. The test evaluates the GitHub expressions with JavaScript semantics (==,!,||), which matches GitHub here only because every compared value is a string or a boolean. Thesession/idle-minutes/PROMPTchecks are regex matches on text, and theif:of the session step and the endpoint job are not covered (the description says so). - nit:
.github/workflows/supervisor.yml:28-33.poolis ignored silently withoutsession. A dispatch withpool=openaiand only a prompt runs on the trigger's pool with no warning. - nit: a
mistralsession usessupervisor.toml, which lackshosted.toml's[agent.context-windows]entry forzai-glm-5-3, so it gets 32k and the frequent compaction described in agentic-job#821. Copy the table, or extend the config test to compare it too.
|
Merge queue: review approved and checks green. Generated-by: https://github.com/cgwalters/#llms |
The operator wants to talk to the supervisor while it investigates, with whatever it decides going through the executor and its caps, not back as a comment as session.yml's research run does (#884, steps 1, 2, 3 and 5).
A
sessiondispatch input (with apoolchoice, as session.yml has) makes the agent job call the wrapper withsession: trueand session.json's idle timeout. Everything else is as for a prompt: supervisor.toml, the same handback, and the executor applying the returnedsupervisor-actionsblock once the session ends. A session counts as an explicit request (it sweeps and skips the cooldown) and goes insupervisor-agent, so health passes keep running beside it. Since the wrapper posts the endpoint on its item, which here is the requests issue, a small job also posts it on #728 in the same form, as github-actions[bot]. supervisor.toml gets hosted.toml's[session]operators and origins, which the wrapper needs for a session.The concurrency and health-gate test that bb8363a deleted is back, covering
session: true, as ajust test-supervisor-workflowrecipe. Note that this is Node in the Justfile: ci's shrink job refuses .cjs growth, and there is precedent for Python test recipes there (356f7b4), but this is the first Node one, so please confirm that is acceptable.Step 4: I kept session.yml. Once the chat pane can start and attach to supervisor.yml sessions, I think it becomes redundant, because a supervisor session is the same profile with its requests executed. The one thing session.yml still offers is a read-only chat that cannot spend executor caps. I propose deleting it as a follow-up.
Verified:
just testpasses, and the new test fails against the old workflow.test-supervisor-configwas skipped because agent-board was not built. The shared lint passes (actionlint, runner labels, zizmor offline only, since there was no network for its online audits). The session-settings step was run against each pool and against a mismatched repository. No session was started.One review pass, no blockers. It left these open:
session.supervise sessionvalidates them.ifof the session step and the endpoint job.poolis ignored withoutsession.Caller settings
src/github/chat-config.jsonsources{ "repo": "cgwalters-forge/tracker", "workflow": "supervisor.yml", "issue": 728, "publisher": "github-actions[bot]" }Starting a session from the pane also needs it to dispatch with
session: true, which is a code change there.Generated-by: https://github.com/cgwalters/#llms
Closes #884