Skip to content

supervisor: Start the supervisor as an interactive session - #947

Merged
cgwalters-bot merged 1 commit into
cgwalters-forge:mainfrom
cgwalters-bot:dispatch/implement/agent-run-40268959319
Oct 26, 2026
Merged

cgwalters-bot merged 1 commit into
cgwalters-forge:mainfrom
cgwalters-bot:dispatch/implement/agent-run-40268959319

Conversation

@cgwalters-bot

Copy link
Copy Markdown
Collaborator

The operator wants to talk to the supervisor while it investigates, with whatever it decides going through the executor and its caps, not back as a comment as session.yml's research run does (#884, steps 1, 2, 3 and 5).

A session dispatch input (with a pool choice, as session.yml has) makes the agent job call the wrapper with session: true and session.json's idle timeout. Everything else is as for a prompt: supervisor.toml, the same handback, and the executor applying the returned supervisor-actions block once the session ends. A session counts as an explicit request (it sweeps and skips the cooldown) and goes in supervisor-agent, so health passes keep running beside it. Since the wrapper posts the endpoint on its item, which here is the requests issue, a small job also posts it on #728 in the same form, as github-actions[bot]. supervisor.toml gets hosted.toml's [session] operators and origins, which the wrapper needs for a session.

The concurrency and health-gate test that bb8363a deleted is back, covering session: true, as a just test-supervisor-workflow recipe. Note that this is Node in the Justfile: ci's shrink job refuses .cjs growth, and there is precedent for Python test recipes there (356f7b4), but this is the first Node one, so please confirm that is acceptable.

Step 4: I kept session.yml. Once the chat pane can start and attach to supervisor.yml sessions, I think it becomes redundant, because a supervisor session is the same profile with its requests executed. The one thing session.yml still offers is a read-only chat that cannot spend executor caps. I propose deleting it as a follow-up.

Verified: just test passes, and the new test fails against the old workflow. test-supervisor-config was skipped because agent-board was not built. The shared lint passes (actionlint, runner labels, zizmor offline only, since there was no network for its online audits). The session-settings step was run against each pool and against a mismatched repository. No session was started.

One review pass, no blockers. It left these open:

  • The chat pane lists only session.yml and reads only the newest 10 runs, which a busy supervisor.yml pushes a session out of. Its Start does not pass session.
  • The session-settings step trusts session.json and pools.json with plain jq, where supervise session validates them.
  • No test covers the if of the session step and the endpoint job.
  • The endpoint is posted even if the wrapper then refuses the run.
  • pool is ignored without session.
  • A mistral session lacks hosted.toml's context window.

Caller settings

  • Repository: cgwalters-forge/review
  • File: src/github/chat-config.json
  • Key: sources
  • Value: add { "repo": "cgwalters-forge/tracker", "workflow": "supervisor.yml", "issue": 728, "publisher": "github-actions[bot]" }

Starting a session from the pane also needs it to dispatch with session: true, which is a code change there.

Generated-by: https://github.com/cgwalters/#llms

Closes #884

Generated by dispatch · ◷

The operator wants to talk to the supervisor while it investigates, with whatever it decides going through the executor and its caps, not back as a comment as session.yml's research run does (cgwalters-forge#884, steps 1, 2, 3 and 5).

A `session` dispatch input (with a `pool` choice, as session.yml has) makes the agent job call the wrapper with `session: true` and session.json's idle timeout. Everything else is as for a prompt: supervisor.toml, the same handback, and the executor applying the returned `supervisor-actions` block once the session ends. A session counts as an explicit request (it sweeps and skips the cooldown) and goes in `supervisor-agent`, so health passes keep running beside it. Since the wrapper posts the endpoint on its item, which here is the requests issue, a small job also posts it on cgwalters-forge#728 in the same form, as github-actions[bot]. supervisor.toml gets hosted.toml's `[session]` operators and origins, which the wrapper needs for a session.

The concurrency and health-gate test that bb8363a deleted is back, covering `session: true`, as a `just test-supervisor-workflow` recipe. Note that this is Node in the Justfile: ci's shrink job refuses .cjs growth, and there is precedent for Python test recipes there (356f7b4), but this is the first Node one, so please confirm that is acceptable.

Step 4: I kept session.yml. Once the chat pane can start and attach to supervisor.yml sessions, I think it becomes redundant, because a supervisor session is the same profile with its requests executed. The one thing session.yml still offers is a read-only chat that cannot spend executor caps. I propose deleting it as a follow-up.

Verified: `just test` passes, and the new test fails against the old workflow. `test-supervisor-config` was skipped because agent-board was not built. The shared lint passes (actionlint, runner labels, zizmor offline only, since there was no network for its online audits). The session-settings step was run against each pool and against a mismatched repository. No session was started.

One review pass, no blockers. It left these open:
- The chat pane lists only session.yml and reads only the newest 10 runs, which a busy supervisor.yml pushes a session out of. Its Start does not pass `session`.
- The session-settings step trusts session.json and pools.json with plain jq, where `supervise session` validates them.
- No test covers the `if` of the session step and the endpoint job.
- The endpoint is posted even if the wrapper then refuses the run.
- `pool` is ignored without `session`.
- A mistral session lacks hosted.toml's context window.

## Caller settings

- Repository: cgwalters-forge/review
- File: `src/github/chat-config.json`
- Key: `sources`
- Value: add `{ "repo": "cgwalters-forge/tracker", "workflow": "supervisor.yml", "issue": 728, "publisher": "github-actions[bot]" }`

Starting a session from the pane also needs it to dispatch with `session: true`, which is a code change there.

Generated-by: https://github.com/cgwalters/#llms

Generated-by: https://github.com/cgwalters/#llms

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: APPROVE: no blocker; the session reuses the existing gates, caps and trusted settings files

Reviewed head: 36135bc278270b4a9b44879fde5a30a288321007 (merge-base 5203807).

Worst merged outcome and what bounds it. Before this change, only the operator's GitHub comments on the locked requests issue and write-access dispatch prompts could produce a supervisor-actions block that the executor applies with CGWALTERS_BOT_PAT. Now a prompt sent over the tailnet to a supervisor session can do the same. The session's prompts are limited to [session] operators and origins. Those are byte-identical to hosted.toml's, and config.test.cjs now asserts that. The executor is unchanged. It runs only at the pinned AGENT_BOARD_REV, in supervisor-execute and the agent-apply environment, and at that revision every write except the log is capped at zero unless config.json sets a cap. The session step and the endpoint job keep the default-branch gate on health, SUPERVISOR preview/apply and act == 'true'. A dispatcher picks only a pool name from a closed choice. The repository, issue and idle timeout come from the checked-out default-branch session.json, and the step fails closed unless .repository == $GITHUB_REPOSITORY. If that step fails, health fails, so agent and session-endpoint are skipped. The endpoint job gets only issues: write and github.token, and it writes to a repository already checked to be this one. Per docs/bring-up.md, nothing enforces the pool allowance, so overriding the trigger's pool with the session's pool bypasses no budget gate. No dependency manifests changed, and there is no CODEOWNERS or merge.toml.

Follow-ups

  1. important: .github/agentic-job/supervisor.toml:24 (origins) and .github/workflows/supervisor.yml:250-262. The origin https://cgwalters-forge.github.io covers every Pages site of the org, not just the review app. That includes cgwalters-forge/cgwalters-forge.github.io, which dispatch.toml and fix.toml let agents open PRs against. Under session.yml, a page there that ran script in the operator's browser while a session was live could at most steer a read-only run. Now it can steer a run whose supervisor-actions the executor applies with the bot token. The endpoint is also posted publicly on #728. This is bounded by the executor's caps and by needing merged Pages content plus a live session, so it is not a blocker. Before raising any executor cap, consider serving the chat pane from an origin of its own, or adding a per-session secret that the pane gets out of band.
  2. important: the session gets no session-specific rules. session.yml prepends profiles/session.md, which makes the agent answer "ready" first, wait for prompts, write its output only when asked and rewrite it later. Here the agent gets the ordinary supervise compose task, so its first turn runs a full pass and writes its supervisor-actions comment before the operator says anything. Directions given later in the chat take effect only if the agent happens to rewrite that output, and nothing tells it that tailnet prompts are operator instructions or that the result is handed back only when the session ends. As a result, the chat may not affect what the executor applies, which is the purpose of #884. Add a short supervisor-session preamble in compose (or via --rules) when session is set.
  3. important: .github/workflows/supervisor.yml:226-235. The session settings are read with plain jq and not validated the way supervise session validates them: idleMinutes has no 1–60 range check, and the pool fields are not checked for type. The files are trusted, so this is defence in depth. Calling supervise session (or a validating subcommand) would keep the two workflows consistent. (The PR description already lists this.)
  4. important: .github/workflows/supervisor.yml:250-262. session-endpoint posts the endpoint whenever health says session, even if the wrapper then refuses or fails the run. It also hardcodes the wrapper's tailnet hostname scheme, agentic-job-RUN_ID-ATTEMPT. If only the agent job is re-run, the attempt number changes but this job does not run again, so the posted address is stale. If the wrapper renames its host, the address silently breaks. Prefer an output from the wrapper, or post after the agent job reports the endpoint.
  5. important: holding supervisor-agent for a whole session (idle timeout per turn, up to the wrapper's job limit) leaves room for only one pending run behind it. While a session lasts, a newer operator comment or the 7 */2 sweep replaces the earlier pending one. Those comments still reach the session through watch-item, but any comment that was only pending is lost as a run of its own. The header comment says this, but neither the README paragraph nor the test does.
  6. nit: Justfile:7-61. The test evaluates the GitHub expressions with JavaScript semantics (==, !, ||), which matches GitHub here only because every compared value is a string or a boolean. The session/idle-minutes/PROMPT checks are regex matches on text, and the if: of the session step and the endpoint job are not covered (the description says so).
  7. nit: .github/workflows/supervisor.yml:28-33. pool is ignored silently without session. A dispatch with pool=openai and only a prompt runs on the trigger's pool with no warning.
  8. nit: a mistral session uses supervisor.toml, which lacks hosted.toml's [agent.context-windows] entry for zai-glm-5-3, so it gets 32k and the frequent compaction described in agentic-job#821. Copy the table, or extend the config test to compare it too.

Generated by review for #947 at 36135bc

@cgwalters-bot

Copy link
Copy Markdown
Collaborator Author

Merge queue: review approved and checks green.

Generated-by: https://github.com/cgwalters/#llms

@cgwalters-bot
cgwalters-bot added this pull request to the merge queue Oct 26, 2026
Merged via the queue into cgwalters-forge:main with commit b106f9b Oct 26, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

supervisor: Start the supervisor as an interactive session from workflow dispatch

1 participant