fix(test): agent-proof pane identity via tmux pane option (#22) - #72
Merged
Conversation
Maintainer host runs kept flapping on the 4-agent `all` combo's IDENTITY checks
(varying: opencode; then claude+gemini+opencode). Root cause: with real
credentials the agents actually launch and redraw/clear their small 2x2 panes,
wiping the `[sandy:pane-agent]` stdout marker from scrollback before
`capture-pane` reads it. A scrollback marker the agent can erase is
fundamentally racy — the settle gate and per-combo retry reduced but couldn't
eliminate it.
Switch the identity mechanism to a tmux PANE OPTION, which the agent cannot
touch. Under SANDY_TEST_PANE_TAGS each pane's command now also runs
`tmux set-option -p @sandy_pane_agent '<agent>'` (the printf marker is kept for
logs + the §80 grep + a fallback). The option binds to the pane running each
agent's command — correct regardless of the 4-agent pane-index shuffle — and the
harness reads it via `list-panes -F '#{@sandy_pane_agent}'` alongside geometry,
falling back to the scrollback marker only for an image predating the option.
The settle gate now waits on the option; the per-combo retry stays.
- sandy: 4 marker slots set @sandy_pane_agent (mirrored to user-setup.sh.tmpl;
this touches the baked user-setup so BUILD_HASH forces a one-time rebuild).
- harness: read identity from the pane option, capture-pane fallback retained.
- run-tests.sh §80: +2 guards (sandy sets the option; harness reads it).
- CLAUDE.md / RELEASE_NOTES: describe the pane-option mechanism.
SANDY_TEST_PANE_TAGS stays an internal env-only test hook — normal launches
(var unset) are byte-identical. Harness-only + a test-gated sandy hook; not run
in CI (integration acceptance is maintainer-run).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Maintainer host runs of
test/acceptance-pane-topology.shkept flapping on the 4-agentallcombo's identity checks — and, crucially, the failing panes varied run-to-run (opencode; then claude+gemini+opencode). That's the definitive fingerprint of a capture race, not a real defect.Root cause
With real credentials, the agents actually launch and redraw/clear their small 2×2 panes, wiping the
[sandy:pane-agent]stdout marker from scrollback beforecapture-panereads it. A marker the agent can erase is fundamentally racy — the settle gate (#822fbc0) and per-combo retry (#71) reduced but couldn't eliminate it.Fix — a tmux pane option the agent can't touch
Under
SANDY_TEST_PANE_TAGS, each pane's command now also runstmux set-option -p @sandy_pane_agent '<agent>'. The harness reads identity from that option vialist-panes -F '#{@sandy_pane_agent}', alongside geometry, falling back to the scrollback marker only for an image predating it.Why this is race-free:
$TMUX_PANE) — correct regardless of the 4-agent pane-index shuffle.select-pane -T(OSC-2-title-clobberable), the agent cannot touch a user pane option.The settle gate now waits on the option; the per-combo retry stays as backstop.
Scope
sandy: 4 marker slots set@sandy_pane_agent(mirrored touser-setup.sh.tmpl). This touches the bakeduser-setup.sh, soBUILD_HASHforces a one-time image rebuild on the next--start(automatic — just a slower first harness run).run-tests.sh §80: +2 structural guards (sandy sets the option; harness reads it) — 18/18 standalone.SANDY_TEST_PANE_TAGSstays an internal env-only test hook — normal launches (var unset) are byte-identical.bash -n+ regen--checkclean. Not run in CI (integration acceptance is maintainer-run) — final confirmation is a maintainer re-run showing steady 66/0.