feat(console): unread nudge on the read path (pull/list) + final-check convention#253
feat(console): unread nudge on the read path (pull/list) + final-check convention#253ivanmkc wants to merge 11 commits into
Conversation
…der, list summary) The nudge fired only on writes (push/patch/scoped status) — a human message arriving after the agent's LAST write was never surfaced. Close the gap on both halves of the read loop: - server: scoped GET /state (pull) now sets the x-termchart-inbox-unread header, same once-per-new-message semantics as the write path - cli pull: prints the content nudge off that header - cli list: prints a workspace-wide unread summary via GET /inbox?all=1 — the one read an agent still does when it's done writing, and the only place cross-scope messages can surface; re-announces until read (an explicit catalog read should show what's unread); stderr-only so --json stdout stays clean - skill: final-check convention — run `termchart list` as the last termchart action; plugin 0.16.4, marketplace entry synced (was stale 0.15.0) - eval: TC-INBOX-MIDTASK — seed_inbox.py gains @await scopes posted by a background watcher only after the agent's board appears (a human typing mid-task); run.sh wires the watcher up and kills it after the runner Verified live: pull nudges after a post-push message; list re-announces until read and shows cross-scope unread; quiet after a real read; --json stdout unpolluted. Touched tests 33/33.
…ecklist tick - TC-INBOX-DECIDE: agent posts two chips, blocks on inbox --wait; console_human.py (the simulated human, launched by run.sh) clicks 'green' via GET /suggest -> POST /inbox action once the chips are up. The choice reaches the agent only through the console. Deterministic gate: button color prop is the clicked id. - TC-INBOX-TICK: human ticks an editable Checklist out-of-band (POST /interact — produces NO inbox event by design); the agent must observe it by polling pull and reply naming the item. Gates: tick landed in the spec + reply names it. - pull-grep objectives now grep raw `pull` output (content in --json is an escaped string, so quote-anchored patterns silently miss — caught in dry-run). Dry-run verified: watcher clicks + ticks; inbox --wait returns the action; all objective commands pass against simulated-agent state.
…rop name Run 1 near-miss: the flow worked end-to-end (chips, --wait click, green button per judge, reply) but the agent expressed the color without a literal "color" prop. Gate on content containing the clicked id and NOT the un-clicked one (blue is also Mantine's default primary, so a click-blind agent still fails).
The inbox cases share one viewer/workspace; an agent following the list unread-summary in one case can consume another case's seed (observed: DECIDE read UNPROMPTED's seed before that case ran -> false FAIL). Singleton case-sets + a per-set clear/seed/watch/run cycle make that state-sharing unrepresentable.
…naming requirement Run-1 miss was real signal: the agent parsed pull --json (content is a double-encoded string), extracted an empty item id, and replied 'You checked: '. Keep the strict objective; stop steering the agent into the harder surface.
The scenario said the item 'shows "checked": true' — spaced — but interact
re-stringifies the spec compact ("checked":true), so agents that grepped the
prompt's literal form polled for 2 minutes against a pattern that cannot match.
Say compact explicitly; the parse robustness burden stays on the agent.
K=3 × 2 agents × 5 cases — repeat-run grid (isolated mode)
Core console flows are stable: 24/24 case-runs green across PROMPTED, UNPROMPTED, MIDTASK, and DECIDE — every repeat, both agents. In particular:
TICK (checklist observation) exposed two real papercuts, both diagnosed from transcripts and fixed in-flight (each run re-reads the cases, and run 3 passed both agents on the final config):
Harness hardening shipped along the way: CI green. Ready for review. |
…low coverage Mechanical battery (9/9): delivery/order, ack-advance, SIGINT<3s, --json lines, two concurrent followers, backoff growth on server death, RESTART recovery, auto-provisioned-wsid semantics. It caught a real bug: after a viewer restart (or clear) the server seq regresses and a live follower's stale cursor filtered the first post-restart message out FOREVER. follow() now detects the regression (cursor < ours, empty events), logs 'server ring reset', rewinds to 0, and re-polls — unit-tested via the maxPolls hook (sinces 0,7,0). Two agent cases: TC-INBOX-FOLLOW (background follow catches a mid-work @await seed while multi-step drawing) and TC-INBOX-FOLLOWDECIDE (DECIDE forced through --follow, no --wait — the agent must bound the unbounded stream itself). The pair measures whether --follow can stand in for --wait under real agents.
FOLLOWDECIDE r1: Claude guessed --id/--label/--type chips, got bare 'Unknown flag' errors, and burned its whole follow window on the guess-retry loop.
Heavy
|
| Case | Claude | Gemini |
|---|---|---|
| FOLLOW (background listener catches a mid-work message) | 3/3 | 2/3 † |
FOLLOWDECIDE (--follow standing in for --wait) |
1/3 | 3/3 |
(reference: DECIDE with --wait, from the earlier grid) |
3/3 | 3/3 |
† the one Gemini FOLLOW miss passed both deterministic gates (board carried the console content + reply posted — the follow mechanics worked); only the LLM-rubric leg failed (partial banner application or judge blip).
Answer to "can we remove --wait in lieu of --follow": no — keep both. The data says they're different execution models, not redundant flags:
--waitis a bounded single call returning data in-band — it fits a turn-based tool loop natively, and it makes unrelated errors cheap: in DECIDE, agents fumbledsuggest's flags too, recovered in a retry loop, and went 6/6.--follownever exits, so every agent had to self-orchestrate (background + file +timeout). Set up once, first it works great (FOLLOW 5/6, 6/6 on deterministic gates). But composed with any other friction it fails compoundingly: Claude's FOLLOWDECIDE losses weresuggestflag-guessing burning entire follow timeout windows, with the click landing in a file it wasn't watching. 1/3 vs--wait's 3/3 on the same task.- Removing
--waitsaves almost nothing anyway — it's a query param + timeout branch on the same endpoint follow uses.
Drive-by product fix: suggest's unknown-flag error now names the real chip forms (--items '<json>' / repeated --item id:label) — Claude guessed --id/--label/--type and a bare Unknown flag sent it into a guess-retry loop. Also refreshed inbox-watch's mechanism-1 example to the current content-inline nudge.
What
Closes the biggest gap left after #252: the nudge only fired on writes. If the human typed after the agent's last push, the message was never surfaced — the eval passed only because its seed existed before the push.
GET /state(pull) now setsx-termchart-inbox-unread(same once-per-new-message semantics as the write path).pull: prints the content nudge from that header.list: workspace-wide unread summary viaGET /inbox?all=1— the one read agents still do when done writing, and the only surface where cross-scope messages can appear. Re-announces until read (an explicit catalog read should show what's unread); stderr-only,--jsonstdout stays clean.termchart listas your last termchart action. Plugin 0.16.4; stale marketplace entry (0.15.0) synced.TC-INBOX-MIDTASKcase —seed_inbox.pygains@awaitscopes posted by a background watcher only once the agent's board appears, modelling a human typing mid-task. The write-path nudge can't fire; thelistsummary must catch it.Verification
Live (local viewer): push → human message after →
pullnudges with content;listre-announces until read + shows cross-scope unread; quiet after a realtermchart inboxread;--jsonstdout still valid JSON. Touched unit tests 33/33.Agent eval: K=3 × {Claude, Gemini} × 3 cases (PROMPTED / UNPROMPTED / MIDTASK) running now — results will be posted as a comment before undrafting.
Known race (documented in the case): if the agent's
listlands before the watcher posts the seed (~0.5s poll), MIDTASK can miss — acceptable for an eval of a genuinely asynchronous channel.