feat: autonomous delivery harness — event-driven worker/watcher split - #5
Draft
IcyWednesdays wants to merge 6 commits into
Draft
feat: autonomous delivery harness — event-driven worker/watcher split#5IcyWednesdays wants to merge 6 commits into
IcyWednesdays wants to merge 6 commits into
Conversation
Architectural spec for an event-driven CI/CD delivery harness built on Craft. Core ideas: - Pluggable state backend (file-based today, Linear-ready later) so the orchestrator's read/write of task state goes through a single interface. - Worker/watcher split: workers do the code work and exit; a separate long-lived bash watcher polls each PR and dispatches narrow remediation agents only when events fire (CI failure, review comment, merge conflict, merge, close). - Two-stage operator gating: approve a task, then merge a PR. The loop in between is autonomous. - Per-PR watcher pane in tmux/cmux with a live status snapshot (PR header, CI matrix, reviews, last action) above the polling log. - Multiplexer-agnostic: tmux and cmux providers behind a shared interface. The remaining plan documents (state backend, worker/watcher split, UI polish, comment dispatch) describe each subsystem in detail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces a backend-agnostic state interface so task state can move from local markdown files to Linear (or any other tracker) without touching the orchestrator or skills. - bin/lib/state.sh — dispatcher; sources the right backend based on STATE_BACKEND config (default: local). - bin/lib/state-local.sh — file-based backend wrapping the existing queue.sh helpers. Adds high-level operations: state_claim_task, state_mark_waiting, state_mark_done, state_mark_blocked, state_append_note, state_create_subtask, state_find_task_by_id, state_list_tasks, state_count_tasks, plus state_task_* accessor aliases over queue.sh's task_* low-level helpers. - bin/orchestrator.sh and bin/craft migrated to source state.sh. The orchestrator's timeout path uses state_mark_blocked. - templates/craft.conf — adds STATE_BACKEND=local default. - bin/craft doctor — reports the current state backend. - Comprehensive test coverage (test/test-state.sh): 58 assertions covering every operation plus an end-to-end round-trip. queue.sh remains untouched. Existing direct-queue.sh consumers (test-queue.sh, plugin hooks) continue to work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the in-worker PR polling loop with a separate long-lived watcher process. The worker exits as soon as it creates the draft PR; a deterministic bash watcher takes over post-PR, polls gh, and dispatches narrow remediation agents only when events fire. Key pieces: - bin/lib/watcher-events.sh — pure event-detection functions (watcher_extract_state, watcher_diff_events, watcher_dispatch_action) testable with JSON fixtures. - bin/watcher.sh — the polling loop. Reads the PR via gh, diffs state between iterations, persists state to .state/watchers/<pr>.state, invokes hooks/skills/state-transitions per event. - Multiplexer helpers (bin/lib/mux-tmux.sh, mux-cmux.sh, mux.sh): spawn_watcher_pane creates a per-PR window/surface where the watcher runs. - bin/orchestrator.sh: ACTIVE_WATCHERS tracking, ensure_watchers_for_waiting spawns a watcher for any task in waiting/ with a pr: field, reap_finished_watchers cleans up dead/exited watchers (incl. detecting panes that died unexpectedly). - /work-task skill rewritten: ends at state_mark_waiting, no polling. Worker prompt now exposes CRAFT_ROOT/PROJECT_DIR/QUEUE_DIR so it can source state.sh and call state_mark_waiting. - /fix-ci-failure skill: first remediation skill, dispatched by the watcher on ci_failed events. - 34 fixture-based tests (test/test-watcher-events.sh + 3 paired state/JSON fixtures). Watcher event taxonomy at this point: ci_failed → /fix-ci-failure (LLM dispatch) merged → state_mark_done (terminal) closed → state_mark_blocked (terminal) draft_to_ready → on_ready plugin hook others → log only (deferred) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the silent log-only watcher window with a horizontally-split pane (live status snapshot above, scrolling log below) and adds freshness signals to the orchestrator dashboard. - bin/render-watcher-status.sh — one-shot ANSI snapshot renderer. Reads the watcher's flat state file + cached gh JSON; prints PR header (number, title, state badge), CI check matrix, reviews block, and last-action timestamp. - bin/render-dashboard.sh — extracted from inline render_dashboard in orchestrator.sh. New affordances: per-watcher freshness via state file mtime (green <60s / yellow <5m / red STALE), CI status badge per watcher, PR titles, recent activity feed. - spawn_watcher_pane (tmux + cmux): splits the new window into a status snapshot above (re-rendered every 5s) and the log below. - bin/watcher.sh: caches raw gh pr view JSON to <pr>.json each poll, records last_action/last_action_at in the state file, suppresses event:first_poll log noise, dedupes consecutive gh-failure messages. - bin/orchestrator.sh: render_dashboard now wraps the script call with the runtime-only footer; check_milestone_completion uses .state/notified-milestones/ markers to fire the completion log once instead of every poll forever (pre-existing bug). - 17 fixture-based tests for the watcher renderer (test-render- watcher-status.sh) bringing the suite to 143 assertions. Pure bash + ANSI; no Node.js dependency. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the watcher's event-driven dispatch to handle PR comments and
non-approval reviews, with the LLM-dispatched fix agent running in a
detached multiplexer pane that's torn down cleanly when the agent
either finishes or pushes a new commit.
Watcher events:
- watcher_diff_events: split review handling — emit
new_review_received only when the review-count delta exceeds the
approved-count delta (i.e., not just an approval landed).
Pure approvals are filtered upstream so they don't trigger an LLM.
- watcher_dispatch_action: new_comment_received and new_review_received
both → /address-review-comment (LLM dispatch).
New skill:
- templates/.claude/commands/address-review-comment.md — reads
comments/reviews, decides actionability, makes one-fix-per-commit
changes, pushes, exits.
Smarter remediation lifecycle:
- bin/lib/mux-tmux.sh, bin/lib/mux-cmux.sh: new spawn_task_pane_detached
helpers that create a window/surface without stealing focus.
- bin/watcher.sh::dispatch_remediation rewritten:
· Spawns the agent in a new (unfocused) pane via spawn_task_pane_detached.
· Captures initial PR head SHA before dispatch.
· Loops on three completion signals: pane exit / new commit pushed
(SHA changed) / 5-minute timeout.
· Force-kills the pane unconditionally at the end, fixing the
Claude-Code-/exit-doesn't-terminate issue.
· Falls back to inline execution if the detached spawn fails.
Plus testing-discovered fixes that landed during Plan 4/5 validation
(folded in here since they're tightly coupled to dispatch behaviour):
- bin/lib/state.sh + state-local.sh: resolve STATE_DIR via CRAFT_ROOT
for zsh callers (BASH_SOURCE is unreliable under zsh; orchestrator
exports CRAFT_ROOT into the worker prompt).
- bin/render-watcher-status.sh: coalesce CI check name across
context → workflowName → name → "(unnamed)" so Buildkite/Actions
checks render correctly.
- bin/lib/mux-cmux.sh: TMUX_SESSION compat alias so the
orchestrator's session-name reference works under cmux too.
Test coverage extended to 149 assertions covering approval
suppression, both new dispatch entries, and existing happy paths.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updates the repo-level CLAUDE.md to describe the architecture this branch introduces — worker/watcher split, event-driven dispatch, state backend abstraction, UI surfaces, skill taxonomy — so future Claude sessions in this repo don't need to rediscover the design from the plan/spec docs. Includes: - Pointers to docs/superpowers/specs/ and docs/superpowers/plans/ for deep dives. - Process types (architect / orchestrator / worker / watcher / remediation) and their lifecycles. - Event-to-action dispatch table. - State backend interface signatures. - UI surface descriptions (dashboard, watcher pane, remediation pane). - Skill taxonomy (worker / remediation / deferred intervention). - Test suites (4 files, 149 assertions, run via make test). - Conventions including known macOS bash 3.2 / zsh quirks. - Guidance for extending watcher events, state operations, and skills. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces Craft's in-worker PR polling loop with a separate, event-driven harness. Workers now exit after creating a draft PR; a deterministic bash watcher takes over post-PR, polls
gh, and dispatches narrow remediation agents (e.g./fix-ci-failure,/address-review-comment) only when events fire. Adds a pluggable state backend so the harness can later swap to Linear (or any tracker) without changes to the orchestrator or skills.What's in the change
5 commits, each a coherent unit:
docs: autonomous delivery harness designfeat(state): pluggable state backend abstractionstate.shdispatcher +state-local.shfile backend with high-level operations (state_claim_task,state_mark_waiting,state_mark_done,state_mark_blocked,state_append_note,state_create_subtask, etc.); orchestrator +bin/craftmigrated;STATE_BACKEND=localconfigfeat(watcher): worker/watcher split + event-driven dispatchbin/lib/watcher-events.sh(pure event detection),bin/watcher.sh(polling loop),spawn_watcher_paneon tmux + cmux, orchestrator watcher tracking,/work-taskrewritten to exit after PR, first remediation skill/fix-ci-failurefeat(ui): split-pane watcher UI + dashboard freshnessbin/render-watcher-status.shsnapshot + tmux/cmux split (status above, log below, refreshes every 5s);bin/render-dashboard.shwith per-watcher freshness signals, CI badges, recent activity feed; milestone-completion log dedupefeat(watcher): comment dispatch + detached remediation panes/address-review-commentskill;new_comment_receivedandnew_review_received(filtered to non-approvals) → dispatch the skill;spawn_task_pane_detachedso remediation runs in its own unfocused pane; smart completion (pane-exit OR PR head SHA changed OR 5min timeout) with force-kill at the endArchitecture
Watcher event taxonomy:
ci_failed/fix-ci-failuremergedstate_mark_done, terminalclosedstate_mark_blocked, terminaldraft_to_readyon_readyplugin hooknew_comment_received/address-review-commentnew_review_received(non-approval only)/address-review-commentnew_review_approved,mergeable_conflicting,first_poll,ci_passedValidation
Tested live on real Linktree PRs (
developer-platform-libraries):/address-review-comment, which made fixes and pushed; watcher detected the SHA change and tore down the remediation pane cleanly.watcher_diff_events).Test plan
make testgreen: 149 assertions across 4 suites (test-queue.sh,test-state.sh,test-watcher-events.sh,test-render-watcher-status.sh)bash -nclean on all modified shell scripts/address-review-comment, merge tears down watcher cleanlyKnown limitations / deferred follow-ups
comment_countonly, so the agent's own replies could re-dispatch; folded into a future plan.templates/.claude/commands/aren't auto-copied into existing projects; requires either acraft synccommand or a fallback indispatch_remediation./resolve-merge-conflict,/investigate-pr-close) and operator-override skills (/pause,/kill,/restart,/watch) are designed but not built.WORKER_AUTO_PERMISSIONSconfig knob to gate--dangerously-skip-permissions— not yet added (workers run autonomously by default, which is the existing upstream behaviour)./exitdoesn't terminate Claude Code's process (only fixed for remediation panes via the force-kill in commit 5).🤖 Generated with Claude Code