Skip to content

feat: autonomous delivery harness — event-driven worker/watcher split - #5

Draft
IcyWednesdays wants to merge 6 commits into
mainfrom
mo/autonomous-delivery-harness
Draft

feat: autonomous delivery harness — event-driven worker/watcher split#5
IcyWednesdays wants to merge 6 commits into
mainfrom
mo/autonomous-delivery-harness

Conversation

@IcyWednesdays

Copy link
Copy Markdown
Collaborator

Summary

Replaces Craft's in-worker PR polling loop with a separate, event-driven harness. Workers now exit after creating a draft PR; a deterministic bash watcher takes over post-PR, polls gh, and dispatches narrow remediation agents (e.g. /fix-ci-failure, /address-review-comment) only when events fire. Adds a pluggable state backend so the harness can later swap to Linear (or any tracker) without changes to the orchestrator or skills.

What's in the change

5 commits, each a coherent unit:

# Commit What it adds
1 docs: autonomous delivery harness design Architectural spec covering all subsystems
2 feat(state): pluggable state backend abstraction state.sh dispatcher + state-local.sh file backend with high-level operations (state_claim_task, state_mark_waiting, state_mark_done, state_mark_blocked, state_append_note, state_create_subtask, etc.); orchestrator + bin/craft migrated; STATE_BACKEND=local config
3 feat(watcher): worker/watcher split + event-driven dispatch bin/lib/watcher-events.sh (pure event detection), bin/watcher.sh (polling loop), spawn_watcher_pane on tmux + cmux, orchestrator watcher tracking, /work-task rewritten to exit after PR, first remediation skill /fix-ci-failure
4 feat(ui): split-pane watcher UI + dashboard freshness bin/render-watcher-status.sh snapshot + tmux/cmux split (status above, log below, refreshes every 5s); bin/render-dashboard.sh with per-watcher freshness signals, CI badges, recent activity feed; milestone-completion log dedupe
5 feat(watcher): comment dispatch + detached remediation panes /address-review-comment skill; new_comment_received and new_review_received (filtered to non-approvals) → dispatch the skill; spawn_task_pane_detached so remediation runs in its own unfocused pane; smart completion (pane-exit OR PR head SHA changed OR 5min timeout) with force-kill at the end

Architecture

Architect (Claude Code)            User-facing planning agent
       ↓
Orchestrator (bash daemon)         Polls queue, spawns workers/watchers
       ↓
Worker (short-lived agent)  →  exits after creating draft PR
       │ handoff via state_mark_waiting
       ↓
Watcher (long-lived bash)          Polls gh, detects events
       ↓ on event
Remediation agent (transient)      Narrow skill: /fix-ci-failure, etc.

Watcher event taxonomy:

Event Action
ci_failed dispatch /fix-ci-failure
merged state_mark_done, terminal
closed state_mark_blocked, terminal
draft_to_ready fire on_ready plugin hook
new_comment_received dispatch /address-review-comment
new_review_received (non-approval only) dispatch /address-review-comment
new_review_approved, mergeable_conflicting, first_poll, ci_passed log only (deferred)

Validation

Tested live on real Linktree PRs (developer-platform-libraries):

  • Worker → PR creation → watcher spawn → polling → event detection → remediation dispatch → terminal exit on merge: end-to-end working.
  • Comment-driven dispatch: posted comments triggered /address-review-comment, which made fixes and pushed; watcher detected the SHA change and tore down the remediation pane cleanly.
  • Approval suppression: pure approvals don't fire the LLM dispatch (verified by upstream filter in watcher_diff_events).

Test plan

  • make test green: 149 assertions across 4 suites (test-queue.sh, test-state.sh, test-watcher-events.sh, test-render-watcher-status.sh)
  • bash -n clean on all modified shell scripts
  • End-to-end smoke on a real repo: worker creates draft PR, watcher polls, comment triggers /address-review-comment, merge tears down watcher cleanly
  • Approval-only review correctly suppressed at the watcher level (no LLM dispatch)
  • For reviewer: kick the tyres on cmux — the cmux mux provider has a known issue with workspace creation that's not fully resolved (tested fully on tmux)

Known limitations / deferred follow-ups

  • Comment ID idempotency — currently the watcher tracks comment_count only, so the agent's own replies could re-dispatch; folded into a future plan.
  • Skill template propagation — new skills under templates/.claude/commands/ aren't auto-copied into existing projects; requires either a craft sync command or a fallback in dispatch_remediation.
  • Additional remediation skills (/resolve-merge-conflict, /investigate-pr-close) and operator-override skills (/pause, /kill, /restart, /watch) are designed but not built.
  • WORKER_AUTO_PERMISSIONS config knob to gate --dangerously-skip-permissions — not yet added (workers run autonomously by default, which is the existing upstream behaviour).
  • Worker pane /exit doesn't terminate Claude Code's process (only fixed for remediation panes via the force-kill in commit 5).

🤖 Generated with Claude Code

IcyWednesdays and others added 6 commits April 30, 2026 13:05
Architectural spec for an event-driven CI/CD delivery harness built
on Craft. Core ideas:

- Pluggable state backend (file-based today, Linear-ready later) so
  the orchestrator's read/write of task state goes through a single
  interface.
- Worker/watcher split: workers do the code work and exit; a separate
  long-lived bash watcher polls each PR and dispatches narrow
  remediation agents only when events fire (CI failure, review
  comment, merge conflict, merge, close).
- Two-stage operator gating: approve a task, then merge a PR. The
  loop in between is autonomous.
- Per-PR watcher pane in tmux/cmux with a live status snapshot
  (PR header, CI matrix, reviews, last action) above the polling log.
- Multiplexer-agnostic: tmux and cmux providers behind a shared
  interface.

The remaining plan documents (state backend, worker/watcher split,
UI polish, comment dispatch) describe each subsystem in detail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces a backend-agnostic state interface so task state can move
from local markdown files to Linear (or any other tracker) without
touching the orchestrator or skills.

- bin/lib/state.sh — dispatcher; sources the right backend based on
  STATE_BACKEND config (default: local).
- bin/lib/state-local.sh — file-based backend wrapping the existing
  queue.sh helpers. Adds high-level operations: state_claim_task,
  state_mark_waiting, state_mark_done, state_mark_blocked,
  state_append_note, state_create_subtask, state_find_task_by_id,
  state_list_tasks, state_count_tasks, plus state_task_* accessor
  aliases over queue.sh's task_* low-level helpers.
- bin/orchestrator.sh and bin/craft migrated to source state.sh.
  The orchestrator's timeout path uses state_mark_blocked.
- templates/craft.conf — adds STATE_BACKEND=local default.
- bin/craft doctor — reports the current state backend.
- Comprehensive test coverage (test/test-state.sh): 58 assertions
  covering every operation plus an end-to-end round-trip.

queue.sh remains untouched. Existing direct-queue.sh consumers
(test-queue.sh, plugin hooks) continue to work.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the in-worker PR polling loop with a separate long-lived
watcher process. The worker exits as soon as it creates the draft PR;
a deterministic bash watcher takes over post-PR, polls gh, and
dispatches narrow remediation agents only when events fire.

Key pieces:

- bin/lib/watcher-events.sh — pure event-detection functions
  (watcher_extract_state, watcher_diff_events, watcher_dispatch_action)
  testable with JSON fixtures.
- bin/watcher.sh — the polling loop. Reads the PR via gh, diffs state
  between iterations, persists state to .state/watchers/<pr>.state,
  invokes hooks/skills/state-transitions per event.
- Multiplexer helpers (bin/lib/mux-tmux.sh, mux-cmux.sh, mux.sh):
  spawn_watcher_pane creates a per-PR window/surface where the
  watcher runs.
- bin/orchestrator.sh: ACTIVE_WATCHERS tracking, ensure_watchers_for_waiting
  spawns a watcher for any task in waiting/ with a pr: field,
  reap_finished_watchers cleans up dead/exited watchers (incl. detecting
  panes that died unexpectedly).
- /work-task skill rewritten: ends at state_mark_waiting, no polling.
  Worker prompt now exposes CRAFT_ROOT/PROJECT_DIR/QUEUE_DIR so it
  can source state.sh and call state_mark_waiting.
- /fix-ci-failure skill: first remediation skill, dispatched by the
  watcher on ci_failed events.
- 34 fixture-based tests (test/test-watcher-events.sh + 3 paired
  state/JSON fixtures).

Watcher event taxonomy at this point:
  ci_failed       → /fix-ci-failure (LLM dispatch)
  merged          → state_mark_done (terminal)
  closed          → state_mark_blocked (terminal)
  draft_to_ready  → on_ready plugin hook
  others          → log only (deferred)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the silent log-only watcher window with a horizontally-split
pane (live status snapshot above, scrolling log below) and adds
freshness signals to the orchestrator dashboard.

- bin/render-watcher-status.sh — one-shot ANSI snapshot renderer.
  Reads the watcher's flat state file + cached gh JSON; prints PR
  header (number, title, state badge), CI check matrix, reviews
  block, and last-action timestamp.
- bin/render-dashboard.sh — extracted from inline render_dashboard
  in orchestrator.sh. New affordances: per-watcher freshness via
  state file mtime (green <60s / yellow <5m / red STALE), CI status
  badge per watcher, PR titles, recent activity feed.
- spawn_watcher_pane (tmux + cmux): splits the new window into a
  status snapshot above (re-rendered every 5s) and the log below.
- bin/watcher.sh: caches raw gh pr view JSON to <pr>.json each
  poll, records last_action/last_action_at in the state file,
  suppresses event:first_poll log noise, dedupes consecutive
  gh-failure messages.
- bin/orchestrator.sh: render_dashboard now wraps the script call
  with the runtime-only footer; check_milestone_completion uses
  .state/notified-milestones/ markers to fire the completion log
  once instead of every poll forever (pre-existing bug).
- 17 fixture-based tests for the watcher renderer (test-render-
  watcher-status.sh) bringing the suite to 143 assertions.

Pure bash + ANSI; no Node.js dependency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the watcher's event-driven dispatch to handle PR comments and
non-approval reviews, with the LLM-dispatched fix agent running in a
detached multiplexer pane that's torn down cleanly when the agent
either finishes or pushes a new commit.

Watcher events:
- watcher_diff_events: split review handling — emit
  new_review_received only when the review-count delta exceeds the
  approved-count delta (i.e., not just an approval landed).
  Pure approvals are filtered upstream so they don't trigger an LLM.
- watcher_dispatch_action: new_comment_received and new_review_received
  both → /address-review-comment (LLM dispatch).

New skill:
- templates/.claude/commands/address-review-comment.md — reads
  comments/reviews, decides actionability, makes one-fix-per-commit
  changes, pushes, exits.

Smarter remediation lifecycle:
- bin/lib/mux-tmux.sh, bin/lib/mux-cmux.sh: new spawn_task_pane_detached
  helpers that create a window/surface without stealing focus.
- bin/watcher.sh::dispatch_remediation rewritten:
  · Spawns the agent in a new (unfocused) pane via spawn_task_pane_detached.
  · Captures initial PR head SHA before dispatch.
  · Loops on three completion signals: pane exit / new commit pushed
    (SHA changed) / 5-minute timeout.
  · Force-kills the pane unconditionally at the end, fixing the
    Claude-Code-/exit-doesn't-terminate issue.
  · Falls back to inline execution if the detached spawn fails.

Plus testing-discovered fixes that landed during Plan 4/5 validation
(folded in here since they're tightly coupled to dispatch behaviour):
- bin/lib/state.sh + state-local.sh: resolve STATE_DIR via CRAFT_ROOT
  for zsh callers (BASH_SOURCE is unreliable under zsh; orchestrator
  exports CRAFT_ROOT into the worker prompt).
- bin/render-watcher-status.sh: coalesce CI check name across
  context → workflowName → name → "(unnamed)" so Buildkite/Actions
  checks render correctly.
- bin/lib/mux-cmux.sh: TMUX_SESSION compat alias so the
  orchestrator's session-name reference works under cmux too.

Test coverage extended to 149 assertions covering approval
suppression, both new dispatch entries, and existing happy paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updates the repo-level CLAUDE.md to describe the architecture this
branch introduces — worker/watcher split, event-driven dispatch,
state backend abstraction, UI surfaces, skill taxonomy — so future
Claude sessions in this repo don't need to rediscover the design
from the plan/spec docs.

Includes:
- Pointers to docs/superpowers/specs/ and docs/superpowers/plans/
  for deep dives.
- Process types (architect / orchestrator / worker / watcher /
  remediation) and their lifecycles.
- Event-to-action dispatch table.
- State backend interface signatures.
- UI surface descriptions (dashboard, watcher pane, remediation pane).
- Skill taxonomy (worker / remediation / deferred intervention).
- Test suites (4 files, 149 assertions, run via make test).
- Conventions including known macOS bash 3.2 / zsh quirks.
- Guidance for extending watcher events, state operations, and skills.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant