Embedding contract for claude-swarm #115

Summary

claude-swarm assumes a person is driving: you launch it from the repository the agents should work on, watch the output, clean up afterwards. That works, and nothing here proposes changing it.

Driving it from another program: a CI job, a scheduler, anything that starts runs it doesn't watch and has to clean up after them. It is possible today, but only by depending on things that were never meant to be an interface: container name prefixes, /tmp/<project>-* paths reconstructed from the repo basename, a state file you have to source. It holds together until a rename in launch.sh silently breaks the integration, two runs share state that was never protected, or the code to inspect lives in a repository the engine checkout doesn't contain.

This issue lists the specific gaps and a set of changes that close them. Reference implementations exist as a stack of PRs, linked at the end.

Motivation

The problems

• Runtime state is predictable and shared. Everything lives under /tmp/<project>-*, keyed by the sanitized basename of the launch repo. The bare repo and submodule mirrors are chmod a+rwX (core.sharedRepository world), the state file is shell meant to be sourced, and nothing is locked: two checkouts with the same basename collide outright, and two start runs for the same project share one world-writable bare repo with no mutual exclusion.
• The engine has no target input. Agents always work on a clone of the launch repo. USAGE.md's answer for auditing another repository is passing TARGET_REPO/TARGET_REV through docker_args env passthrough: each container resolves the ref itself (a branch stays a branch, so agents in one run can land on different commits), clones over the network N times for an N-agent roster, and any credential needed to read a private target lives in every container's environment for the whole session.
• Submodules mirror top-level only. Mirrors are keyed by top-level submodule name and assume a $toplevel/.git/modules/<path> layout, so nested submodules fall back to network clones, and the assumption breaks under worktrees.
• Cleanup is destructive or manual. A stale container name is silently docker rm -f'd; a stale or unharvested bare repo is refused with an error message that tells you to rm -rf it. Both are fine when someone's watching; in an unattended loop they're where results get dropped without anyone noticing.
• There is no declared API. No control script, no versioned JSON, no container labels. An embedder greps docker ps for name prefixes and sources the state file and any rename inside launch.sh silently breaks that.
• post_process: null still builds a driver. A swarmfile without post_process appends the default driver to the SWARM_AGENTS build arg, so the image installs a CLI that nothing runs.

The changes

Five changes, independent enough to review separately, stacked in this order.

1. Private per-project runtime directory

Move the bare repo, submodule mirrors, lock, and state file under one 0700 directory per project, defaulting to $XDG_STATE_HOME/claude-swarm/<project>, with CLAUDE_SWARM_RUNTIME_DIR to override. The state file becomes validated, mode-600 JSON (claude-swarm.state/v1) instead of a sourced shell file.

Legacy /tmp state migrates itself on first run as a same-owner move. The move is validated first and fails closed on a held lock, symlink, foreign ownership, or destination collision, leaving the old paths untouched.

2. Host-resolved target mirrors

Make the target an explicit engine input. The host resolves TARGET_REPO@TARGET_REV once into a private mirror, fscks it, swaps it in transactionally, and mounts it read-only at /target-upstream. TARGET_REV is rewritten to the resolved 40-byte SHA before any container starts, so every agent works from the same immutable snapshot.

Submodule mirroring becomes recursive and manifest-driven: each initialized submodule gets a numbered mirror, and the host writes mirrors/manifest.tsv mapping display path to mirror. Containers walk the manifest, so nested .gitmodules are interpreted by the repo that owns them.

The read token is used only through a temporary host-side GIT_ASKPASS script and never enters a container. Agent passwordless sudo is revoked after the setup hook finishes.

docker_args can no longer set SWARM_READER_TOKEN, TARGET_REPO, TARGET_REV, TARGET_REV_BASE, TARGET_REV_BASE_REPO, SWARM_TARGET_MIRROR, or SWARM_TARGET_BASE_MIRROR; the engine strips them.

3. Transactional, fail-closed replacement

The bare repo is built aside, fscked, then swapped atomically with rollback. Replacement is refused entirely while any agent-work or swarm tip is not an ancestor of HEAD; stale-but-contained bares refresh in place. Bare permissions drop to owner-only.

Every agent and post-process container gets org.claude-swarm.{managed,project,engagement,role,agent-index} labels. An already-existing container name now fails the launch with a clear error instead of being force-removed. Labels are absent for containers created by older versions, so the code still falls back to the name prefix.

4. Versioned control plane

Add control.sh as the only programmatic surface embedders should touch:

• capabilities, paths, init, project-id, containers return versioned JSON (claude-swarm.control/v1, claude-swarm.containers/v1) and have no side effects.
• validate, start, stop, harvest, status, dashboard, post-process delegate to the engine.

paths reports runtime dir, bare repo, lock, state file, mirror dir, image name, and container prefix. containers reports status, exit code, OOM kill, engagement, role, and agent index per container. capabilities carries a feature list so future capabilities can be negotiated.

launch.sh validate runs the checks already done in cmd_start (prompt files exist, drivers exist, every profile resolves usable credentials) before the image build, so a missing key costs seconds instead of minutes.

Two one-line driver fixes ride along: kimi falls back to KIMI_MODEL_API_KEY, and a host ANTHROPIC_AUTH_TOKEN reports the token auth label.

5. Don't add a driver when post_process is absent

A swarmfile without post_process no longer appends the default driver to SWARM_AGENTS. Four lines plus a test fixture.

Breaking changes for embedders

1. Runtime paths move out of /tmp. Anything that read them should call control.sh paths instead.
2. The state file is JSON at mode 600; sourcing it no longer works.
3. docker_args cannot pass the target or reader-token variables anymore; set them in the host environment.
4. Agents lose passwordless sudo after the setup hook. Toolchain installs still run as root.
5. Every recursive submodule must be initialized at its pinned gitlink before start.
6. An existing container name fails the launch instead of being force-removed.
7. A bare repo holding an uncontained agent-work/swarm tip refuses replacement until harvested.
8. The bare repo is owner-only; core.sharedRepository world is gone.

Migration on the embedder side is small: drop the reserved docker_args passthroughs, clone from /target-upstream in the setup hook, call control.sh paths instead of reconstructing paths, and optionally lint against the reserved passthroughs.

Reference implementation

All five are implemented as a stack of PRs on a fork:

The stack builds on the two open driver PRs (#113 Kimi, #114 Qwen), so those would need to land first. It can be opened as five PRs in that order, or folded into fewer if that reviews better.

One open question

Dropping core.sharedRepository world assumes the container's agent uid matches the host user's uid. That holds on a typical uid-1000 workstation and Docker Desktop remaps anyway, but on a Linux host with a different operator uid the agent would lose write access to /upstream. Keeping a slightly wider mode or pinning the uid at image build time are both fine alternatives; preference welcome before opening.