feat(agent-runtime): autonomous OpenMontage agent runtime + LLM prompt director (Gap B)

docs/adr/0016-autonomous-agent-runtime.md

@@ -0,0 +1,123 @@
+# ADR 0016 — Autonomous agent runtime for the public gateway (Gap B, Phase 1)
+
+**Status:** Accepted; MVP implemented + offline-validated. Live run pending a cloud LLM key.
+**Date:** 2026-06-23
+**Related:** ADR 0011/0013 (agent gateway, re-anchor), AGENT_GUIDE.md (instruction-driven model).
+
+## 1. Problem (Gap B)
+
+`agent.kakeya.ai` only did **bare text→video** through the WAN cluster. The gateway's `mode=agent`
+requires an `AGENT_RUNTIME_CMD`, which was unattached (`agent_runtime=false`) — so none of
+OpenMontage's pipeline/skills (script → scene_plan → assets via selectors → compose) ran. The user
+asked to **integrate the OpenMontage agent** so the public service runs a real multi-stage pipeline.
+
+## 2. Key finding
+
+OpenMontage is **instruction-driven**: there is **no Python pipeline runner** — the *host LLM*
+(Cursor/Claude) is normally the orchestrator, reading manifests + director skills and driving tools.
+To run **unattended** behind the gateway, we must supply our own LLM reasoner. In-repo LLM access is
+only `kakeya_llm`/`llm_selector` (mechanical text). So this ADR introduces a new **autonomous agent
+runtime**.
+
+## 3. Decision
+
+New module `services/agent_runtime/` exposed as `AGENT_RUNTIME_CMD`
+(`python -m services.agent_runtime.run --prompt … --out …`):
+
+- **`llm.py`** — LLM client: `AGENT_LLM = anthropic|openai|kakeya|kakeya_grpc|stub` (auto-detected
+  from `ANTHROPIC_API_KEY`/`OPENAI_API_KEY`/`KAKEYA_GRPC_ADDRESS`/`KAKEYA_ENDPOINT`). `complete_json`
+  parses + retries. Creative reasoning runs here; the host agent is no longer required at runtime.
+  - The cloud + `kakeya` (HTTP shim) providers are **urllib-only** (no heavy deps).
+  - **`kakeya_grpc`** (the head's live path) talks **directly** to the user-run Kakeya engine over
+    its native gRPC `RuntimeService` (token-id level — the real stable surface; the HTTP shim is
+    deprecated/single-session). It is an **external dependency contract**: OpenMontage does **not**
+    vendor the Kakeya SDK/proto — point `KAKEYA_REPO` at a checkout (must expose a top-level `kakeya`
+    package + `sdks/python`) and the `kakeya`/`transformers` imports happen **lazily** only when this
+    provider is selected, so grpc/transformers never become core deps (ADR 0001 D5/D7). Env:
+    `KAKEYA_GRPC_ADDRESS` (e.g. `127.0.0.1:51051`), `KAKEYA_TOKENIZER_ID` (HF/MLX tokenizer dir;
+    falls back to `KAKEYA_VERIFIER_ID`). Flow: tokenizer `apply_chat_template(enable_thinking=False)`
+    → `Client.create_session(eos_token_ids=…)` → `append` → `generate(max_tokens)` → `decode`.
+    Host deps (head `~/.venv-distwan`): `grpcio`, `transformers`, `mlx`/`mlx-lm`. Missing
+    `KAKEYA_REPO` → clear actionable `RuntimeError`.
+- **`run.py`** — minimal unattended orchestrator that reuses the real machinery:
+  - LLM produces **brief → script → scene_plan** (each schema-validated via `schemas.artifacts`
+    and checkpointed via `lib/checkpoint`, exactly like the governed pipeline; director-skill text
+    is loaded as LLM context — "intelligence in the skills").
+  - **prompt director** (`direct_video_prompt`): before each clip, the LLM rewrites the scene
+    description into ONE rich, model-appropriate **native-T2V** prompt (subject+action+setting+
+    lighting+camera+style, ~40–70 words, positive phrasing). Guidance is the **Layer-3 skill the
+    video provider advertises** (`_resolve_video_skill` reads the provider's `agent_skills`, e.g.
+    `ai-video-gen`/`ltx2`, from `.agents/skills` or `.claude/skills`) — i.e. it activates the prompt-
+    director knowledge OpenMontage already ships rather than inventing a new one. This is the
+    cross-attention insight from ADR 0015: enrich the text condition and hand a strong prompt to T2V,
+    instead of refining a weak low-res seed via I2V. Directed prompts are persisted to
+    `assets/director_prompts.json` for transparency. Needs no video GPU — runs as soon as the LLM is up.
+  - assets: per-scene video via **`video_selector`** (auto-routes to the BEST available provider —
+    premium Seedance/Kling/Veo if configured, else the local WAN draft cluster) using the **directed**
+    prompt, narration via **`tts_selector`**, one music track via **`music_gen`** (skipped when unconfigured).
+  - deterministic **edit_decisions** → **`audio_mixer`** + **`video_compose`** (ffmpeg) → final mp4.
+- `AGENT_RUNTIME_FAKE_ASSETS=1` swaps providers for ffmpeg fixtures so the harness is CI-testable
+  with no keys/GPU.
+
+**Topology** (per the cost/stability recommendation): agent runtime runs on the **head Mac CPU**
+(cloud LLM) next to the gateway; headless Mac stays a video worker; vast does heavy generation. The
+agent uses a cloud LLM (light CPU/network) — it does **not** steal GPU from video.
+
+## 3b. Quality, consistency, governance (added)
+
+Three upgrades toward production-grade, all in `run.py` (LLM-driven, schema-light sidecars):
+
+- **(1) Render quality passthrough.** `generate_clip` renders each scene through the **gateway**
+  `/v1/videos` when `AGENT_GATEWAY_URL` is set (`quality=AGENT_VIDEO_QUALITY`, default `standard`;
+  `high` → Hunyuan native 720p — the tier from ADR 0017), reusing the whole distributed pipeline.
+  Without a gateway URL it falls back to `video_selector` with a `resolution` hint. The runtime is the
+  director/editor; the gateway is the render farm. (`AGENT_GATEWAY_API_KEY` or `~/.kakeya/...` key.)
+- **(2) Cross-scene consistency.** `plan_style_bible` produces ONE fixed `subject`+`style` (a textual
+  "continuity bible") that `direct_video_prompt` embeds verbatim into EVERY shot's prompt, so the
+  recurring subject/style stays consistent across scenes (mitigates the "different dog each shot"
+  gap). Written to `assets/style_bible.json`. Toggle `AGENT_CONSISTENCY` (default on).
+- **(3) Governance reviewer.** `review_plan` is an LLM reviewer over the brief + directed prompts
+  (subject consistency / coherence / prompt quality) → `{approved, issues, revised_prompts}`. With
+  `AGENT_AUTO_REVISE` (default on) flagged prompts are auto-fixed; `AGENT_REQUIRE_APPROVAL=1` gates
+  (halts with the plan written for human approval). Written to `assets/review.json`. Toggle
+  `AGENT_REVIEW` (default on). All visible under `--plan-only` (no GPU).
+
+## 4. Honest scope / limits
+
+- This is an **MVP "auto" path**, not the full governed `animated-explainer` pipeline (no
+  research/proposal, human-approval gates, reviewer governance, or per-stage director nuance).
+- **Needs a cloud LLM key** to reason (none configured). Without it, `mode=agent` stays honestly
+  "not attached".
+- **Output quality is provider-bounded.** With only the WAN cluster, clips are draft quality.
+  Configure a **premium video provider** (e.g. fal.ai/Seedance) and `video_selector` auto-upgrades —
+  that is the real path to production-grade. TTS/music/images need their own keys.
+- Not "all skills integrated": skills light up as their providers are configured + as more pipelines
+  are wired. This is the first integration, not the finish line.
+
+## 5. Validation
+
+`tests/tools/test_agent_runtime.py` (offline, no keys/GPU): stub LLM emits valid brief/script/
+scene_plan; `AGENT_RUNTIME_FAKE_ASSETS=1` → full plumbing (plan → validate → **prompt director** →
+checkpoint → per-scene assets → audio mix → ffmpeg compose) → **valid h264 mp4** + checkpoints for
+idea/script/scene_plan/assets/edit. The test asserts each scene received a **directed** prompt
+(enriched, `!= raw` description) via `director_prompts.json`. JSON-extraction unit test. 2 pass.
+
+## 6. Next
+
+- Deploy: set `AGENT_RUNTIME_CMD="<venv> -m services.agent_runtime.run"` + an LLM endpoint on the
+  head; live-test `mode=agent` via `agent.kakeya.ai`. **LLM endpoint smoke test:** run
+  `python -m services.agent_runtime.run --check-llm` first — one tiny round-trip prints
+  provider/model + latency and exits 0/1/2 (ok/error/no-endpoint), so the reasoner is verified
+  before a full (slow) pipeline run. The head's live config uses **`AGENT_LLM=kakeya_grpc`** against
+  the Gemma 26B gRPC runtime (`KAKEYA_GRPC_ADDRESS=127.0.0.1:51051`, `KAKEYA_REPO`, `KAKEYA_TOKENIZER_ID`)
+  — `--check-llm` prints `OK`, confirming end-to-end reachability.
+- **`--plan-only`**: runs the LLM planning (brief→script→scene_plan) + the prompt director and writes
+  `director_prompts.json`, then exits **before any video render**. This is the fast,
+  **no-GPU / no-gateway / no-API-key** way to validate that the LLM + director actually work on the
+  head: `python -m services.agent_runtime.run --prompt "…" --plan-only`. The gateway's `X-API-Key`
+  (set as `AGENT_GATEWAY_API_KEY`) is only needed for the *public* path; the runtime itself is a plain
+  module and needs no key — run it directly to verify the feature.
+- Configure premium video provider for production-grade clips (the quality answer from the
+  draft-vs-premium discussion).
+- Grow toward the governed pipeline (proposal/approval/reviewer, more director skills) and more
+  skills (avatar, images, music) as keys are added.

docs/adr/0017-hunyuanvideo-quality-provider.md

@@ -0,0 +1,48 @@
+# ADR 0017 — HunyuanVideo as the default quality video provider (A/B vs WAN)
+
+**Status:** Accepted (decision: adopt Hunyuan as the quality layer). Productization pending.
+**Date:** 2026-06-23
+**Related:** ADR 0015/0016 (quality/duration, agent runtime), the "four open-source models as
+providers" design.
+
+## 1. Question
+
+Does replacing/augmenting WAN 2.1 with **HunyuanVideo** (13B) improve video quality enough to make it
+the default quality layer? Decided by a **real A/B** on the same prompt, judged on actual frames.
+
+## 2. Setup
+
+- vast RTX PRO 6000 Blackwell (97 G). Same prompt: *"a border collie dog running through a snowy
+  forest, cinematic, soft winter light."* Both 1280×720, ~2 s.
+- **WAN** = current `quality=high` path (1.3B T2V seed → I2V-14B @720p), via the public gateway.
+- **Hunyuan** = `hunyuanvideo-community/HunyuanVideo` (diffusers `HunyuanVideoPipeline`), 45 frames,
+  30 steps, no offload. New `HunyuanBackend` added to `grpc_worker.py` (`--backend hunyuan`, ops
+  framework/i2v).
+
+## 3. Result (frame-level visual judgment)
+
+- **WAN**: a chaotic, abstract blob field — **no recognizable dog or forest**, heavy SR artifacts.
+  Effectively unusable. (Root cause: the 1.3B low-res T2V *seed* is garbage; I2V amplifies it.)
+- **Hunyuan**: a **coherent, photorealistic** dog walking through a snowy forest, cinematic shallow
+  depth of field, realistic snow + blurred trunks. **Clearly production-looking** for a single clip.
+
+**Verdict: adopt HunyuanVideo as the default quality layer.** The gap is night-and-day.
+
+## 4. Honest caveats
+
+- **Speed/VRAM:** Hunyuan ~13–16 s/step × 30 + VAE decode ≈ **~8 min for a ~2 s 720p clip**, and it
+  needs the GPU largely to itself (~49 GB active; the resident 85 GB WAN worker had to be evicted to
+  avoid OOM). Slower + heavier than WAN.
+- **Prompt adherence:** it rendered a Shiba/Akita-type dog, not specifically a *border collie* — scene
+  quality is excellent but fine-grained subject control needs prompt work (Layer-3 skill).
+- **Long-form:** multi-shot >5 s still needs `HunyuanVideo-I2V` chunking (drift risk remains).
+- **License:** Tencent community license (region/MAU limits) — stricter than WAN's Apache-2.0.
+
+## 5. Decision / next
+
+1. **Hunyuan = default quality provider** (single-clip T2V hero path); WAN demoted to fast/draft +
+   fallback. Both stay (the four-model provider design).
+2. **Productize:** register `hunyuan_video` as a `video_generation` provider so the agent/
+   `video_selector` routes to it for quality; resolve the VRAM tradeoff (can't keep WAN-14B (85 G) +
+   Hunyuan resident — load the quality model on demand or dedicate the box to Hunyuan).
+3. Wire HunyuanVideo-I2V for long-form continuity; improve prompting for subject fidelity.

services/agent_gateway/deploy/vast_refiner_supervisor.sh

@@ -87,9 +87,18 @@ once() {
     if ! pgrep -f "KAKEYA_HELD=$VAST_SSH_HOST" >/dev/null 2>&1; then
       pkill -f "KAKEYA_HELD=" 2>/dev/null   # clean stale holds (e.g. old host)
       OPS="framework,refine,i2v"; [ -z "${VAST_I2V_MODEL:-}" ] && OPS="framework,refine"
+      # Two hardening fixes for INTERNAL BrokenPipe on real jobs:
+      #  1) --preload: load the T2V model at BOOT (server binds only after), so the first RPC is
+      #     instant instead of triggering a ~30s cold load mid-RPC.
+      #  2) Redirect the worker's stdout/stderr to a FILE ON THE BOX (not through the held-SSH pipe).
+      #     The diffusion writes progress bars to stderr; sending that over the SSH channel both
+      #     (a) raised BrokenPipeError inside the op when the channel hiccuped, and (b) the heavy
+      #     traffic itself triggered "Connection reset by peer" that killed the worker mid-job.
+      #     With output on a local file the held SSH carries ~no data and just keeps the process alive.
+      #     HF/tqdm bars are also disabled as defense in depth. Read the worker log at $VAST_WORKER_LOG.
       nohup ssh $SSH_OPTS -o ServerAliveInterval=20 -o ServerAliveCountMax=1000 \
           -p "$VAST_SSH_PORT" "$VAST_SSH_USER@$VAST_SSH_HOST" \
-          "cd /workspace/distwan && KAKEYA_HELD=$VAST_SSH_HOST HF_HOME=${VAST_HF_HOME:-/root/.hf_home} CUDA_I2V_MODEL='${VAST_I2V_MODEL:-}' CUDA_I2V_OFFLOAD='${VAST_I2V_OFFLOAD:-0}' exec ${VAST_VENV_PY:-/venv/main/bin/python} grpc_worker.py --backend cuda --host 0.0.0.0 --port $VAST_REMOTE_PORT --ops $OPS" \
+          "cd /workspace/distwan && KAKEYA_HELD=$VAST_SSH_HOST HF_HOME=${VAST_HF_HOME:-/root/.hf_home} HF_HUB_DISABLE_PROGRESS_BARS=1 TQDM_DISABLE=1 CUDA_I2V_MODEL='${VAST_I2V_MODEL:-}' CUDA_I2V_OFFLOAD='${VAST_I2V_OFFLOAD:-0}' exec ${VAST_VENV_PY:-/venv/main/bin/python} grpc_worker.py --backend cuda --host 0.0.0.0 --port $VAST_REMOTE_PORT --ops $OPS --preload > ${VAST_WORKER_LOG:-/workspace/distwan/worker.log} 2>&1" \
           >> "$HOME/.openmontage-logs/vast_held_worker.log" 2>&1 &
       log "spawned held worker ssh -> $VAST_SSH_HOST (ops=$OPS); model load may take minutes"
       sleep 3
@@ -114,10 +123,29 @@ once() {
     sleep 2
   fi
 
+  # Optional Hunyuan premium worker (subproc) for the 'high' tier (ADR 0017). Runs as a SECOND tmux
+  # worker on the same box (port VAST_HUNYUAN_REMOTE), tunneled to VAST_HUNYUAN_LOCAL on the head, and
+  # appended to the worker list so the gateway's --prefer-backend=hunyuan routing can reach it.
+  HY=""
+  if [ "${VAST_HUNYUAN:-0}" = "1" ]; then
+    HR="${VAST_HUNYUAN_REMOTE:-50053}"; HL="${VAST_HUNYUAN_LOCAL:-50054}"
+    ssh $SSH_OPTS -p "$VAST_SSH_PORT" "$VAST_SSH_USER@$VAST_SSH_HOST" \
+      "tmux has-session -t hunyuan 2>/dev/null || tmux new-session -d -s hunyuan \"cd /workspace/distwan && HF_HOME=${VAST_HF_HOME:-/root/.hf_home} HUNYUAN_OFFLOAD=${VAST_HUNYUAN_OFFLOAD:-0} DISTWAN_TMP=/workspace HF_HUB_DISABLE_PROGRESS_BARS=1 TQDM_DISABLE=1 ${VAST_VENV_PY:-/venv/main/bin/python} grpc_worker.py --backend hunyuan --subproc --ops framework --host 0.0.0.0 --port $HR > /workspace/distwan/hunyuan_sp.log 2>&1\"" 2>/dev/null
+    if ssh $SSH_OPTS -p "$VAST_SSH_PORT" "$VAST_SSH_USER@$VAST_SSH_HOST" \
+         "bash -c 'exec 3<>/dev/tcp/127.0.0.1/$HR' 2>/dev/null && echo up" 2>/dev/null | grep -q up; then
+      nc -z 127.0.0.1 "$HL" >/dev/null 2>&1 || {
+        pkill -f "L $HL:localhost:$HR" 2>/dev/null; sleep 1
+        ssh $SSH_OPTS -o ExitOnForwardFailure=yes -fN -L "$HL:localhost:$HR" \
+            -p "$VAST_SSH_PORT" "$VAST_SSH_USER@$VAST_SSH_HOST" 2>/dev/null; sleep 2; }
+      nc -z 127.0.0.1 "$HL" >/dev/null 2>&1 && HY=",127.0.0.1:$HL"
+    fi
+  fi
+
   if reachable_local; then
-    write_workers "$BASE_WORKERS,127.0.0.1:$VAST_LOCAL_PORT"; log "vast refiner ONLINE -> 3-node (added 127.0.0.1:$VAST_LOCAL_PORT)"
+    write_workers "$BASE_WORKERS,127.0.0.1:$VAST_LOCAL_PORT$HY"
+    log "vast refiner ONLINE -> $BASE_WORKERS,127.0.0.1:$VAST_LOCAL_PORT$HY"
   else
-    write_workers "$BASE_WORKERS"; log "tunnel failed -> 2-Mac fallback"
+    write_workers "$BASE_WORKERS$HY"; log "wan tunnel failed -> $BASE_WORKERS$HY"
   fi
 }