tsugite

English | 日本語 | 中文 | 한국어

Vendor-neutral video pipeline that connects generation adapters and editing backends through a single manifest contract.

Each video job has its own project.yaml. For distribution, the repository keeps copyable examples under examples/ and ignores user projects under projects/. The safe flow is:

1. Validate the project and manifest.
2. Create a plan.
3. Stop at Gate 1 for human approval.
4. Run generation or assembly only after Coordinator approval.
5. Stop at Gate 2 for output QA.
6. Render only after Gate 2 approval.
7. Stop at Gate 3 for final video QA.

Agent Skills

Codex discovers the repository skill at .agents/skills/tsugite/SKILL.md. Invoke it with $tsugite, or let Codex select it for matching Tsugite video work.

Claude Code exposes .claude/skills/tsugite/SKILL.md as /tsugite and loads the same canonical workflow. The existing /tsugite-plan, /tsugite-verify, /tsugite-finalize, /tsugite-learning-review, and /shitate-import commands remain focused shortcuts. The root SKILL.md is a legacy compatibility entry.

Current Scope

• Manifest validation and local asset checks.
• Adapter registry for cli, mcp-agent, and mcp-client styles.
• CLI generation adapter wrappers for PixVerse/Kling.
• Source- and freshness-backed T2V/I2V prompt knowledge catalogs for PixVerse, Kling, and Seedance.
• A story-guide catalog covering 34 narrative, persuasion, documentary, genre, and music-video structures plus 35 contextual film-grammar and AI-video principles.
• TopView skill CLI generation adapter for T2V and single-frame I2V.
• Optional Hermes analysis handoff adapter.
• Local-media and generated-media assembly into dist/<run-id>/.
• Gate-bound editorial EDL compilation that retimes selected cuts, captions, and chapters for both Remotion and HyperFrames without modifying source media.
• Gate 2 QC report generation using manifest and media probes.
• Gate 3 QC report generation for final duration, resolution, fps, and audio/video streams.
• First-class image assets, speaker/pose metadata, and guarded presentation presets.
• Remotion and HyperFrames backend contracts.
• A Gate-bound audio adapter contract and an official HyperFrames media-use integration for BGM/SFX.
• Guarded run / render commands that require Coordinator role and prior Gate approval.
• A loopback-only project launcher and an optional, read-only 3D detail viewer under apps/workflow-viewer/.

Local Workflow Launcher and 3D Viewer

Public distribution of the Desktop app has ended. Tsugite is now distributed as source through GitHub and used from Codex or Claude Code; use the browser-based local launcher and 3D Viewer for inspection. The Electron source remains for development and regression testing only.

The supported flow is to open the Tsugite repository in your usual Codex or Claude environment, then use the loopback-only browser launcher beside it for project, template, Gate, and Viewer inspection. The launcher does not install an AI CLI, consume provider credits, send assets, start generation, render media, or change a Gate.

The development-only Electron shell still contains an embedded terminal for installed Codex CLI or Claude Code. Those CLIs retain their normal permissions and may read or write workspace files, run commands, or use the network; the shell is not a separate Tsugite sandbox. AI subscriptions are also separate from generation-provider billing. Gates do not sandbox general file operations, and run, render, and Gate decisions still require explicit human approval and the Coordinator role.

In the development-only Desktop shell, an empty project shelf can reopen the native workspace chooser. This behavior remains covered by regression tests but is not a supported distribution path.

The existing 3D Viewer remains available for detailed, seekable inspection. It turns bundled samples or a refreshed Tsugite snapshot into a navigable production floor with status-aware nodes, dependency lines, node details, and event playback. The 3D artifact itself stays static and read-only.

For a non-technical local entry point, install the nested Viewer dependencies once and then open the project launcher:

npm --prefix apps/workflow-viewer ci  # first time only
npm run viewer:open

Use the CLI for validate, plan, analyze, compose, review, run --dry-run, run, render, and Gate decisions. The existing Gate prerequisites, explicit approvals, and Coordinator actor checks still apply. Opening the launcher itself does not consume provider credits, send prompts or assets, start generation, render media, or change a Gate.

The launcher and its artifact server bind only to dynamically selected 127.0.0.1 ports. It lists direct projects/*/project.yaml entries and can refresh or open their read-only 3D snapshots. Refreshed snapshots are written only to a private 0700 temporary directory for the current launcher session, never back through a project output path, and are removed when the launcher closes. At startup, its Preferences & Learnings shelf reads local feedback.jsonl records across those projects and summarizes their observed / recurring / promoted / verified status. It reads at most 128 projects and fairly selects up to 1,000 of their latest records and diagnostics, reporting when either limit is reached. Pending proposals created by the dedicated learning-promotion automation are surfaced in an unread-style tab badge and a local pickup; manual proposals and other workflow results remain in the normal shelf and do not enter the pickup. The badge is a current approval-waiting count, not a separate read-state tracker. A human can review the target, change summary, evidence, and verification plan, then approve or reject the proposal. Either decision clears that pending item by appending a local decision to feedback.jsonl. Approval only permits a separate implementation task: it never rewrites prompts, templates, rules, Gates, or state. The launcher does not request browser notification permission, send desktop notifications, run as a resident service, or use an external notification destination. Stop it with Ctrl+C in the launching terminal.

cd apps/workflow-viewer
npm install
npm run dev
npm run test:coverage
npm run build

See apps/workflow-viewer/README.md for the JSON contract, controls, samples, and current limitations.

Setup

Prerequisites are Git, Node.js 22.12 or newer in the 22.x LTS line, npm 10 or newer, and FFmpeg including ffprobe.

# macOS
brew install ffmpeg

# Debian / Ubuntu
sudo apt-get update && sudo apt-get install -y ffmpeg

# Windows
winget install --id Gyan.FFmpeg -e

On Windows, reopen the terminal after installation. npm ci installs Remotion, HyperFrames, and the other repository dependencies locally; no global Remotion or HyperFrames install is needed. HyperFrames is a development dependency, so do not use npm ci --omit=dev. See the native Windows and PowerShell guide for the canonical launcher and CLI entrypoints.

From the repository root, the native PowerShell quick start is:

npm ci
npm --prefix apps/workflow-viewer ci
node bin/pipeline doctor --config examples/local-fixture/project.yaml --json
npm run viewer:open

Use node bin/pipeline ... in PowerShell instead of invoking the extensionless bin/pipeline file directly. Reopen PowerShell after installing or updating Node.js, FFmpeg, or a provider CLI so the updated PATH and PATHEXT are visible. Provider authentication, entitlements, and billing remain separate manual setup.

Provider CLIs such as PixVerse/Kling, external TopView/Hermes runtimes, credentials, and billing configuration are not installed or configured automatically. Prepare only the adapter you select, then rerun doctor. For TopView, doctor probes the skill's video_gen.py with the non-charging list-models command. It does not submit generation tasks; authentication and credits remain manual checks. Any unresolved blocking check makes the overall ok value false.

After a successful local first-time setup, Codex and Claude Code ask once, before the next substantive proposal, whether to add the optional learning-promotion automation and its host-standard completion notification. Choosing it requires selecting one primary host (Codex, Claude Desktop/Cowork, or Claude Code) and a cadence; choosing not to set it up suppresses the repeat question for that setup flow. The automation stays local, creates only human-approval candidates, and never enables browser/OS notifications or external destinations. See Learning Promotion Review Automation.

See docs/hyperframes-audio.md for HyperFrames-first BGM generation and SFX resolution. This path never falls back to ElevenLabs automatically.

Commands

Start with the built-in command catalog. General help lists every command and its safety level; command-specific help shows the accepted options without reading a project or contacting a provider.

node bin/pipeline --help
node bin/pipeline help validate

Add --json to help or operational commands when a script needs stable machine-readable output.

npm ci
npm run check
node bin/pipeline story-guides --request "A 30-second vertical ad showing value and proof" --duration 30 --json
node bin/pipeline guides --json
node bin/pipeline presets --backend remotion --json
cp -R examples/local-fixture projects/my-first-run
node bin/pipeline doctor --config projects/my-first-run/project.yaml --json
node bin/pipeline validate --config projects/my-first-run/project.yaml --json
node bin/pipeline plan --config projects/my-first-run/project.yaml --json
node bin/pipeline review --config projects/my-first-run/project.yaml --open --json
node bin/pipeline viewer --config projects/my-first-run/project.yaml --open --json
node bin/pipeline run --config projects/my-first-run/project.yaml --dry-run --json
node bin/pipeline finalize --config projects/my-first-run/project.yaml --json

presets is a project-independent, read-only query of the presentation presets declared by an installed backend. Use its returned presets list when creating or changing a manifest instead of typing an unverified preset ID.

review derives dist/<run-id>/review/index.html and review-data.json from the validated project, manifest, and plan. It presents a caption-first storyboard, character sheets, shot details, cost, plus overall and shot-level motion direction with a safe HTML/CSS approximation. The Gate 1 decision appears once, after every review section and the production conditions, without changing state.json or executing generation. Gate 1 approval and run start verify that both artifacts exist and belong to the current project. Use --output <directory> to override the destination, --state-dir <directory> for an alternate state root, and --open only when you want to open the local HTML. Use the canonical output location when the artifact must satisfy Gate 1.

For a project with composition, run analyze and then compose before review. compose reads the fixed local analysis and creates at most three backend-neutral proposals in analysis/composition-proposals.json; it does not change source media, the manifest, or Gate state. Compare the proposals in the review, set exactly one edit.composition.proposal_id, and regenerate the review. Only the selected proposal enters the Gate 1 digest. composition-edl.json and the reordered manifest are materialized by run after Gate 1 approval.

viewer converts the validated project and plan plus the current state.json, run-log.md, review, and Gate 2 / Gate 3 QC artifacts into dist/<run-id>/viewer/index.html and workflow.json. Run summaries and generation request records from run-log.md appear in the material-generation details. When Gate 2 QC references real media, the snapshot copies a bounded preview set (2 generated videos, 4 images, and 2 audio files) into viewer/previews/; the Gate 2 panel can display or play them directly. The Gate 3 final video is also copied and appears on the render, final-approval, and completion steps. References outside the run directory, links, missing files, and unsupported extensions are not copied. It is a read-only snapshot: it does not run adapters, change gates, or write state. Install the Viewer dependencies once with npm --prefix apps/workflow-viewer ci; rerun the command after the pipeline state changes. The timeline is deterministically reconstructed from the plan order and current artifacts because Tsugite does not yet persist a complete event history. --output, --state-dir, and --open follow the same local-artifact conventions as review.

run and render are intentionally gated:

node bin/pipeline gate --config projects/my-first-run/project.yaml --actor coordinator --gate gate-1 --decision approve --json
node bin/pipeline run --config projects/my-first-run/project.yaml --actor coordinator --json
node bin/pipeline gate --config projects/my-first-run/project.yaml --actor coordinator --gate gate-2 --decision approve_all --json
node bin/pipeline render --config projects/my-first-run/project.yaml --actor coordinator --json
node bin/pipeline gate --config projects/my-first-run/project.yaml --actor coordinator --gate gate-3 --decision approve --json

Do not run non-dry-run run or render without explicit human approval. Gate 3 also accepts re-render, which preserves Gate 1 and Gate 2 approval and returns the run to rendering. Gate 2 retry_specific is not implemented yet; use revise for a full re-plan.

Only after the user explicitly declares the selected video complete, first record the canonical output, QA evidence, and a closeout retrospective: failures, improvements, and next-run lessons (including an explicit no-failure result). Append failures to project feedback.jsonl and reusable rules to LESSONS.md. Repeated feedback keys or lessons with matching symptoms and causes are recorded as recurring and assessed as promotion candidates; a pending proposal requires a concrete target, change summary, and verification plan, and remains human-approval-gated. Then use finalize to clean up superseded media. The default preview is read-only. After reviewing its scope, a Coordinator may add --apply; this keeps the final run, source media referenced by the final manifest, and text records, while deleting video, audio, and image files from older runs, older QA, and unused project media. The result is recorded in completion-record.json inside the final run.

node bin/pipeline finalize --config projects/my-first-run/project.yaml --json
node bin/pipeline finalize --config projects/my-first-run/project.yaml --apply --actor coordinator --json

Optional Shitate Import

When using the separate Shitate repository, optionally import a selected run and anchor as an immutable, SHA-256-locked project snapshot. Shitate is not required for normal Tsugite usage.

node bin/pipeline shitate-import \
  --config projects/my-project/project.yaml \
  --shitate-root /absolute/path/to/shitate \
  --character hero \
  --run-id 20260713_three-view_v1 \
  --anchor references/images/main-anchor.png \
  --request-id shot-001 \
  --json

The command copies local files, adds the anchor and speaker to the manifest, and optionally changes one request to I2V. It never runs generation or changes a Gate. negative.txt is preserved but not silently applied because the current PixVerse video CLI has no negative-prompt option. See Shitate Integration.

Project File

Minimal local-media project, as used by examples/local-fixture/project.yaml:

slug: local-fixture
run_id: local-fixture-run
manifest: manifest.json
dist_dir: dist
edit:
  backend: remotion

Generation projects add a generation section:

generation:
  adapter: pixverse
  requests:
    - id: shot-001
      prompt: short prompt
      model: v6
      duration: 5
      aspect: "16:9"
      input_mode: text-to-video
      params: {}

plan returns request-specific prompt_guidance when the model and input mode match. Set prompt_guide.catalog when the knowledge catalog differs from the execution adapter. A catalog never implies execution capability and never rewrites the prompt. See Model Prompt Knowledge.

The optional Hermes adapter is a distribution-time opt-in. The base install does not require them; set them up only when a project.yaml selects one of those adapters. See Optional Adapters.

Growing the Pipeline

Tsugite does not become more personalized just because you generate many videos. It improves when you feed review notes, retry reasons, and repeated preferences back into the repository.

Structured feedback stays local in each projects/<job>/feedback.jsonl. Record a stable key for the same preference across projects so repetitions can be identified without treating generation count as learning. The lifecycle is observed (recorded once), recurring (repeated evidence), promoted (the approved change was implemented in a shared target), then verified (later output confirms the improvement). Approval is always a human decision and only grants implementation permission; neither pipeline feedback nor the launcher changes prompts, templates, checks, or operating rules automatically.

Use this loop:

1. Create a project under projects/.
2. Generate or assemble only after the Gate approvals.
3. Review the output and record what worked, what failed, and why you retried with pipeline feedback.
4. Keep one-off notes and the local feedback.jsonl inside that project.
5. Use repeated records with the same key as evidence, then promote a reusable change only after human approval.
6. Verify the promoted change against a later output before marking the feedback verified.

An optional Codex Automation, Claude Desktop/Cowork scheduled task, or Claude Code session may prepare this approval queue independently of whether the launcher is running. It is dedicated to preference/learning promotion review, not to reporting other automations. Each run stays local, records its supported source, adds at most three complete and non-duplicate pending proposals through the existing pipeline feedback CLI, and never edits shared source. See Learning Promotion Review Automation for registration and native host-notification behavior. Keep only one durable schedule active to avoid duplicate runs.

For example, record feedback against a copied local project and inspect the resulting JSON without exposing an absolute local path:

node bin/pipeline feedback --config projects/my-first-run/project.yaml \
  --key opening-audio --category audio --signal prefer --stage observed \
  --summary "Start music within the first 0.5 seconds" --json

Recommended promotion rule:

One-off preference       -> projects/<job>/notes.md + feedback.jsonl (observed)
Repeated preference key  -> feedback.jsonl (recurring; review for promotion)
Reusable style choice    -> examples/ or templates/
Machine-checkable issue  -> constraints.yaml / validate / doctor + tests/fixtures
Judgment-based rule      -> LESSONS.md -> .agents/skills/tsugite/SKILL.md / CLAUDE.md / AGENTS.md
QA rule                  -> Gate 2 / Gate 3 checks + report schema/tests
Public contract change   -> README / manifest/schema.md / docs/requirements.md

Every promotion requires human approval and should leave either a reproducing fixture and test, or a human-readable operating rule. Gate 2 / Gate 3 check changes should update the report shape and tests together. After promotion, use later project evidence to decide whether the preference is verified.

This is how the repo can grow toward your taste while still staying safe for distribution. Local projects stay ignored under projects/, and only reusable improvements are committed back to the source.

Repository Rules

• Keep core code vendor-neutral. Vendor-specific execution behavior belongs under adapters/ or backends/; source-backed advisory data belongs under knowledge/video-models/.
• Adapter directories must include constraints.md.
• mcp-agent adapters must include SKILL.md.
• Put user work under projects/; keep examples/ copyable and resettable.
• Failures that produce reusable rules should be recorded in LESSONS.md.

Production Notes

• examples/local-fixture/project.yaml is a fixture-style local validation config. Copy it into projects/ before editing.
• projects/* is ignored by git so local prompts, media, manifests, dist/, and run state stay out of distributable commits.
• npm ls may report @emnapi/runtime as extraneous after npm ci on npm 11 because optional wasm child packages remain in the lockfile while their platform-specific parents are skipped. Treat this as non-blocking only when npm ci, npm audit, build, tests, validate, plan, and run --dry-run all pass.
• npm run check enforces the vendor boundary, TypeScript build, the full test suite, and minimum coverage of 80% statements, functions, and lines, plus 75% branches for src/. Coverage uses at most four Vitest workers so process-heavy fixtures remain stable on high-core machines and CI runners.
• npm run security:audit checks both the production dependency tree and the full development tree, failing on moderate-or-higher advisories.
• Vite may warn because this workspace path contains *. Tests currently pass in this path; move the repo to a path without * if that warning becomes operationally noisy.