Skip to content

feat(F205): Video Forge — declarative protocol engine + inline video delivery#1144

Open
mindfn wants to merge 11 commits into
zts212653:mainfrom
mindfn:clean/f205-media-plugins
Open

feat(F205): Video Forge — declarative protocol engine + inline video delivery#1144
mindfn wants to merge 11 commits into
zts212653:mainfrom
mindfn:clean/f205-media-plugins

Conversation

@mindfn

@mindfn mindfn commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Declarative protocol engine for video generation and video analysis providers (Zhipu, Kling, Jimeng, Gemini), driven by YAML templates instead of per-provider SDK code
  • Server-side poll loop with bounded retries (interval/maxAttempts from YAML), AbortSignal.timeout(30s) on every fetch, and terminal state detection (succeeded/failed/malformed)
  • Inline video delivery via kind:"file" rich blocks — poll tool auto-emits media blocks when callback credentials are available; FileBlock.tsx renders <video> for video MIME types
  • Auth strategies: API key (Bearer), query-param (with configurable paramName), JWT-HS256, HMAC-SHA256-v4 — all wired through protocol-server.ts assembly
  • Plugin manifests: video-gen (text2video, image2video) and video-analysis (analyze_url) with per-provider protocol YAML
  • MCP config drift detection improvements and capability orchestrator fixes for plugin-managed MCP entries

Resolves the protocol engine and provider plugin portion of community PR #689 under feature anchor F205.

Test Evidence

  • Protocol engine: 43 tests — template rendering, JSONPath extraction, auth signing, schema validation, real YAML validation, buildProviderFromEnv assembly
  • Protocol tools handler: 22 tests — poll loop (succeed/retry/fail/malformed/exhaustion/inheritance), MIME heuristic regression, AbortSignal.timeout(30_000) interception, camelCase variable body wiring, paramName full production chain (via exported buildProtocolToolConfig), auth schema validation
  • Capability route: extended tests for plugin-managed MCP entries
  • MCP config adapters: refactored tests aligned with simplified adapter logic
  • Peer review: 5 rounds (R5→R9) by Sol (gpt-5.6-sol), all P1/P2 findings resolved
  • Manual acceptance: full worktree validation — enable plugin → configure provider → invoke generation → verify inline media playback

Acceptance Criteria Coverage

  • AC-B1: Protocol templates are schema-validated (Zod ProtocolTemplateSchema)
  • AC-B2: Auth strategies do not leak secrets (credentials in memory only, not in tool output)
  • AC-B3: Bounded polling with retry, timeout, and terminal failure states
  • AC-B4: Provider config stored through F202 plugin config boundary
  • AC-C1: Focused tests for template rendering, JSONPath, schema, auth, and provider templates
  • AC-C2: End-to-end proof via manual worktree acceptance testing

Files Changed

New (protocol engine + plugins): 20 files — YAML templates, auth strategies, engine, loader, types, protocol-server, protocol-tools, 2 test files, plugin manifests, skill docs

Modified: 50 files — capability orchestrator, MCP config adapters (simplified), rich block rules (file block schema), FileBlock.tsx (video rendering), capability-tips (F205 tip), test updates

🐾 Peer-reviewed by Sol (gpt-5.6-sol) — R9 verdict: PASS (0×P1 / 0×P2)

🤖 Generated with Claude Code

…delivery

Two complete plugin capabilities (video-gen, video-analysis) with:

Protocol Engine:
- YAML-driven MCP tool generation (submit/poll/execute)
- 5 auth strategies (apikey, query-param, jwt-hs256, hmac-sha256-v4, custom)
- Template rendering with filters (default, base64)
- JSONPath response extraction
- Default baseUrl from protocol YAML (provider fallback)
- Capability-aware MIME inference (video + image types)

Inline Video Delivery:
- RichFileBlock schema (kind:"file") for media playback in chat
- FileBlock.tsx renders <video>/<img> based on mimeType
- emitMediaRichBlock() auto-emits via callback API on poll success
- Best-effort graceful degradation when callback unavailable

Plugin Infrastructure:
- PluginResourceActivator registers plugin MCP servers
- Plugin capability source migration (plugin → cat-cafe)
- MCP drift detection for plugin-managed servers
- Skill sync for plugin-owned skills

Providers: zhipu, kling, jimeng (video-gen); gemini, zhipu (video-analysis)
Tests: 39 protocol engine tests, plugin manifest safety tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mindfn mindfn requested a review from zts212653 as a code owner July 13, 2026 09:10

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3d3ed8e2ef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/api/src/mcp/mcp-sync-engine.ts Outdated
Comment thread packages/api/src/plugins/video-gen/protocols/jimeng.yaml Outdated
Comment thread packages/api/src/domains/plugin/PluginResourceActivator.ts Outdated
mindfn and others added 4 commits July 13, 2026 17:42
…ation tests

The protocol engine (mcp-server) is separate from the skills management
system.  Three F205 changes over-reached into skills code:

1. Source migration in readCapabilitiesConfig converted source:'plugin'
   to source:'cat-cafe' on read — this leaked plugin disable state into
   loadDisabledCatCafeSkillNames and collectCatCafeSkillPolicy, breaking
   global inheritance for projects with same-id plugin entries.

2. !cap.pluginId filter in managedCaps excluded plugin skills from
   configDisabledSet — disabled plugin skills were treated as enabled.

3. !cap.pluginId filter in updateSkillMountPaths and
   removeCatCafeSkillCapabilities — plugin skills were neither synced
   nor cleaned up properly.

All three upstream tests now pass: same-id plugin policy (GET), global
guard (sync-skill), mount/unmount lifecycle (sync).

[宪宪/claude-opus-4-6🐾]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The video and monitor-wave SVG paths (from 暹罗猫's design) were added
in earlier F205 commits but lost during squash into the final commit.
Without these paths, HubIcon returns null and only the colored
background renders — no icon inside.

[宪宪/claude-opus-4-6🐾]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…lt, runtime root

1. (P1) Restore source!='external' guard in MCP orphan cleanup.
   Project-only external MCPs are legitimate (installed via
   POST /api/capabilities/mcp/install with projectPath) and must
   not be deleted during cascade sync.

2. (P2) Add model defaults to all Jimeng poll req_key fields.
   Without defaults, poll sends req_key="" when VIDEO_GEN_MODEL
   is blank, mismatching submit's default and causing task
   lookup failures.

3. (P2) Use resolveMainProjectRoot() for plugin MCP entrypoint
   resolution. The video plugin's protocol-server.js is in the
   monorepo; resolveProjectRoot() may return a config workspace
   in split deployments, breaking the relative path lookup.

[宪宪/claude-opus-4-6🐾]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mindfn

mindfn commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

All three codex findings addressed in 4bacda8a6:

P1 — Preserve project-local MCPs during sync

Restored source !== 'external' guard. The install endpoint (POST /api/capabilities/mcp/install) can write project-only entries when projectPath is set, so external MCPs in project config are legitimate and must not be orphan-cleaned.

P2 — Jimeng poll req_key defaults

Added | default: to all three poll req_key fields (text2video, image2video, text2image) matching their submit counterparts. Without defaults, poll sends empty req_key when VIDEO_GEN_MODEL is unset.

P2 — Runtime root for plugin MCP entrypoints

Changed to resolveMainProjectRoot() for arg resolution and workingDir. Plugin MCP entrypoints (packages/mcp-server/dist/protocol-server.js) are repo-relative — using resolveProjectRoot() would break in split deployments where the active project is not the monorepo.

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4bacda8a69

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/api/src/mcp/mcp-drift-detector.ts Outdated
Comment thread packages/mcp-server/src/protocol-engine/loader.ts
Comment thread packages/api/src/domains/plugin/PluginResourceActivator.ts Outdated
…rkingDir

P1 — Restore source!='external' guard in mcp-drift-detector to match
sync engine (project-local MCP installs via POST /api/capabilities/mcp/install
are legitimate, not orphans).

P1 — Add yaml to mcp-server/package.json dependencies. The protocol
loader imports yaml but only the root devDependencies declared it,
which breaks production installs that strip devDependencies.

P2 — Restore plugin-local workingDir in PluginResourceActivator. Args
are already resolved to absolute paths via existsSync fallback, so
workingDir should remain the plugin folder for plugin-local file access
compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mindfn

mindfn commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

All three R2 codex findings addressed in 3626da200:

P1 — Restore external MCP guard in drift detector

Restored source !== 'external' filter in mcp-drift-detector.ts orphan detection — same class of bug as the sync engine fix in R1. Project-local MCPs installed via POST /api/capabilities/mcp/install with projectPath are legitimate and must not be flagged as orphans. Updated corresponding test to verify preservation.

P1 — Declare yaml as mcp-server runtime dependency

Added yaml: ^2.8.1 to packages/mcp-server/package.json dependencies. The protocol engine loader imports yaml at runtime, but it was only declared in root devDependencies — production installs stripping devDeps would fail with Cannot find package 'yaml'.

P2 — Restore plugin-local workingDir

Reverted workingDir back to join(pluginsDir, manifest.id) (the plugin's own folder). The R1 fix already resolves args to absolute paths via existsSync fallback, so repo-relative entrypoints like packages/mcp-server/dist/protocol-server.js work regardless of cwd. Keeping workingDir as the plugin folder preserves backward compatibility for plugin-local file access patterns.

@codex review

pnpm-lock.yaml was out of sync after adding yaml to
packages/mcp-server/package.json dependencies. CI uses
--frozen-lockfile which rejects mismatches.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3626da2000

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/mcp-server/package.json
- mcp-drift-sync test: external-source MCPs are preserved (not flagged
  as orphans) since drift detector restored source!='external' guard
- plugin-manifest-safety test: workingDir is plugin folder
  (join(pluginsDir, manifestId)), not project root — args are resolved
  to absolute paths separately via existsSync fallback

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mindfn

mindfn commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

R3 lockfile finding was stale — already fixed in 87f46a4db. Test assertions aligned in 402caf888.

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 402caf8888

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/api/src/routes/capabilities.ts Outdated
When DEFAULT_OWNER_USER_ID is unset, allowMissingOwner:true caused
requireCapabilityWriteOwner to return null for any sessionUserId,
making secrets visible to any authenticated session regardless of
network origin. Add isLocalCapabilityWriteRequest check so
single-user mode only exposes secrets on localhost — matching the
docstring intent of "any authenticated local user."

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@mindfn

mindfn commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

R4 finding addressed in e260ecc70:

P1 — Require localhost for MCP secrets in single-user mode

When DEFAULT_OWNER_USER_ID is unset, allowMissingOwner: true made requireCapabilityWriteOwner return null for any session — secrets were visible to any authenticated request regardless of network origin. Added isLocalCapabilityWriteRequest(request) check in the single-user fallthrough path so secrets are only exposed on localhost, matching the docstring intent.

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: e260ecc70d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant