Skip to content

autoviral skill gaps: operator-manual recipes this short-video build needed (decoupled TTS, beat-cutting, ASR subtitle burn, generation resilience) #95

Description

@nanxingw

Summary

While producing a full 中文 short-video科普 end-to-end, I relied on several operational patterns that the skills/autoviral operator manual does not currently document. These aren't "taste" (which the skill correctly defers to sibling skills) — they're how to drive the workstation toward a short-video result, and I had to (re)derive them the hard way. Proposing they be added as recipes/gotchas.

Gaps found (each = proposed recipe or manual note)

1. Decouple narration from picture — never rely on i2v embedded voice

Multi-shot narration via a video model's embedded voice drifts in timbre and drops out between shots (the user hit exactly this). The correct operation: synthesize one continuous TTS track with a single locked voice id, generate silent picture, lay VO on its own track. Belongs in recipes/video/.

2. Beat-driven short-video assembly (native speed, one visual per phrase)

  • Cut to ~4–5s beats at NATIVE speed. Do not slow a 5–8s clip to fill a 15–20s shot — setpts frame-duplication at ~2.8× produces visible judder (the user's "卡顿").
  • Generate one dedicated visual per narration phrase instead of reframe-reusing a clip; interleaving [A,B,A,B] / reframe-reuse produces the "片段1→片段2→切回片段1" repetition the user flagged.
  • Order beats to match the narration's meaning, and keep same-source segments consecutive (never re-insert an earlier clip).

3. Subtitle burn-in recipe (ASR timing + ground-truth text)

ASR (stable-whisper) on the clean TTS VO gives accurate timestamps but wrong text (it mis-hears "内存条"→"那村调", "AI"→"黑爱", "DRAM"/"HBM"). Recipe: keep ASR timings, substitute the known script text, split lines ≤~14 chars for 抖音, burn with a libass-enabled ffmpeg. A recipes/video/burn-subtitles-asr-aligned.md would save a lot.

4. Generation resilience (gotchas)

  • Seedance/image endpoints occasionally return transient 500s — retry, don't abort.
  • A client-side timeout does NOT cancel server-side generation → you get billed for an orphan clip. Use generous timeouts.
  • Make batch generation idempotent/resumable via a manifest (skip already-done). Under session interruptions, single-clip foreground calls survived better than long background jobs (background jobs got torn down at turn boundaries).

5. Host ffmpeg vs managed ffmpeg

The host's Homebrew ffmpeg may lack libass (no subtitles filter) and libfreetype (no drawtext). The managed ~/.autoviral/bin/ffmpeg (6.0) has libass. Any skill step that burns text must prefer the managed binary. Worth a one-line gotcha in 05-conventions.

6. Covers: the image model renders native 中文 titles well

Verified: the image endpoint produced a poster with a large, correct, 大气 中文 title + subtitle + logo natively (no garbling) in one shot. A recipes/video/generate-cover.md (theme + title + logo prompt pattern) is worth adding.

7. Cost/quality knob to document

Seedance 720p ≈ $0.15/s vs 1080p ≈ $0.34/s; 720p upscaled to 1080p is fine for 抖音/小红书. Belongs in the video cost notes.

Note

Items 2 (beat pacing feel) shade into editorial taste — the mechanics (native speed, one-visual-per-phrase, no re-insertion) are operational and belong in the manual; the aesthetic judgment stays in a sibling taste skill.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions