Skip to content

Add Audio8 TTS engine integration - #336

Draft
diodiogod wants to merge 1 commit into
mainfrom
codex/audio8-engine
Draft

Add Audio8 TTS engine integration#336
diodiogod wants to merge 1 commit into
mainfrom
codex/audio8-engine

Conversation

@diodiogod

Copy link
Copy Markdown
Owner

Summary

  • add Audio8 TTS Preview 0.6B as a unified Text and SRT engine
  • support reference-free generation, exact-transcript voice cloning, character voices, caching, inline tags, model download, and shared Transformers 4 isolation
  • harden Audio8/Step Audio EditX unload-reload behavior and normalize post-edited audio before chunk assembly
  • document Audio8 capabilities, license, model layout, and the isolated-worker import-path pitfall found during testing

Current status

Important

Audio8 is functional and produces valid audio in ComfyUI, but engine implementation testing is not finished. This PR is intentionally a draft until broader manual coverage is complete.

Validation completed

  • live reference-free and exact-transcript cloned speech generation
  • character switching and cache reuse
  • Audio8 → Step Audio EditX → Audio8 lifecycle, including repeated unload/reload
  • inline emotion/paralinguistic edits and mixed 2D/3D segment assembly
  • focused runtime/audio-processing suite: 31 passed
  • full unit suite: 143 passed, with 3 known unrelated failures

Remaining testing

  • broader greedy/sampling parameter combinations
  • SRT timing modes and longer subtitle sets
  • long-form chunking and interruption/cancellation
  • clean-install/model-download flow and additional hardware/runtime coverage
  • Transformers 5 compatibility remains future work; the current known-working implementation uses the shared Transformers 4 profile

Scope note

Audio8 is not added to Voice Designer in this implementation. The official checkpoint does not expose an instruction-conditioned voice-design mode, and cloning instead requires reference audio paired with its exact transcript.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant