MiniMax H3 Three-Worker Pipeline

A process-isolated MiniMax H3 FL2VA serving stack for 24 GB NVIDIA GPUs such as the RTX 3090.

Security: the coordinator and Studio do not implement authentication. Keep them on loopback or behind an authenticated reverse proxy. See SECURITY.md.

The pipeline never keeps the text encoder, diffusion transformer, and decoders resident together. Each stage runs in a fresh ComfyUI OS process, writes a SHA-256-verified safetensors handoff, and exits before the next stage starts.

This repository contains the complete coordinator, custom ComfyUI handoff nodes, browser Studio, llama-swap examples, service templates, tests, and the adapter that connects the backend to Hermes Agent's existing native video_generate tool. It does not add a second video tool or alter Hermes core.

No model weights are included.

Why this exists

A conventional H3 graph can leave several large components mapped or cached at once. Logical unload calls and torch.cuda.empty_cache() are not hard isolation boundaries for CUDA allocations, CPU mappings, pinned buffers, or child threads. This implementation treats OS-process exit as the release boundary.

client / Hermes native video_generate / Studio
                        |
                 FastAPI coordinator
                        |
      +-----------------+------------------+
      |                 |                  |
  1. encoder        2. sampler         3. decoder
  Qwen3-VL/cu128    FL2VA DiT/cu130     video + audio VAE/cu128
      |                 |                  |
 conditioning.*      latent.*              MP4
      |                 |                  |
 process exits       process exits       process exits

An optional keyframe-VAE process runs between encoder and sampler for first-frame, last-frame, or first-to-last-frame generation.

Included components

• coordinator/ — OpenAI-style synchronous video API and serial worker supervisor.
• comfyui_nodes/minimax_h3_staged/ — tensor-safe conditioning and joint audio/video latent handoffs.
• studio/ — lightweight manual generation UI and recent-render gallery.
• integrations/hermes/ — backend adapter for Hermes' native video_generate provider registry.
• examples/ — environment, systemd, and llama-swap templates.
• scripts/ — pinned runtime bootstrap, asset verification, and Hermes adapter installer.
• docs/ — installation, architecture, operations, and Hermes integration guides.
• tests/ — workflow separation, handoff integrity, Studio behavior, and provider URL tests.

Safety properties

• Serial worker execution enforced by a process and generation lock.
• Child process groups receive SIGTERM, then SIGKILL after a bounded timeout.
• CUDA_VISIBLE_DEVICES is set on every child; using a GPU UUID is recommended.
• Every stage uses --cache-none, --disable-pinned-memory, and --disable-async-offload.
• Stage-specific live MemAvailable checks fail closed before model load.
• The sampler uses a separately pinned cu130 Python environment; other stages retain the base cu128 environment.
• CUDA library paths and compiler caches are scoped to the sampler child instead of leaking across stages.
• Prompts are capped at 50,000 characters and reference images at 50 MiB by default.
• Handoffs use safetensors plus an identity/schema manifest and SHA-256 verification.
• Bundle IDs and output paths reject traversal.
• Final output must contain both video and audio streams according to ffprobe.
• The coordinator binds to loopback in the supplied service examples.

Validated software and asset pins

Component	Revision
ComfyUI	14b05228cef127ce529bc0c08660770d4af3e9a8
ComfyUI-GGUF	6ea2651e7df66d7585f6ffee804b20e92fb38b8a
INT8 ConvRot FL2VA assets	8934a2f90cb410a3b430ecce81c1791cd92377ff
Q4_K_M encoder/VAEs	d455f3ef499c708e2dc14366027a485533fe60c9
Base PyTorch family	2.11.0+cu128
Sampler PyTorch family	2.11.0+cu130
Sampler comfy-kitchen	0.2.26

The default implementation uses:

• MiniMax_H3_FL2VA_pruned_int8_convrot.safetensors
• qwen3vl_32b_minimax_h3-Q4_K_M.gguf
• minimax_h3_video_vae_fp16.safetensors
• minimax_h3_audio_vae_fp32.safetensors

The Q4_K_M encoder is an INT4-class mixed GGUF. The optional ConvRot INT4 encoder remains selectable, but operators should benchmark both on their own stack rather than assume that a nominally lower-bit file is faster.

Quick start

1. Read Installation.

2. Copy and edit the generic environment template:

   sudo install -m 0644 examples/minimax-h3.env.example /etc/minimax-h3-three-worker.env
   sudoedit /etc/minimax-h3-three-worker.env

3. Bootstrap the pinned ComfyUI runtime and separate cu130 sampler venv:

   ./scripts/bootstrap-runtime.sh

4. Download the separately licensed model files and verify them:

   set -a; source /etc/minimax-h3-three-worker.env; set +a
   "$MINIMAX_H3_PYTHON" scripts/verify-assets.py

5. Start the coordinator directly for a smoke test:

   ./scripts/run-coordinator.sh 9396
   curl -fsS http://127.0.0.1:9396/health | python3 -m json.tool

6. Submit a minimal request:

   curl -fsS http://127.0.0.1:9396/v1/videos/generations \
     -H 'content-type: application/json' \
     -d '{
       "model":"minimax-h3-fl2va",
       "prompt":"A cinematic four-second scene with natural synchronized sound. No text or watermark.",
       "size":"832x480",
       "duration":4,
       "steps":20,
       "seed":1
     }' | python3 -m json.tool

For on-demand GPU sharing, use the llama-swap example. For manual use, run the Studio service. For Hermes, follow Hermes native video generation.

The dedicated sampler environment is still part of the same three-worker service: it is not a second server or persistent GPU process. See Dedicated cu130 sampler runtime for the measured 107/192-frame results, exact quality gate, configuration, runtime verification, and one-line rollback.

API

• GET /health — assets, selected encoder, worker state, and telemetry.
• GET /capabilities — generation capabilities.
• GET /v1/models — OpenAI-style model list.
• GET /state — current or last generation state.
• POST /v1/videos/generations — JSON generation endpoint.
• POST /generate — multipart endpoint with optional first/last images.
• POST /free — terminate the active stage worker.
• GET /outputs/{path} — retrieve a generated MP4.

MiniMax H3 FL2VA produces native audio on every generation. The API intentionally has no audio-disable switch.

Resource expectations

The INT8 FL2VA DiT is intended for a 24 GB-class NVIDIA GPU. Host-memory demand varies by PyTorch, ComfyUI, driver, quantization, page cache, and offload behavior. The supplied defaults require live MemAvailable of 20 GiB for encoding, 24 GiB for sampling, and 12 GiB for decoding/keyframes. The additional cu130 venv consumes disk, not persistent GPU memory; every sampler still exits before decoding. Tune only after measuring your own peak RSS, swap, and GPU memory.

Testing

Controller-only tests:

python -m pytest tests/test_workflow.py tests/test_studio.py -q

Full tests with the pinned ComfyUI runtime and Hermes source importable:

export MINIMAX_H3_COMFY_ROOT=/path/to/runtime/ComfyUI
export PYTHONPATH=/path/to/hermes-agent
"$MINIMAX_H3_PYTHON" -m pytest -q

The full suite validates worker graph separation, stage-specific interpreter/library/cache isolation, memory flags, safetensors round trips, joint NestedTensor reconstruction, tamper rejection, path traversal rejection, telemetry reduction, MP4 workflow shape, Studio unload handling, and Hermes output URL composition.

Documentation

• Installation
• Architecture and invariants
• Operations and troubleshooting
• Dedicated cu130 sampler runtime
• Hermes native video_generate integration
• Security policy

Licensing and attribution

Repository code is MIT licensed. Model weights are not included and are governed by the MiniMax H3 Community License and the terms of their distributors. ComfyUI and ComfyUI-GGUF remain under their respective licenses. See NOTICE.