ComfyUI-MiniMaxH3-SGLang

A ComfyUI custom node that bridges MiniMax-H3 Reference-to-Video (Ref2VA) through a local SGLang server running on 2 × RTX 4090 with tensor parallelism (TP=2) and layerwise memory offload.

Unlike the official ComfyUI-native MiniMaxH3ReferenceToVideo node (which runs the model directly inside the ComfyUI process on a single 4090), this node delegates the full model inference to a separate SGLang service. This allows the dual-4090 layerwise-offload profile to fit the ~30 GB diffusion transformer alongside the 9.7 GB video VAE and 0.56 GB audio VAE.

Output: Native VIDEO with H.264 + AAC stereo audio, 1344×768, 24 fps.

---

Features

• Dynamic reference inputs — up to 9 images, 3 videos, 3 video-soundtracks, 3 standalone audio clips, matching the official MiniMaxH3ReferenceToVideo contract
• Native VIDEO output — directly connects to SaveVideo without intermediate save nodes
• SGLang bridge — communicates with a local SGLang server via its HTTP API (/v1/videos)
• Automatic media handling — encodes image/video/audio references to server-local files, constructs conditions[] payload, polls for completion, downloads the result, and muxes audio
• Layerwise offload — verified on 2 × RTX 4090 (24 GB each) with TP=2, 20 resident DiT layers, ~170–500 s per generation

---

Installation

1. Install the node

Clone this repository into your ComfyUI custom_nodes/ directory:

cd ComfyUI/custom_nodes/
git clone https://github.com/sterraMind/ComfyUI-MiniMaxH3-SGLang.git

Restart ComfyUI. The node appears in the node menu under model/conditioning/minimax as "MiniMax H3 Reference to Video (SGLang)".

2. Install and start the SGLang server

This node requires a running SGLang server on the same machine. See docs/SGLANG_INSTALL.md for the complete installation guide with 2 × RTX 4090 as the reference profile.

Quick start (after following the install guide):

bash /root/set_sglang_ref2va.sh
# Wait ~2–4 minutes for model loading
curl -s http://127.0.0.1:30010/health   # → {"status":"ok"}

---

Node Reference

MiniMax H3 Reference to Video (SGLang)

Category	model/conditioning/minimax
Display name	MiniMax H3 Reference to Video (SGLang)
Python module	custom_nodes.ComfyUI-MiniMaxH3-SGLang
Output	VIDEO — single video file

Required Inputs

Input	Type	Default	Description
prompt	STRING (multiline)	—	Text prompt. Reference images/videos/audio by tag: <Picture 1>, <Video 1>, <Audio 1> etc. in connection order
seconds	INT	5	Generation duration (4–15). Actual frame count is rounded to 17n+5 by the VAE grid
short_edge	COMBO	768	Canvas short edge in pixels. Fixed to 768 — SGLang server-side validation rejects any other value (request_validation.py: "short_edge must be 768 for minimax_h3"). Use aspect_ratio to change the canvas shape
aspect_ratio	COMBO	auto	auto, 16:9, 9:16, 1:1
num_inference_steps	INT	50	Sampling steps (5–100). 20 steps is a good quality/speed sweet spot
seed	INT	1101	Random seed. Note: H3 generation is not deterministic across calls
endpoint	STRING	http://127.0.0.1:30010	SGLang server URL. Change if running on a different host/port
timeout_seconds	FLOAT	3600	Maximum wait time for generation completion
flow_shift	FLOAT	12.0	Video flow shift (model default; usually leave unchanged)
audio_flow_shift	FLOAT	3.0	Audio flow shift (model default; usually leave unchanged)

Optional Inputs (Autogrow)

Group	Template	Max	Description
ref_images	IMAGE	9	Reference images. Connect from LoadImage
ref_videos	IMAGE (frames)	3	Reference videos as frame batches. Connect from LoadVideo or LoadVideoFrames
ref_video_audios	AUDIO	3	Paired soundtracks for reference videos (one per video)
ref_audios	AUDIO	3	Standalone reference audio clips. Must be paired with ≥1 image/video

Widget Order

The widget values in workflow JSONs must include a seed_mode entry between seed and endpoint (automatically added by ComfyUI for Int seed inputs). A correct 11-value array looks like:

[prompt, seconds, short_edge, aspect_ratio, steps, seed, "fixed", endpoint, timeout, flow_shift, audio_flow_shift]

---

Workflow Example

A minimal workflow is included at:

examples/workflows/ref2va_image.json

This workflow loads example.png from ComfyUI's input directory, connects it as the first reference image, generates a 5-second 10-step video, and saves the output via SaveVideo.

To use:

1. Open ComfyUI in your browser
2. Click Load (or drag-drop ref2va_image.json)
3. Ensure the SGLang server is running on port 30010
4. Click Queue Prompt
5. The output video appears in ComfyUI/output/video/

---

SGLang Server Dependencies

This node is a client to a separate SGLang server. The server requires:

Component	Reference Version (2 × RTX 4090)
GPU	2 × NVIDIA RTX 4090 (24 GB, sm_89)
CUDA driver	≥ 535.104
Python	3.12
PyTorch	2.11.0+cu130
SGLang	0.5.17
ffmpeg / ffprobe	any recent (mandatory)
Model	MiniMaxAI/MiniMax-H3 (ref2va partition, ~135 GB)
Optional	SageAttention 2.2.0 (auto-resolved, ~2–3× attention speedup)

See docs/SGLANG_INSTALL.md for the full installation walkthrough.

---

Architecture

┌──────────────────────────────────────────────────────────┐
│  ComfyUI (port 8188)                                     │
│  ┌──────────────┐    ┌───────────────────────────────┐   │
│  │ LoadImage     │    │ MiniMax H3 Reference to Video │   │
│  │ LoadVideo     │───▶│ (SGLang)                     │──▶ VIDEO
│  │ LoadAudio     │    │                               │   │
│  └──────────────┘    └──────────────┬────────────────┘   │
│                                     │ HTTP POST          │
│                                     │ /v1/videos         │
└─────────────────────────────────────┼────────────────────┘
                                      │
┌─────────────────────────────────────┼────────────────────┐
│  SGLang Server (port 30010, TP=2)   │                     │
│  ┌────────────────┐ ┌─────────────┐ ┌──────────────────┐ │
│  │ Text Encoder    │ │ DiT         │ │ VAE + Audio Dec  │ │
│  │ (Qwen3-VL 32B)  │ │ (Ref2VA)    │ │ (video + audio)  │ │
│  │ NVFP4-AWQ       │ │ 30.9 GB     │ │ 9.7 GB + 0.56 GB│ │
│  └────────────────┘ └─────────────┘ └──────────────────┘ │
│  Layerwise offload: 20/50 DiT layers resident in VRAM    │
│  2 × RTX 4090 · TP=2 · Ulysses=1                         │
└──────────────────────────────────────────────────────────┘

---

Troubleshooting

Problem	Likely cause
Connection refused when queueing	SGLang still loading the model (2–4 min). Wait for curl -s http://127.0.0.1:30010/health → {"status":"ok"}
DistStoreError: ... 1/2 clients joined	Instance launched with 1 GPU, but the profile is TP=2 — relaunch with --gpu-count 2
ComfyUI built-ins nodes_latent / nodes_post_processing / nodes_morphology IMPORT FAILED	flash-attn ABI mismatch with torch 2.13 — pip uninstall flash-attn (kornia degrades to SDPA)
unknown url type: '3600/v1/videos'	Node endpoint widget misconfigured — set http://127.0.0.1:30010

See docs/SGLANG_INSTALL.md §7 for the full troubleshooting table and the required pre-snapshot validation checklist.

---

License

GNU General Public License v3.0 or later. See LICENSE.

---

Related

• Official MiniMax-H3 SGLang guide
• MiniMax-H3 on HuggingFace
• ComfyUI-native MiniMaxH3ReferenceToVideo