ComfyUI nodes for Viggle-Animate β a 33.1 B full finetune of MiniMax-H3's ref2va transformer for character replacement in video: it takes a driving video and a reference still, and re-renders the performer(s) in the clip as the character in the still. Motion, camera, timing, background and lighting come from the video; identity comes from the image.
No text encoder, no prompt: conditioning is one frozen 362-token embedding computed once by the Viggle team with Qwen3-VL (assets/fixed_prompt.txt), identical for every render.
The sampler is DMD2-distilled and works with very low step counts. The upstream baseline is 4 sigma points: 3 Euler updates. The included workflows also contain manual sigma schedules derived from the upstream sampling formula, allowing the distilled schedule to be reproduced directly with ComfyUI's existing ManualSigmas node.
For these manual presets with Euler and BasicGuider / CFG 1.0, 4 sigma points = 3 sampling updates / model forward passes, 6 points = 5, and 8 points = 7. The final 0.0 is included in the point count. For ComfyUI and the converted/quantized models, choose 4 points for speed, 6 for balance, or 8 for quality (may over-sharpen).
Added windowed conditioning and the Viggle Chunked Sampler for longer clips, with latent carry, chunk reuse and seed overrides for another take. Added 4-, 6- and 8-point custom sigma presets, derived from the upstream shift-3 formula: fast, balanced, and quality-focused (may over-sharpen), respectively.
The node pack includes four loop nodes for disk checkpoints, external VAE decoding and live chunk progress. See the long-generation guide for wiring, recovery and a first-run checklist.
| Node | What it does |
|---|---|
| Load Text Conditioning (Viggle) | Dropdown loader for frozen text conditioning in models/text_cond/ |
| Viggle-Animate Conditioning (H3) | Builds conditioning + AV latent: video-first reference order, both references nested on the canvas short edge (the driving clip's, unless width/height are overridden) β the layout the finetune was trained with |
| Viggle-Animate Conditioning (H3, Windowed) | Splits the driving clip into overlapping windows and builds each chunk's references; outputs cond_set for the chunked sampler and guider_positive for the guider |
| Viggle Chunked Sampler | Samples each window, preserves overlap from the preceding chunk, reuses eligible cached chunks, and decodes the assembled video; outputs frames and a readable chunk_map |
| Viggle Chunk Loop Start | Creates the run folder and starts the automatically sized chunk loop; leave initial_state disconnected |
| Viggle Sample Chunk | Samples/checkpoints one chunk; outputs LATENT, loop state, save filename prefix; shows live progress |
| Viggle Chunk Loop End | Waits for chunk decoding and any connected save dependency, then advances the loop |
| Viggle Assemble Chunk Latents | Loads one saved chunk or assembles the matching chain for final VAE decoding; reports complete/partial status |
Model loading and sampling controls use ComfyUI core: Load Diffusion Model, Load LoRA (Model Only),
ModelSamplingMiniMaxH3 (video/audio shifts 3.0), BasicGuider, KSamplerSelect and ManualSigmas.
KJNodes CustomSigmas can supply the same schedules below. Viggle Chunked Sampler decodes internally; connect its frames directly to your video-saving node. Viggle Sample Chunk outputs LATENT and needs an external VAE Decode.
Download a JSON workflow or drag its PNG into ComfyUI:
| Workflow | JSON | PNG |
|---|---|---|
| Single shot (v1.2.0) | JSON | PNG |
| Chunked Sampler β memory cache, one final decode | JSON | PNG |
| Long Video Advanced β loop, disk checkpoints, external decode | JSON | PNG |
| Your goal | Use |
|---|---|
| One shot, typically 124 frames (~5.2 s at 24 fps) | Original Viggle-Animate Conditioning (H3) β core sampler β VAE Decode β save |
| Longer clip with a compact workflow and one final decode | Windowed Conditioning β Viggle Chunked Sampler; reuse is memory-only |
| Longer/expensive run, per-chunk previews, or recovery after cancellation/restart | Windowed Conditioning β Start / Sample Chunk / End β Assemble β VAE Decode; checkpoints are on disk |
Use the loop workflow when losing a run would be costly, even for two chunks. You only place one Sample Chunk node: the plan determines how often it repeats. Start with 124-frame windows, 22-frame overlap, 24 fps and the four-point sigma baseline below. Long-generation setup and troubleshooting.
| cond_vid | ref_img | output |
|---|---|---|
a49f2848-9d75-4f0c-9bf5-14d78dd20d95_s.mp4 |
 |
00000_00032-audio.mp4 |
| example_1 | example_2 | example_3 | example_4 |
|---|---|---|---|
00000_00006.mp4 |
00000_00007.mp4 |
00000_00009.1.mp4 |
00000_00013.mp4 |
| example_5 | example_6 |
|---|---|
00000_00010.mp4 |
00000_00023.1.mp4 |
| example_7 | example_8 |
|---|---|
compressO-00000_00005-audio.1.mp4 |
compressO-00000_00010-audio.mp4 |
cd ComfyUI/custom_nodes
git clone https://github.com/Saganaki22/ComfyUI-Viggle-Animate-H3Restart ComfyUI and refresh the browser after updating to load the live-progress extension.
diffusion_models (pick one β pruned is the VRAM-friendly option)
- minimax_h3_ref2va_viggle_pruned_int8_convrot.safetensors (21 GB)
- minimax_h3_ref2va_viggle_int8_convrot.safetensors (47 GB)
- minimax_h3_ref2va_viggle_bf16.safetensors (66.3 GB, max quality)
loras (DMD accelerator β pick one)
- viggle_animate_dmd_lora_r64.safetensors (0.94 GB, recommended)
- viggle_animate_dmd_lora.safetensors (3.8 GB, full rank)
text_cond
- fixed_embed_fwd_anyframe.safetensors β precomputed text conditioning, load with Load Text Conditioning (Viggle) (no text encoder needed)
vae
- minimax_h3_video_vae_int8_convrot.safetensors (3.17 GB, low VRAM)
- or minimax_h3_video_vae_fp16.safetensors (5.21 GB)
π ComfyUI/
βββ π models/
β βββ π diffusion_models/
β β βββ minimax_h3_ref2va_viggle_pruned_int8_convrot.safetensors
β βββ π loras/
β β βββ viggle_animate_dmd_lora_r64.safetensors
β βββ π text_cond/
β β βββ fixed_embed_fwd_anyframe.safetensors
β βββ π vae/
β βββ minimax_h3_video_vae_int8_convrot.safetensors
- Custom nodes required: ComfyUI-Viggle-Animate-H3 (Viggle Animate Conditioning + Load Text Conditioning) and ComfyUI-KJNodes (fast preview).
- Load Video: use
force_rate = 24. Single-shotlengthis a maximum: leave it at 124 and a 56-frame input automatically generates 56 frames. Off-grid inputs round up to the next17k+5length without discarding reference frames. For windowed conditioning, load the desired full clip (frame_load_cap = 0in VHS loads all frames). - Output resolution follows the driving video by default; set the conditioning node's
width/heightto override (each axis rounds to 32), or pre-scale the clip with Scale Image to Total Pixels. Tested canvas range: 0.4β1.2 MP β 1.2 MP holds up reliably, but the driving video and reference image then need to be high quality, not pixelated. - Choose 4 points / 3 updates for speed, 6 points / 5 updates for balance, or 8 points / 7 updates for quality (may over-sharpen). Compare results with your chosen ComfyUI model and quantization.
- Tested samplers/schedulers include
euler,er_sde,exp_heun_2_x0,lcm/simple,normal,beta, andbong_tangent, with CFG1.0and ModelSamplingMiniMaxH3 shift3.0. - The included workflows carry both ordinary scheduler configurations and the manual sigma schedules derived from the upstream formula (built in via KJNodes CustomSigmas).
- Upstream sampling baseline: use ManualSigmas with
1.0, 0.8571428571428571, 0.6, 0.0, euler, BasicGuider (or CFG 1.0), and the originalviggle_animate_dmd_lora.safetensorsat strength 1.0. Connect ManualSigmas to SamplerCustomAdvanced's or Viggle Chunked Sampler'ssigmasinput. These are four sigma points and three model evaluations, matching upstream's β4 steps.β Keep ModelSamplingMiniMaxH3 at video 3.0, audio 3.0; it does not shift the supplied ManualSigmas tensor again. Do not add a separate sigma-transform node afterward. - Other samplers/schedulers and the rank-64 adapter are experimental alternatives. The bundled example's LCM/bong_tangent eight-step settings differ from the upstream baseline.
- KJNodes CustomSigmas: for the four values above, set
interpolate_to_stepsto 3. Setting it to 4 interpolates throughlog(0)and produces a schedule ending in0, 0; Euler returns NaNs, which turn the final video black and contaminate subsequent chunks. The chunked sampler rejects this invalid schedule before rendering. - Stacks with Comfy Kitchen and block sparse attention patches.
Paste one list into ManualSigmas or KJNodes CustomSigmas, then connect its SIGMAS output to the sampler. These presets use sigma = 3*t / (1 + 2*t) with evenly spaced t from 1 to 0. The four-point preset matches the upstream baseline; the six- and eight-point presets extend the same pattern.
| Suggested workflow title | Sigma points, including final zero | Sampling updates / KJNodes interpolate_to_steps |
|---|---|---|
| Viggle DMD β 3 Steps (Upstream β4-Stepβ) | 4 | 3 |
| Viggle β 5 Steps (6 Sigma Points) | 6 | 5 |
| Viggle β 7 Steps (8 Sigma Points) | 8 | 7 |
4 points / 3 Euler updates β fast:
1.0, 0.8571428571428571, 0.6, 0.0
6 points / 5 Euler updates β balanced:
1.0, 0.9230769230769231, 0.8181818181818182, 0.6666666666666666, 0.42857142857142855, 0.0
8 points / 7 Euler updates β quality (may over-sharpen):
1.0, 0.9473684210526315, 0.8823529411764706, 0.8, 0.6923076923076923, 0.5454545454545454, 0.3333333333333333, 0.0
Keep Euler, BasicGuider / CFG 1.0, and model shifts 3.0 / 3.0. The final 0.0 is required: it is the destination of the last update, not another model evaluation. Do not append another zero or shift these lists again. Choose 4 points for speed, 6 for balance, or 8 for quality (may over-sharpen). These are practical ComfyUI preset choices; results depend on the clip and model/quantization. Encoding and final decoding still take time regardless of the preset.
Example workflow (Windowed Conditioning + Chunked Sampler): example_workflows/viggle-animate-h3_workflow-chunked-sampler.json.
Connect Viggle-Animate Conditioning (H3, Windowed) to Viggle Chunked Sampler. Its guider_positive output supplies BasicGuider's conditioning (or CFGGuider's positive). Start with 124-frame chunks and 22-frame overlap. Prior output is preserved in each overlap, and the assembled latent is decoded once. Motion and appearance can still change at joins; use a repainted reference frame from the driving shot and keep the input/output at 24 fps.
Windowed conditioning reuses complete 17-frame encoder blocks from the preceding window when using the standard ComfyUI H3 VAE. Each window's padded tail is still encoded separately. This reduces repeated VAE work without reducing resolution, changing precision or enlarging the encoding window; custom VAE wrappers retain the full-window path. The log reports how many blocks were reused. Higher resolution and longer clips still cost more to encode.
| Control | Meaning |
|---|---|
seed |
Base sampling seed: chunk 1 uses seed, chunk 2 uses seed + 1, and so on. Both Viggle samplers generate standard noise internally; remove old noise connections when updating a workflow. |
rerender_chunk |
1-based chunk whose seed you want to override. 0 disables the override. Choose a chunk number shown in chunk_map. |
rerender_seed |
Replacement seed for the selected chunk only. It has no effect when rerender_chunk = 0; zero itself is a valid seed. |
These controls let you try another take from a troublesome chunk without changing the base seed for the whole clip. For example, with base seed = 58, four chunks normally use 58, 59, 60, 61. Set rerender_chunk = 2 and rerender_seed = 123 to use 58, 123, 60, 61. Chunk 1 can be reused; chunks 2β4 must be regenerated when that override changes, because each receives content from its predecessor. Their original numeric seeds do not make later chunks independent of the changed carry.
Keep the same override to retain that take, or change rerender_seed for another. This is a seed override, not a force-refresh button: unchanged settings can reuse cached results. Setting rerender_chunk back to 0 restores the base-seed sequence. The chunk_map output shows frame ranges, seeds, overlap and [cached] / [rendered] labels.
- You cannot change one chunk and keep all later chunks fixed: the overlap is carried forward. Existing overlap from the preceding chunk stays pinned, so rerendering a chunk does not repaint its inherited beginning.
- Reuse is an in-memory optimization, not a saved checkpoint or resume system. Restarting ComfyUI clears it. Cache eviction, changed inputs/model/sampling settings, or oversized entries can require earlier chunks to render again.
- Stock guider objects with inspectable sampler/model settings support reuse. Opaque custom options, callbacks or patches bypass caching and still sample normally; patched workflows may rerender every chunk.
- Chunk caching preserves output precision and is limited to 2 GiB of CPU tensor storage. Encoded references have a separate 256 MiB / 64-entry limit. Oversized entries are not cached.
- The assembled latent is decoded again after sampling, even when earlier chunks are reused. Full-clip conditioning, the master latent, final decoding and output frames still need memory; chunking does not make arbitrarily long clips fit in RAM/VRAM.
- Motion, identity and lighting can still change at joins; overlap does not guarantee seamless or stutter-free video.
chunk_framesis the maximum window length: the last window can be shorter while keeping the normal overlap. All loaded reference frames are used; off-grid lengths generate up to 16 extra frames to reach the next17k+5boundary (minimum 5). Output frames are sampled, not appended copies.
With 362 input frames, chunk_frames = 124 and overlap_frames = 22, the windows
are 0β123, 102β225, 204β327, 306β361; the final window renders 56 frames.
A 361-frame input also targets 362 generated frames rather than dropping to 345.
The H3 VAE still applies its own internal temporal padding when encoding references;
this does not duplicate the rendered output. Frame counts refer to the actual images
received from the loader, which may differ from source-video metadata after FPS conversion.
- Generated audio is discarded. Connect the driving clip's audio to the video-saving node and match it to the retained video length; use 24 fps for input and output.
- Invalid sigma schedules and NaN/Inf chunk latents now stop with an actionable error before corrupt output is cached or carried into later chunks.
Full node-by-node breakdown (sockets, slot order, resume rules, typical session): docs/long_video_guide.md. Tested example workflow: example_workflows/viggle-animate-h3_workflow-long-video-advanced.json.
Four nodes turn the same windowed conditioning into a graph-expanded loop with disk checkpoints, so each chunk is decoded and saved through your own nodes while it is produced β no VAE input on the sampler, and a failure mid-run keeps every completed chunk:
| Node | Purpose |
|---|---|
| Viggle Chunk Loop Start | Reads the plan from cond_set, picks the checkpoint directory, initializes the loop state. |
| Viggle Sample Chunk | Samples the current window only. Outputs the chunk's video LATENT for an ordinary VAE Decode, plus the carried state. Checkpoints the latent to disk before returning. |
| Viggle Chunk Loop End | Waits for this iteration's decode/save branch, then either expands the next chunk or returns the finished collection. |
| Viggle Assemble Chunk Latents | Stitches the saved chunks (trimming overlap) into one LATENT for a single final decode. chunk_number > 0 loads one chunk for inspection; interrupted runs assemble partially. |
Loop Start β loop ββββββββββββββββββββββββββββββββ
β state β Sample Chunk β LATENT β VAE Decode ββ¬β Loop End (images)
ββ Video Combine β filenames β (after_save)
- The decode/save branch must feed Loop End β connect VAE Decode's images to
imagesand Video Combine'sfilenamesoutput toafter_save, so a chunk cannot start before the previous one is decoded and saved. (Core SaveWEBM also works: itsimagesoutput goes straight intoimages.) - Checkpoints land in
output/viggle_chunks/<run_name>/as safetensors plus amanifest.json; writes are atomic, so a crash never leaves a half-valid chunk. - With
resumeon, re-running the queue restores every chunk whose graph, models, conditioning, sigmas and per-chunk seed still match (the checkpoint filename embeds that fingerprint). Changed settings sample new files under new names; old takes stay on disk.rerender_chunk/rerender_seedwork as in the single-pass sampler. - In the Long Video Advanced example, change Sample Chunk's seed control from
randomizetofixedbefore testing resume. Enablesave_outputon both chunk and final Video Combine nodes to keep the videos; the example defaults to temporary previews. - Decoding each chunk separately means each preview contains overlap context; run the collection through Viggle Assemble Chunk Latents β one final VAE Decode for the finished video.
- A decode/save failure leaves that chunk's latent checkpoint on disk too. Re-queue with the same
run_nameandresumeenabled to retry previews without resampling matching chunks. Replaying previews preserves an already completed manifest if a preview fails. - Sample Chunk generates standard noise internally from its seed; it has no noise input. Remove the old noise connection when updating a workflow. Linked model filenames are checked conservatively using file metadata across their model category; changing another file there can also invalidate reuse. Code updates invalidate automatic resume; old latent files remain readable.
- Decode/save completes before the next chunk samples. ComfyUI's execution cache may retain decoded chunks in RAM (about 0.82 GiB per 124-frame 1024Γ576 float32 RGB chunk). Enable Video Combine's
save_outputfor durable preview videos; latent checkpoints are saved independently.
Live loop progress: Sample Chunk's read-only live_progress box updates for
every chunk, including restored chunks. No extra node or wiring is needed. Restart
ComfyUI and refresh the browser after updating. βLoop completedβ excludes any
downstream final assembly/decode/save; Start, End and Assemble retain their STRING status outputs.
- Identity drift on re-entry: when the subject leaves the camera view and re-enters, the re-entry settles toward the driving video's original appearance rather than the reference image. The same applies when the subject moves far from the reference pose or makes abrupt large motions (e.g. a backflip) β the further from the still, the weaker the identity hold.
- Lip-sync limitations: the generated subject does not reliably lip-sync to the conditioning video.
- Reference image compatibility: if you cannot get the reference-image character to appear correctly in the output video (identity drift), make the reference image match the pose/stance of the person in the conditioning video as closely as possible (same background). Keep the gen between 0.4β0.6 megapixels and use LCM or normal sampling with 6β8 steps. Higher resolutions have been tested reliably up to 1.2 MP, but the driving video and reference image must then be high quality β pixelated sources will show in the output.
- Original model + inference code: huggingface.co/Viggle/Viggle-Animate Β· viggle.ai
- Base model: huggingface.co/MiniMaxAI/MiniMax-H3
- Comfy-repackaged base VAEs: huggingface.co/Comfy-Org/MiniMax-H3
- Converted ComfyUI weights, quants & LoRAs: huggingface.co/drbaph/Viggle-Animate-ComfyUI
If you use the model in published work, cite the original:
@misc{viggle2026animate,
title = {Viggle-Animate: Character Replacement in Video from a Single Repainted Frame},
author = {Viggle Research},
year = {2026},
url = {https://huggingface.co/Viggle/Viggle-Animate}
}- The weights are a Model Derivative of MiniMax H3 β the MiniMax H3 Community License Agreement applies to them (read it before redistributing or shipping a product on them). This includes the converted/quantized variants linked above.
- This node pack is Apache 2.0 (see
LICENSE). - The model puts a person into footage they did not shoot; identity comes from the image you supply. Do not run it on people who have not consented, and label what you generate as AI-generated (see the original repo's intended-use section).
- issues: ComfyUI-Viggle-Animate-H3/issues
Viggle Chunked Sampler also has a live_progress display for sampling, cache hits and final decoding. Its chunk_map output is retained; reuse is memory-only.