Skip to content

Commit a3b3c1f

Browse files
Docs: fold the quality notes into BRIDGES as neutral reference, drop GUIDE.md
Keyframes, H3 prompts from the timeline and depth are documented in BRIDGES.md; launch-flag advice is hardware-neutral.
1 parent 675217f commit a3b3c1f

8 files changed

Lines changed: 60 additions & 137 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ Difforum becomes a direction layer: a visual timeline drives any renderer.
2727
- **Look Mix**: detail transfer, colour, flicker cuts or crossfade from a feedback pass onto
2828
an H3 / LTX render. New looks: disco_diffusion, vqgan_clip, flicker_experimental.
2929
- Template **12 · long shot with key moments** (30 s, installations).
30-
- **Hover help on every input** (one registry, `nodes/tooltips.py`) and a quality guide (`docs/GUIDE.md`).
30+
- **Hover help on every input** (one registry, `nodes/tooltips.py`, also in docs/NODES.md).
3131
- **H3 structured prompts**: Camera → Prompt and H3 Shot write the whole Director timeline (look,
3232
scenes, camera in H3 vocabulary, key events, soundscape, music) in MiniMax H3's native format,
3333
with the I2VA / FL2VA alignment lines or the full-reference sections.
@@ -54,8 +54,8 @@ Difforum becomes a direction layer: a visual timeline drives any renderer.
5454
- One shared feedback engine behind the Feedback Sampler, Live Sampler and Storyboard.
5555
- Depth follows the image in 3D, and pseudo-3D replaces the silent freeze without depth.
5656
- Steps scale with the energy, like Deforum (about 2x faster); the run report shows
57-
seconds per frame and warns when ComfyUI was launched with `--lowvram` /
58-
`--disable-smart-memory`, which reload the model every frame.
57+
seconds per frame and warns when low-VRAM launch flags make ComfyUI reload the model
58+
every frame.
5959
- Cadence crossfades between keys. Revealed areas are repainted with extra noise.
6060
- Colour anchoring per scene. Lazy prompt travel. Ring buffer in the Live Sampler.
6161
- Deterministic 3D z-buffer on CUDA / MPS / CPU.

‎README.md‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -155,8 +155,7 @@ Deforum hard to use:
155155

156156
| | |
157157
|---|---|
158-
| [Quality guide](docs/GUIDE.md) | The dials that matter, keyframe quality, depth, AE cameras, H3 prompts |
159-
| [Bridges](docs/BRIDGES.md) | MiniMax H3, LTX-2, After Effects, Blender |
158+
| [Bridges](docs/BRIDGES.md) | MiniMax H3, LTX-2, keyframes, H3 prompts, depth, After Effects, Blender |
160159
| [Node reference](docs/NODES.md) | Every input and output |
161160
| [Performance](docs/PERFORMANCE.md) | Speed vs. quality, Apple Silicon |
162161
| [Models](docs/MODELS.md) | What works well, and the settings |

‎docs/BRIDGES.md‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -168,3 +168,51 @@ one picture per key (or takes times such as `0, 4s, 9.5s`) and outputs
168168
The **Animatic** is the previz of the whole shot (low resolution, a few seconds
169169
even for a minute of footage). Press **Previz only** on the Director to mute every
170170
render output, queue, check, then press it again to render.
171+
172+
## Keyframes for video models
173+
174+
The guide keyframes are what H3 / LTX copy: their sharpness, palette and texture.
175+
176+
**How many.** The Director's **Keys** track marks exact moments (with Keyframe
177+
Images). **Keyframes** samples a clip every `every_seconds` (or at explicit
178+
`indices` such as `0, 48, 96, -1`) on the model's latent grid. **H3 Guides**
179+
keeps up to `max_guides` of them: first, last, and the best spread in time. More
180+
guides follow the frames closely; fewer leave more to the model.
181+
182+
**Quality.** Guide Frames warps one image along the camera, so keyframes soften as
183+
the camera travels and the uncovered areas are empty. The H3 templates run three
184+
switchable blocks before H3 Guides:
185+
186+
| block | what it does |
187+
|---|---|
188+
| **Depth (Depth Anything 3)** | core depth estimation on the anchor, so 3d moves have real parallax |
189+
| **Fill Reveal (AI)** | an image model paints the uncovered areas (inpainting checkpoints seam best) |
190+
| **Keyframe Polish** | Restyle `clean restyle` at denoise ~0.35 re-paints each keyframe: detail back, composition kept |
191+
192+
Consistent keyframes (same model, prompt and seed) interpolate better than a mix of
193+
styles. Hand-made stills on the Keys track beat warped frames for important moments.
194+
195+
## H3 prompts from the timeline
196+
197+
Camera → Prompt (`format = H3 structured`) and H3 Shot (`prompt_style = H3
198+
structured`) write the Director timeline in MiniMax H3's native prompt format:
199+
200+
| timeline | prompt |
201+
|---|---|
202+
| look | style that opens `[Shot 1]` |
203+
| scenes | what is on screen; a transformation in one take, or a new `[Shot N]` at its cut time with `cuts` on |
204+
| camera blocks | H3 camera vocabulary (push in, truck, arc, pedestal...) with amplitude and speed |
205+
| key labels | the event at that moment |
206+
| `soundscape` / `music` | `overall_soundscape` / `non_diegetic_music` |
207+
208+
`h3_mode` adds the I2VA or FL2VA alignment line, or writes the full-reference
209+
sections for ref2va with H3 Guides. Scene prompts and key labels work best as
210+
things that can be seen or heard.
211+
212+
## Depth
213+
214+
Set the Director to **3d** and connect a depth map (white = near) to Guide Frames
215+
or the Feedback Sampler. Templates 03, 08 and 10 use the core Depth Anything 3
216+
nodes (`depth_anything_3_mono_large` in `models/geometry_estimation`, `v2_style`
217+
render). The Feedback Sampler also outputs the tracked depth per frame for
218+
compositing. Lower `translation_scale` if the parallax is too strong.

‎docs/GUIDE.md‎

Lines changed: 0 additions & 122 deletions
This file was deleted.

‎docs/PERFORMANCE.md‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -20,11 +20,10 @@ Only the first row is worth optimising. The rest is noise.
2020

2121
## Before anything: launch flags and step scaling
2222

23-
- **Launch flags.** `--lowvram`, `--novram` and `--disable-smart-memory` (useful for
24-
MiniMax H3 / LTX on 24 GB) make ComfyUI re-stage the image model for *every*
25-
frame of a feedback render. SDXL fits easily on a 24 GB card: start ComfyUI
26-
without these flags for Feedback / Live renders. The run report warns when they
27-
are on.
23+
- **Launch flags.** `--lowvram`, `--novram` and `--disable-smart-memory` make
24+
ComfyUI re-stage the image model for *every* frame of a feedback render. If the
25+
image model fits in VRAM, run Feedback / Live renders without them. The run
26+
report warns when they are on.
2827
- **Steps scale with the energy** (`step_scaling = by energy`, the default): a
2928
frame at denoise 0.5 runs 10 of 20 steps, as in Deforum. `fixed` runs all steps.
3029

‎example_workflows/06_seamless_loop.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@
5656
"Node name for S&R": "MarkdownNote"
5757
},
5858
"widgets_values": [
59-
"## Loop without a crossfade\n\nThe Camera path is made periodic over 120 frames and rendered for 3 laps; the feedback settles onto its cycle and **Loop** keeps the last lap. For projections that run for hours.\n\n**Speed:** cadence 2 and steps x energy (10 of 20 at 0.5). Faster: a DMD2 / Lightning LoRA (steps 4-6, cfg 1-2) or `long_edge` 512. Launch ComfyUI without `--lowvram` / `--disable-smart-memory` for feedback renders: they reload the model every frame."
59+
"## Loop without a crossfade\n\nThe Camera path is made periodic over 120 frames and rendered for 3 laps; the feedback settles onto its cycle and **Loop** keeps the last lap. For projections that run for hours.\n\n**Speed:** cadence 2 and steps x energy (10 of 20 at 0.5). Faster: a DMD2 / Lightning LoRA (steps 4-6, cfg 1-2) or `long_edge` 512. Low-VRAM launch flags (`--lowvram`, `--disable-smart-memory`) reload the model every frame."
6060
]
6161
},
6262
{

‎nodes/render.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ def launch_warning() -> str:
115115
if not bad:
116116
return ""
117117
msg = (f"launched with {' '.join(bad)}: the image model is re-loaded for every frame. "
118-
"For Feedback / Live renders start ComfyUI without these flags (keep them for H3 / LTX).")
118+
"If the image model fits in VRAM, run Feedback / Live renders without these flags.")
119119
if not _WARNED:
120120
_WARNED.append(1)
121121
print(f"[Difforum] {msg}")

‎tools/build_workflows.py‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -609,9 +609,8 @@ def wf_loop():
609609
control(g, "## Loop without a crossfade\n\nThe Camera path is made periodic over 120 frames and "
610610
"rendered for 3 laps; the feedback settles onto its cycle and **Loop** keeps the last lap. "
611611
"For projections that run for hours.\n\n**Speed:** cadence 2 and steps x energy (10 of 20 at "
612-
"0.5). Faster: a DMD2 / Lightning LoRA (steps 4-6, cfg 1-2) or `long_edge` 512. Launch ComfyUI "
613-
"without `--lowvram` / `--disable-smart-memory` for feedback renders: they reload the model "
614-
"every frame.")
612+
"0.5). Faster: a DMD2 / Lightning LoRA (steps 4-6, cfg 1-2) or `long_edge` 512. Low-VRAM "
613+
"launch flags (`--lowvram`, `--disable-smart-memory`) reload the model every frame.")
615614
with g.block(B_DIRECT, C_DIRECT, col=1, row=0):
616615
s = g.add("Difforum_Setup", size=(320, 300), duration_mode="frames", duration=360.0)
617616
cam = g.add("Difforum_Camera", size=(380, 360), keys="0: orbit_right 1.0 0.8\n60: spiral 1.2 1.0",

0 commit comments

Comments
 (0)