Comparison to noisy KV-cache

[toyot-li]

Hi @Jacky-hate , thanks for your great work!

At the first glance, I think it is somehow similar to "noisy KV-cache", for example, as mentioned in https://arxiv.org/abs/2512.04677.

Would you mind elaborating the distinctions? Thanks in advance.

[toyot-li]

As an additional ablation, I am also curious about the result of $t_c = 0$ & $\mathcal{L}_{FKL}$.

[Jacky-hate]

Hi @toyot-li, thanks for your great question and your interest in our work!

The concept similar to a "noisy KV-cache" indeed exists in other previous practices. For instance, methods like Diffusion Forcing also maintain a slight amount of noise during inference to reduce degradation. However, in the context of Text-to-Video (T2V) generation, the cost of using a highly noisy KV-cache is unacceptable and was previously unsolvable..

Here is a detailed breakdown:

1. Issues with Noisy KV-Cache in T2V: When applying the noisy KV-cache (especially when adjusting to the same noise level), the visual degradation is reduced, but noticeable inter-frame jumping occurs. As shown in the 1st row of the video below, the background lacks inter-frame consistency or motion not smooth.
2. Limitations of Retraining: DMD retraining can improve continuity, but it causes a massive drop in dynamic performance (see the 3rd row of the video). Because denoising becomes more difficult under this mode, the degradation of dynamics happens far faster than the improvement in continuity. This trade-off is unavoidable and unacceptable for T2V generation tasks!
3. Why it works in Live Avatar (arXiv:2512.04677): The Audio-Driven Avatar Generation task is fundamentally different from pure T2V:
   • The presence of reference images and frame sinks inherently reduces degradation and lowers the learning difficulty.
   • The continuous audio condition acts as an ongoing "action" signal, which naturally guarantees dynamics.
     Therefore, applying just noisy KV-cache works well in audio-driven avatar tasks, but it fails in T2V generation.
4. Our Solution: To solve this, we designed a forward KL loss ($L_{FKL}$) that is highly decoupled from the causal generation capabilities. This design finally allows us to achieve long-term generation without degradation while perfectly maintaining dynamics, all without relying on auxiliary aids like frame sinks! You can see our final result in the last row of the video.

demo-vid.mp4

Regarding your question about the $t_c=0$ & $L_{FKL}$ ablation:

When the context is kept completely clean, even if we apply the Forward KL loss ($L_{FKL}$) to prevent the degradation of dynamics, the inference errors still propagate and accumulate excessively through the clean context. Consequently, the overall degradation problem remains unavoidable under this setting.

Hope this clarifies the distinctions and answers your curiosity about the ablation! Let me know if you have any further questions.

[toyot-li]

Thanks for your fast and detailed reply! Almost solved my questions.

When the context is kept completely clean, even if we apply the Forward KL loss to prevent the degradation of dynamics, the inference errors still propagate and accumulate excessively through the clean context. Consequently, the overall degradation problem remains unavoidable under this setting.

Conceptually it is. But in practice, my original motivation is to isolate the contribution of L_FKL. How much improvement is expected when applying L_FKL to a Self-Forcing baseline, for example? Thanks.

[toyot-li]

Hi @Jacky-hate I came across a contempoprary work that seems applying the same-level noise, named "neighbor forcing".

The difference is that SoulX-LiveAct first conducts diffusion-forcing-style training and then DMD