Why is the time sampling changed from LogitNorm(0.7, 1.6) in sCM to LogitNorm(-0.8, 1.6) in TF-sCM?

[wesar1]

Thanks for the great work and for open-sourcing it!

I noticed that in the rCM setup, the time sampling for sCM uses LogitNorm(0.7, 1.6), which is biased toward the medium-to-high noise region. However, in Causal-rCM, the time sampling for TF-sCM is changed to LogitNorm(-0.8, 1.6), which is biased toward the medium-to-low noise region.

Could you explain why this change was made?

[zhengkw18]

Thanks for the question and great observation!

Intuitively, this is because in the causal case, the supervision is imposed on each chunk instead of the whole video. The chunk is composed of only 1 or 3 frames, so the data dimension is significantly smaller. As revealed in simple diffusion, higher dimension data (e.g., larger resolution) requires higher relative noise levels to corrupt the information, so the training time is shifted to noisier regions.