问题

[xiaoxiaohe233]

首先感谢你们的这份工作,具有很大的启发价值,你们的方法在论文中的评估结果特别好!

我在推理时遇到了关于参数设置的问题,想要请教一下:
我在复现的时候按照仓库给出的默认推理参数,发现无法还原出相对于deepforcing的性能提升.而且代码和论文中给出的参数不太一样?请问论文中的评估结果是按照仓库配置得到的吗?sink size在不同head下好像也是不同的,中间的key的动态rope方式也是不一样的?似乎在第一个块生成的时候三帧的rope就已经重新统一映射到了训练最远时间,而不是连续的,该怎么理解?

[winbeau]

Apologies for the late reply!
Thanks a lot for the careful read and for the detailed write-up — these are great questions. Quick answers to all three, plus a small script so you can check the RoPE behavior on your own box.

1 & 2 · Config vs. paper, and the per-head sink

Please treat the repository config as the source of truth. The arXiv post is an early (v1) write-up, and a couple of the implementation-detail numbers in §5.1 describe an earlier sweep rather than the shipped setup. The current configs/pyramid-forcing.yaml — including the per-head sink sizes you noticed — is intentional and is what we recommend for reproduction. The labels (-1 / 1 / 2) route to per-head [sink + middle + recent] compositions, so a head's sink length depending on its type is by design, not a bug. We'll reconcile the paper text with the repo in the next revision.

So: nothing to "fix" on your side for 1 & 2 — just run with the repo defaults.

3 · Dynamic RoPE on the first block

Short answer: the first block is not remapped to the far end of the training window. It stays continuous. The dynamic RoPE (window_clamp) only engages once the sequence grows past the 21-frame training window, and even then it clamps the query–key relative distance, not the absolute positions.

The mechanism (pyramidkv/rope.py):

def map_sink_time(sync_t_raw, mode="window_clamp", clamp_min=18, clamp_max=21):
    delta_t = min(max(sync_t_raw, clamp_min), clamp_max)   # clamp to [18, 21]
    return max(0, sync_t_raw - delta_t)
# sync_t <= 21  ->  returns 0           (continuous, no remap)
# sync_t  > 21  ->  returns sync_t - 21 (relative distance pinned to 21)

We measured the value actually fed to the fused C++ RoPE kernel (anchor_t_remap) across a 72-frame run, layer 0:

block	sync_t	sink t_remap	Q-K relative distance
0	0	0	0
7	21	0	21
8	24	3	21 (= 24 − 3)
15	45	24	21 (= 45 − 24)
23	69	48	21 (= 69 − 48)

Block 0's sink t_remap = 0 — continuous, not 21. The clamp only starts at block 8 (sync_t > 21), and from there the relative distance is held at exactly 21.

Reproduce it yourself (one command)

We added a self-check that hooks the C++ planner and prints the per-block t_remap with a PASS/FAIL. It forces the MegaCache path internally, so it works from a bare checkout (no extra env needed):

CUDA_VISIBLE_DEVICES=0 PYRAMIDKV_LOG_MODE=print \
uv run --no-sync python tools/verify_rope_window_clamp.py \
    --config_path configs/pyramid-forcing.yaml \
    --checkpoint_path checkpoints/self_forcing_dmd.pt \
    --data_path prompts/MovieGenVideoBench_num32.txt \
    --output_folder /tmp/rope_check --use_ema --num_output_frames 72

It runs a single prompt and stops, then prints (abridged):

 block sync_t | sink t_remap | Q-K rel dist
--------------------------------------------
            0 |            0 |            0
           21 |            0 |           21
           24 |            3 |           21
           45 |           24 |           21
           69 |           48 |           21
--------------------------------------------
[block 0 ] sink t_remap = 0  ->  continuous (== 0), NOT mapped to far end (21)
[clamp   ] for sync_t > 21: Q-K relative distance pinned to 21
RESULT   : PASS  window_clamp behaves as designed

(One detail worth flagging, since it tripped us up too: bare inference.py only takes the C++ MegaCache path when PYRAMIDKV_USE_MEGA_CACHE=1 — scripts/run_pyramid_forcing.sh sets this, the README quick-start does not. The script above forces it on so you're always checking the shipped fast path.)

If your dump shows something different, it's most likely reading t_raw (the raw frame index) or the Q-side positions rather than t_remap (the K-side value that goes into RoPE). Happy to look — just share the tensor name + block index.

One more thing on evaluation

A common source of a reproduction gap is the metric set: we report 12 VBench-Long dimensions (Table 1 + Table 6), excluding the 4 semantic / GrIT dimensions, following prior VBench-Long work. If the total is averaged over all 16, the numbers won't line up with ours. If you can share your exact eval command + which dimensions you averaged, we'll align quickly.

Thanks again — really appreciate the thorough testing. 🙏

[winbeau]

首先感谢你们的这份工作,具有很大的启发价值,你们的方法在论文中的评估结果特别好!

我在推理时遇到了关于参数设置的问题,想要请教一下: 我在复现的时候按照仓库给出的默认推理参数,发现无法还原出相对于deepforcing的性能提升.而且代码和论文中给出的参数不太一样?请问论文中的评估结果是按照仓库配置得到的吗?sink size在不同head下好像也是不同的,中间的key的动态rope方式也是不一样的?似乎在第一个块生成的时候三帧的rope就已经重新统一映射到了训练最远时间,而不是连续的,该怎么理解?

Also, could you be a bit more specific about what you mean by "couldn't reproduce the improvement over Deep Forcing"?
If possible, could you share —

• Which metric: VBench-Long total? dynamic degree? or a specific dimension?
• How many dimensions is your total averaged over? (Our paper uses 12 dimensions, excluding the 4 semantic + GrIT ones; if it's averaged over all 16, the numbers won't line up.)
• Roughly how large is the gap? And is it a metric-number issue, or visible visual quality (drift / blur / no motion)?

That would help us pinpoint it directly 👍

[xiaoxiaohe233]

首先感谢你们的这份工作,具有很大的启发价值,你们的方法在论文中的评估结果特别好!

我在推理时遇到了关于参数设置的问题,想要请教一下: 我在复现的时候按照仓库给出的默认推理参数,发现无法还原出相对于deepforcing的性能提升.而且代码和论文中给出的参数不太一样?请问论文中的评估结果是按照仓库配置得到的吗?sink size在不同head下好像也是不同的,中间的key的动态rope方式也是不一样的?似乎在第一个块生成的时候三帧的rope就已经重新统一映射到了训练最远时间,而不是连续的,该怎么理解?

Also, could you be a bit more specific about what you mean by "couldn't reproduce the improvement over Deep Forcing"? If possible, could you share —

• Which metric: VBench-Long total? dynamic degree? or a specific dimension?
• How many dimensions is your total averaged over? (Our paper uses 12 dimensions, excluding the 4 semantic + GrIT ones; if it's averaged over all 16, the numbers won't line up.)
• Roughly how large is the gap? And is it a metric-number issue, or visible visual quality (drift / blur / no motion)?

That would help us pinpoint it directly 👍

我在同一环境下保持参数不变,重复测了多次,发现每次生成结果不同,这是因为某些算子实现的导致引入了随机性吗?不过大部分的性能都比较好,在不同环境下也测试了,对baseline的vbench得分涨幅各有不同,所以对于性能结果我已经没有问题了.另外就是chunk 0 的 q在所有 heads下的时间rope是0.1.2 ,而key的时间rope在不同head下好像不太一样? osc head因为sink=1,所以只有第一帧会动态编码,但是对于其他head,sink=3,可是key的时间rope似乎是0.0.0而不是0.1.2,我是说块内部不是连续而是统一的时间吗?
对于第一个chunk的三个q而言:
sync_t_raw = 0 -> delta_t=18 -> max(0, 0-18)=0
sync_t_raw = 1 -> delta_t=18 -> max(0, 1-18)=0
sync_t_raw = 2 -> delta_t=18 -> max(0, 2-18)=0
这样子的目的是为什么呢,不应该是第一个chunk的qk时间是对齐的吗?毕竟第一个chunk的生成还没有middle,不应该和self forcing第一个chunk的生成一致吗