Cherry-pick PR #24: Fix pad_len issue for t2v and ti2v sp case#25

Draft

Wei-Lin-Intel with Copilot wants to merge 48 commits into

mainfrom

copilot/add-hpu-fp8-linear-gemm-support-again

Copilot AI commented May 13, 2026

Summary

Cherry-picks the changes from PR #24 to the copilot/add-hpu-fp8-linear-gemm-support branch.

Changes

This cherry-picks commit 7453f71 which fixes the pad_len issue for t2v and ti2v sequence parallel cases. The original pad_len was added in commit edf8b3d for s2v support, but it broke the SP case for t2v and ti2v. This patch fixes the issue.

Original PR

PR: Fix pad_len issue for t2v and ti2v sp case #24
Author: @mengker33

mengker33 and others added 30 commits

May 13, 2026 05:56


          Wan: Add hpu support for t2v

ddd6205

Replace some cuda specific ops to hpu;
Apply FusedSDPA;
Modify rope to fit hpu because hpu does not support complex type;
Add mark_step to reduce graph compile time.

Co-authored-by: Jincheng Miao <jincheng.miao@intel.com>

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Enable S2v

74e2626

* S2V functional ready on gaudi

* Improve rope performance

* Remove float() to avoid fav3 issue


Signed-off-by: Chen, Wenbin <wenbin.chen@intel.com>
Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Wan: Apply FAV3 in t2v

24953f3

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Wan: Enable SP for t2v

69e97ff

Apply FAV3 in sp attention;
Modify rope for sp case.

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Wan: Improve rope

73f2855

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Add HPU support for wan-animiate animation mode

d7ca822

Signed-off-by: Jincheng Miao <jincheng.miao@intel.com>

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          minor fix

25b6abc

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          fix rope issue for wan-animate

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Add sam2 HPU repo

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          fix wan-animate device

f4aa45e

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Move reduant ops in pre_loop

ba41e30

Signed-off-by: Chen, Wenbin <wenbin.chen@intel.com>

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Add fused rope to improve performance

3c66031

Signed-off-by: Chen, Wenbin <wenbin.chen@intel.com>

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          use flashAttentionV3

b2f080d

Signed-off-by: Chen, Wenbin <wenbin.chen@intel.com>

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Enable sp

ec32920

Signed-off-by: Chen, Wenbin <wenbin.chen@intel.com>

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Set correct device for i2v anf ti2v

9d2abd9

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Use torch roll to optimize for-loop to reduce GC time

9a996ca

Signed-off-by: Chen, Wenbin <wenbin.chen@intel.com>

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Pad freqs in sp case

5a0fe9d

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Refact block fwd to ensure bf16

535a917

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Fix accuracy mismatch between sp and non-sp case

32a7e45

1. Ensure rms norm calculation is under float32;
2. Add several mark_step to align tensor values between sp and non-sp case;

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Change generator device of i2v in ti2v to cpu

fec1fe3

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Do not permute if head dim is already before seq dim in FusedSDPA case

d57e5b7

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Add mark_step to reduce compilation time for VAE

2ab13ea

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Update requirements.txt

b881e38

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Reduce GC time for all pipelines

964844c

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Fix Peft version to 0.17.0

b84d3fc

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Add mark_step in vae2_1 to reduce compile time

f5d0660

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Use barrier only in the distributed case

7ed1fe6

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Apply fav3 to wan animate

eaaf7f9

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Enable SP for wan animate

7249cbe

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Fix wan animate accuracy issue

079df02

This patch includes two fix:
1. Move data to cpu for torch.linalg.qr as hpu generates sign-opposite
   result which causes incorrect face motion;
2. Use fp32 for motion feat matmul.

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>

mengker33 and others added 18 commits

May 13, 2026 05:56


          Use cpu generator in wan animate

2e44d03

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Update sam2 in requirements_animate

bbcd334

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Explicitly import habana torch in sam2_utils preprocess

46d00ad

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Add wan animate web ui demo patches

84614da

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Add wan web ui demo readme

eb1b9c5

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Update requirements_animate.txt

a88bfec

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Wan Animate: Fix sp shape mismatch issue

3f2a7dd

1. Add pad to cos/sin when there is padding;
2. Add the second padding to ensure the seq len is divisible by both
sp_size and T dim;
3. Pass pad_len to fav3 in sp case.

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          WanAnimate: Improve motion encoder to boost perf

cfe25fb

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          WanAnimate: Use sam2 cpu version and modify the preprocess accordingly

b9e4347

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          WanAnimate: Add 480p to animate config

12b6bc9

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          WanAnimate: Remove padding in face encoder

5635fed

In face encoder, seq_len will be divided by T dim, so we need to
remove sp padding to make seq_len divisible by T, then after face
attention, we add sp padding back to attn output.

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Fix animate web ui demo conflicts

21f7c12

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          WanAnimate: Add mark_step after diag op to avoid acc issue

70c7d3f

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Initial plan

f0fb9e9

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Add HPU FP8 Linear GEMM support for Wan inference models

5615fee

- Add wan/utils/fp8_linear.py with WanFP8Linear, dynamic_quant,
  apply_fp8_linear_hpu, and wrap_blocks_linear_fp8 utilities
- Add --fp8 CLI flag to generate.py
- Update WanI2V, WanT2V, WanTI2V, WanS2V __init__ and _configure_model
  to accept fp8 parameter and wrap blocks Linear layers at init time

Agent-Logs-Url: https://github.com/wenbinc-Bin/Wan2.2/sessions/eadb65ce-e852-465b-a87e-f8dc391a1df7

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Address code review: extract SCALE_EPSILON constant, improve comments

2cbdc66

Agent-Logs-Url: https://github.com/wenbinc-Bin/Wan2.2/sessions/eadb65ce-e852-465b-a87e-f8dc391a1df7

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Update fp8_linear.py to fix FP8 GEMM scale format error

c62b914

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>


          Fix pad_len issue for t2v and ti2v sp case

c8bcb51

In commit edf8b3d, pad_len is added for s2v, but it breaks the sp case
for t2v and ti2v, this patch will fix the issue.

Co-authored-by: Wei-Lin-Intel <63534993+Wei-Lin-Intel@users.noreply.github.com>

Copilot AI assigned Copilot and Wei-Lin-Intel

Copilot created this pull request from a session on behalf of Wei-Lin-Intel

May 13, 2026 05:58

View session

Copilot stopped work on behalf of Wei-Lin-Intel due to an error

May 13, 2026 05:58

The session was cancelled by the user.

Copilot AI requested a review from Wei-Lin-Intel

May 13, 2026 05:58

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet