Skip to content

fix(esimd): declare INT4 output mutations#537

Open
gc-fu wants to merge 1 commit into
upgrade/vllm-xpu-v0.21.0from
fix/int4-op-mutation-schema
Open

fix(esimd): declare INT4 output mutations#537
gc-fu wants to merge 1 commit into
upgrade/vllm-xpu-v0.21.0from
fix/int4-op-mutation-schema

Conversation

@gc-fu

@gc-fu gc-fu commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • declare the preallocated INT4 GEMV output tensors mutable in the dispatcher schemas
  • use independent alias sets for multi-output and residual-mutating fused ops
  • expose these mutation-only ops as void, matching the existing QKV split compile-safe pattern
  • update the Python return annotations to match the dispatcher contract

Why

These kernels write their output arguments in place, but their schemas previously declared every tensor immutable. torch.compile functionalization could therefore treat the calls as pure, reorder them, or eliminate them when the aliased return was unused.

The affected ops are:

  • esimd_gemv_int4
  • esimd_gemv_int4_fused2
  • esimd_resadd_norm_gemv_int4_pert
  • esimd_norm_gemv_int4_pert

Validation

  • built and installed the complete custom-esimd-kernels-vllm wheel for BMG
  • verified actual mutations with torch.library.opcheck
  • all four ops pass test_schema after the change
  • all four ops pass test_faketensor with matching downstream fake implementations
  • a full-graph torch.compile INT4 GEMV smoke test is bitwise equal to eager

Performance impact

The device kernels, launch geometry, tensor shapes, and arithmetic are unchanged. Eager execution only goes through a void C++ wrapper that discards the existing aliased return. No measurable eager performance impact is expected. Compile mode may retain the dependency/copy-back operations required by the declared mutations; those are necessary for correctness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant