[XPU] Add & optimize silu_and_mul_with_clamp activation kernel

[yangulei]

Summary

Adds the silu_and_mul_with_clamp XPU activation kernel (SwiGLU-OAI with input
clamping, used by MiniMax-M3 MoE) and a bandwidth-optimized SYCL implementation.

Semantics (matches the CUDA reference and torch.clamp-based native path):

gate = input[:, :d];  up = input[:, d:]      # contiguous halves
gate_c = min(gate, limit)
up_c   = clamp(up, -limit, limit)
silu   = gate_c / (1 + exp(-alpha * gate_c))
out    = silu * (up_c + beta)                # alpha=1, beta=0 -> silu(gate)*up

Implementation

• silu_and_mul_with_clamp_vec_kernel<T, ACT_FN, VEC> — the fast path used
  when d % VEC == 0 (VEC = 16/sizeof(T), i.e. 8×bf16/half, 4×float):
  • 16-byte aligned vector loads/stores via vllm::xpu::aligned_vec, so each
    work-item moves a full cache-line-friendly chunk instead of scalar 2-byte
    elements.
  • Flat grid-stride dispatch over all (token, chunk) pairs (WG=256, grid
    capped at 2560 workgroups) so GPU occupancy is independent of the token
    count — small-batch shapes no longer launch just a handful of workgroups.
• The original scalar per-token kernel is retained as the fallback for
  d % VEC != 0, preserving correctness for arbitrary d.

Performance (Intel Arc Pro B60 / BMG, bf16)

Profile-guided optimization on the MiniMax-M3 shape matrix. Effective bandwidth
of the fused op (read 2·d + write d):

Shape (num_tokens × d)	Before	After	Speedup
16384 × 768 (dominant, TritonExperts)	262 GB/s	390 GB/s	+49%
4096 × 768	293	757	+158%
2048 × 768	348	790	+127%

The dominant shape reaches 85.7% of the 456 GB/s theoretical peak and
99.5% of the empirical streaming roofline (a trivial same-traffic copy tops
out at 393 GB/s), i.e. it is at the memory-bandwidth wall. ISA analysis
confirmed the remaining ALU (clamp cmp/sel, sigmoid exp) is fully hidden
behind memory latency, so no further ISA-level tuning helps.

Versus the PyTorch-native (torch.clamp/sigmoid/mul) path, the fused kernel
is ~8.4× faster (geomean) across the shape matrix and also more accurate
(fp32 intermediates, single cast: err 0.125 vs 0.24 against an fp32 golden).

Tests

tests/test_silu_and_mul_with_clamp.py validates the op against the native
reference across {half, bfloat16, float}, multiple token counts, and both
d divisible by the vector width (vectorized path) and d = 17 (scalar
fallback). Plus opcheck for schema/meta.

54 passed

[Copilot]

Pull request overview

Adds a new XPU fused activation op, silu_and_mul_with_clamp (SwiGLU-OAI-style with clamping), including a bandwidth-optimized vectorized SYCL implementation, PyTorch dispatcher registration, and Python-side tests/wrappers to validate correctness and schema behavior.

Changes:

• Implement silu_and_mul_with_clamp in csrc/activation.cpp, including a vectorized grid-stride fast path and a scalar fallback.
• Register the new op in csrc/torch_bindings.cpp and expose its C++ signature in csrc/ops.h.
• Add a dedicated test (tests/test_silu_and_mul_with_clamp.py) plus test wrappers (tests/register_ops.py, tests/ops/silu_and_mul_with_clamp_op.py).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
csrc/activation.cpp	Adds the scalar + vectorized SYCL kernels and launch logic for silu_and_mul_with_clamp.
csrc/ops.h	Declares the new op entry point for compilation/linking.
csrc/torch_bindings.cpp	Registers the op schema and XPU implementation with the PyTorch dispatcher.
tests/register_ops.py	Adds a Python wrapper to invoke the new registered op for tests.
tests/ops/silu_and_mul_with_clamp_op.py	Adds a CustomOp test wrapper and a native reference implementation.
tests/test_silu_and_mul_with_clamp.py	Adds correctness + opcheck coverage across dtypes, shapes, and scalar/vector paths.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.