[vllm] Pre-quantized FP8 Qwen3.6-27B-FP8 fails on XPU: missing PlatformEnum.XPU in FP8 block kernel table (0.21.0-b1); AttributeError at construction (0.14.0-b8.3.2)

[MRdodoe]

Environment

• 2x Intel Arc Pro B60 (BMG G21, 24 GB, PCI 8086:e211), Alder Lake-S host
• Host: Ubuntu 24.04, kernel 7.0.0-28-generic (xe driver), host compute-runtime 25.18.33578.15
• Note: the two cards do not share an upstream PCIe bridge; the kernel refuses peer-to-peer DMA (xe ... cannot be used for peer-to-peer DMA as the client and provider do not share an upstream bridge or whitelisted host bridge). Workarounds used so multi-GPU works at all (may be worth documenting): FI_PROVIDER=tcp, FI_TCP_IFACE=lo (oneCCL OFI bootstrap otherwise spins forever inside Docker), CCL_TOPO_P2P_ACCESS=0, and for pipeline parallelism CCL_SEND=direct CCL_RECV=direct (otherwise zeMemOpenIpcHandle fails at the first PP send/recv).
• Model: official Qwen/Qwen3.6-27B-FP8 (fine-grained FP8, block size 128)
• Serve command (both images):
  vllm serve /llm/models/Qwen3.6-27B-FP8 -tp 2 --kv-cache-dtype auto --max-model-len 8192 --gpu-memory-utilization 0.90

Issue 1 — intel/llm-scaler-vllm:0.21.0-b1: KeyError: <PlatformEnum.XPU: 4>

Model construction reaches the FP8 linear path of the GDN linear-attention layers
(Qwen3_5DecoderLayer -> GatedDeltaNetAttention -> MergedColumnParallelLinear) and dies in
choose_scaled_mm_linear_kernel:

File ".../vllm/model_executor/kernels/linear/__init__.py", line 355, in choose_scaled_mm_linear_kernel
    for kernel in possible_kernels[current_platform._enum]:
KeyError: <PlatformEnum.XPU: 4>

_POSSIBLE_FP8_BLOCK_KERNELS (and _POSSIBLE_FP8_KERNELS) only contain CUDA, ROCM and CPU entries. Any model whose FP8 path goes through init_fp8_linear_kernel with per-group activation scales is unusable on XPU in this image.

What we tried

Locally patching the image:

1. added PlatformEnum.XPU: [TritonFp8BlockScaledMMKernel] to _POSSIBLE_FP8_BLOCK_KERNELS;
2. relaxed TritonFp8BlockScaledMMKernel.is_supported() to accept XPU.

With that, the model constructs and loads (14.5 GiB/GPU, TP=2), and the kernel is selected
(Selected TritonFp8BlockScaledMMKernel for Fp8LinearMethod). However:

• default compile mode fails during inductor autotuning:

  File ".../triton/backends/intel/compiler.py", line 385, in make_llir
      pm.run(mod, 'make_llir')
  RuntimeError: PassManager::run failed
  torch._inductor.exc.InductorError: RuntimeError: Failed to run autotuning code block: PassManager::run failed
  

• with --enforce-eager the server comes up and answers correctly, but decode collapses to
  ~1.2 tok/s (prefill ~102 tok/s), which is not usable.

Issue 2 — intel/llm-scaler-vllm:0.14.0-b8.3.2: AttributeError at construction

Same model/checkpoint, same serve command:

File ".../vllm/model_executor/models/qwen3_5.py", line 1108, in __init__
    _qkvz_sz = self.linear_attn.in_proj_qkvz.weight.shape[0]
AttributeError: 'MergedColumnParallelLinear' object has no attribute 'weight'

Happens with and without VLLM_OFFLOAD_WEIGHTS_BEFORE_QUANT=1. So the pre-quantized FP8 checkpoint is not loadable on this image either.

Issue 3 (minor) — 0.14.0-b8.3.2: dtype mismatch with runtime FP8 + bf16

The path that does work is the BF16 checkpoint (Qwen/Qwen3.6-27B) with --quantization fp8, but only with --dtype float16. With the default bf16 the warm-up crashes:

RuntimeError: expected self and mat2 to have the same dtype, but got: c10::Half != c10::BFloat16

i.e. an FP8 GEMM appears to produce fp16 regardless of model dtype. Since Qwen3.6 is trained in bf16 (and mamba_ssm_dtype: float32), having to force fp16 is not ideal.

Working configuration (for reference)

0.14.0-b8.3.2 + BF16 checkpoint + --quantization fp8 --enforce-eager --dtype float16 + the CCL env above. Measured on 2x B60 (1024 in / 256 out): TP=2 ~15 tok/s single-stream, PP=2 ~107 tok/s aggregate at 16 streams. Output quality is coherent.

Requests

1. Add an XPU entry (or an XPU-capable kernel) to the FP8 scaled-mm kernel tables so pre-quantized block-FP8 checkpoints (Qwen3.6-27B-FP8 and similar) work on XPU.
2. Fix the triton-xpu PassManager::run failed in inductor autotuning for this path, or document that eager is required.
3. Fix qwen3_5.py construction with pre-quantized FP8 on the 0.14 line.
4. Fix or document the bf16/fp16 dtype mismatch with runtime --quantization fp8.

Happy to provide full logs or run test builds on this dual-B60 host.

[gc-fu]

Hi, the pre-quantized FP8 models will be supported later.