Summary
Serving a GPTQ INT4-quantized hybrid model (Gated DeltaNet linear attention + MoE) crashes on the first forward pass. _gdn_outproj_esimd_eligible() in gdn_linear_attn.py accesses self.out_proj.weight unconditionally, but for a quantized RowParallelLinear the parameter is named qweight. The model loads and the server starts successfully; the crash occurs only when the first inference request arrives.
Environment
Image: intel/llm-scaler-vllm:0.21.0-b1
vLLM version: 0.21.1.dev0+gad7125a43.d20260709
GPU: Intel Arc Pro B70
Host: Unraid, Docker, --device=/dev/dri, privileged
Model: deepreinforce-ai/Ornith-1.0-35B quantized to GPTQ INT4 (W4, group_size 128, sym) via GPTQModel
Resolved architecture: Qwen3_5MoeForConditionalGeneration (hybrid: GDN linear attention layers + 256-expert MoE)
Reproduction
vllm serve /models/Ornith-1.0-35B-GPTQ-Int4
--dtype float16
--quantization gptq
--enforce-eager
--trust-remote-code
--max-model-len 11200
--block-size 64
--gpu-memory-utilization 0.95
--reasoning-parser qwen3
--enable-auto-tool-choice --tool-call-parser qwen3_xml
--host 0.0.0.0 --port 2999
Server starts normally. Weights load (19.11 GiB), KV cache allocates, Application startup complete. Then send any chat completion request → EngineCore dies.
Traceback
File "vllm/model_executor/models/qwen3_next.py", line 1176, in forward
self.linear_attn.forward_xpu(hidden_states, self_attention_output)
File "vllm/model_executor/layers/mamba/gdn_linear_attn.py", line 1100, in forward_xpu
self._gdn_core_and_output(
File "vllm/model_executor/layers/mamba/gdn_linear_attn.py", line 1193, in _gdn_core_and_output
if self._gdn_outproj_esimd_eligible(num_tokens):
File "vllm/model_executor/layers/mamba/gdn_linear_attn.py", line 1298, in _gdn_outproj_esimd_eligible
w = self.out_proj.weight
File "torch/nn/modules/module.py", line 1968, in getattr
raise AttributeError(
AttributeError: 'RowParallelLinear' object has no attribute 'weight'. Did you mean: 'qweight'?
Root cause
_gdn_outproj_esimd_eligible() at gdn_linear_attn.py:1298 assumes out_proj holds an unquantized weight tensor. When the checkpoint is GPTQ-quantized, the GDN projections are quantized along with the rest of the model.
Confirmed by inspecting the checkpoint — the GDN block contains a mix of quantized and unquantized tensors:
model.language_model.layers.0.linear_attn.out_proj.qweight ← quantized
model.language_model.layers.0.linear_attn.out_proj.scales
model.language_model.layers.0.linear_attn.out_proj.qzeros
model.language_model.layers.0.linear_attn.out_proj.g_idx
model.language_model.layers.0.linear_attn.in_proj_qkv.qweight ← quantized
model.language_model.layers.0.linear_attn.in_proj_z.qweight ← quantized
model.language_model.layers.0.linear_attn.in_proj_a.weight ← not quantized
model.language_model.layers.0.linear_attn.in_proj_b.weight ← not quantized
model.language_model.layers.0.linear_attn.conv1d.weight ← not quantized
So out_proj has no .weight attribute at all, and the eligibility check raises before any kernel selection can happen. There is no serving flag that avoids this path.
Suggested fix
Guard the attribute access and fall back to the generic (non-ESIMD) path when the layer is quantized:
pythondef _gdn_outproj_esimd_eligible(self, num_tokens: int) -> bool:
w = getattr(self.out_proj, "weight", None)
if w is None:
return False # quantized layer — ESIMD out_proj path not applicable
...
It would also be worth auditing the rest of gdn_linear_attn.py (and the surrounding XPU paths) for other unconditional .weight accesses, since the same assumption may hold elsewhere and quantized hybrid models would hit those next.
Impact
Any GPTQ/INT4-quantized model using the GDN linear attention path is currently unservable on XPU. This affects the Qwen3.5-MoE hybrid family, which is exactly the class of model where INT4 matters most — a 35B MoE fits comfortably in a single 32 GB Arc card at INT4 (19 GiB weights, 9.3 GiB KV cache observed) but cannot be served.
Summary
Serving a GPTQ INT4-quantized hybrid model (Gated DeltaNet linear attention + MoE) crashes on the first forward pass. _gdn_outproj_esimd_eligible() in gdn_linear_attn.py accesses self.out_proj.weight unconditionally, but for a quantized RowParallelLinear the parameter is named qweight. The model loads and the server starts successfully; the crash occurs only when the first inference request arrives.
Environment
Image: intel/llm-scaler-vllm:0.21.0-b1
vLLM version: 0.21.1.dev0+gad7125a43.d20260709
GPU: Intel Arc Pro B70
Host: Unraid, Docker, --device=/dev/dri, privileged
Model: deepreinforce-ai/Ornith-1.0-35B quantized to GPTQ INT4 (W4, group_size 128, sym) via GPTQModel
Resolved architecture: Qwen3_5MoeForConditionalGeneration (hybrid: GDN linear attention layers + 256-expert MoE)
Reproduction
vllm serve /models/Ornith-1.0-35B-GPTQ-Int4
--dtype float16
--quantization gptq
--enforce-eager
--trust-remote-code
--max-model-len 11200
--block-size 64
--gpu-memory-utilization 0.95
--reasoning-parser qwen3
--enable-auto-tool-choice --tool-call-parser qwen3_xml
--host 0.0.0.0 --port 2999
Server starts normally. Weights load (19.11 GiB), KV cache allocates, Application startup complete. Then send any chat completion request → EngineCore dies.
Traceback
File "vllm/model_executor/models/qwen3_next.py", line 1176, in forward
self.linear_attn.forward_xpu(hidden_states, self_attention_output)
File "vllm/model_executor/layers/mamba/gdn_linear_attn.py", line 1100, in forward_xpu
self._gdn_core_and_output(
File "vllm/model_executor/layers/mamba/gdn_linear_attn.py", line 1193, in _gdn_core_and_output
if self._gdn_outproj_esimd_eligible(num_tokens):
File "vllm/model_executor/layers/mamba/gdn_linear_attn.py", line 1298, in _gdn_outproj_esimd_eligible
w = self.out_proj.weight
File "torch/nn/modules/module.py", line 1968, in getattr
raise AttributeError(
AttributeError: 'RowParallelLinear' object has no attribute 'weight'. Did you mean: 'qweight'?
Root cause
_gdn_outproj_esimd_eligible() at gdn_linear_attn.py:1298 assumes out_proj holds an unquantized weight tensor. When the checkpoint is GPTQ-quantized, the GDN projections are quantized along with the rest of the model.
Confirmed by inspecting the checkpoint — the GDN block contains a mix of quantized and unquantized tensors:
model.language_model.layers.0.linear_attn.out_proj.qweight ← quantized
model.language_model.layers.0.linear_attn.out_proj.scales
model.language_model.layers.0.linear_attn.out_proj.qzeros
model.language_model.layers.0.linear_attn.out_proj.g_idx
model.language_model.layers.0.linear_attn.in_proj_qkv.qweight ← quantized
model.language_model.layers.0.linear_attn.in_proj_z.qweight ← quantized
model.language_model.layers.0.linear_attn.in_proj_a.weight ← not quantized
model.language_model.layers.0.linear_attn.in_proj_b.weight ← not quantized
model.language_model.layers.0.linear_attn.conv1d.weight ← not quantized
So out_proj has no .weight attribute at all, and the eligibility check raises before any kernel selection can happen. There is no serving flag that avoids this path.
Suggested fix
Guard the attribute access and fall back to the generic (non-ESIMD) path when the layer is quantized:
pythondef _gdn_outproj_esimd_eligible(self, num_tokens: int) -> bool:
w = getattr(self.out_proj, "weight", None)
if w is None:
return False # quantized layer — ESIMD out_proj path not applicable
...
It would also be worth auditing the rest of gdn_linear_attn.py (and the surrounding XPU paths) for other unconditional .weight accesses, since the same assumption may hold elsewhere and quantized hybrid models would hit those next.
Impact
Any GPTQ/INT4-quantized model using the GDN linear attention path is currently unservable on XPU. This affects the Qwen3.5-MoE hybrid family, which is exactly the class of model where INT4 matters most — a 35B MoE fits comfortably in a single 32 GB Arc card at INT4 (19 GiB weights, 9.3 GiB KV cache observed) but cannot be served.