Summary
Running vllm serve with --pipeline-parallel-size 2 across two Intel Battlemage GPUs (Arc Pro B70 32GB + Arc B580 12GB) successfully loads the model and can serve at least one request, but the engine reliably becomes unstable shortly after — manifesting as either a hard UR_RESULT_ERROR_DEVICE_LOST crash or a silent shm_broadcast stall that times out and kills the engine (RPC call to sample_tokens timed out). The failure occurs during live token sampling / cross-worker communication, not during model loading or memory allocation.
Environment
CPU: Intel Core Ultra 9 285H
System RAM: 96GB
GPU 0: Intel Arc Pro B70 (Battlemage, BMG-G31, 32GB VRAM)
GPU 1: Intel Arc B580 (Battlemage, BMG-G21, 12GB VRAM)
iGPU: Intel Arc Pro 130T/140T (integrated, Arrow Lake-P) — present on the system but not exposed to torch.xpu in this container
Host OS: Unraid (Docker)
Container: Custom-built image (my-vllm-xpu:latest), vLLM version 0.23.1rc1.dev70+ga52205bcc
Driver/runtime: Intel xe kernel driver, oneAPI Level-Zero (Intel(R) oneAPI Unified Runtime over Level-Zero V2), driver_version 1.14.36300+8
Model: Qwen/Qwen3.6-27B-FP8 (64 hidden layers, hybrid Gated-DeltaNet/full-attention architecture)
Configuration Used
vllm serve /models/Qwen3.6-27B-FP8
--served-model-name Qwen3.6-27B
--pipeline-parallel-size 2
--tensor-parallel-size 1
--distributed-executor-backend mp
--dtype float16
--gpu-memory-utilization 0.90
--max-model-len 8192
--kv-cache-dtype fp8
--enforce-eager
--trust-remote-code
--port 5555
--host 0.0.0.0
Environment variables:
VLLM_WORKER_MULTIPROC_METHOD=spawn
VLLM_PP_LAYER_PARTITION=60,4
SYCL_UR_USE_LEVEL_ZERO_V2=0
Docker device passthrough:
--ipc=host --privileged --shm-size=10g --device /dev/dri:/dev/dri -v /dev/dri/by-path:/dev/dri/by-path
GPU-to-device mapping confirmed via torch.xpu.get_device_properties():
torch.xpu index 0 → Arc Pro B70 (31023MB reported)
torch.xpu index 1 → Arc B580 (11605MB reported)
VLLM_PP_LAYER_PARTITION=60,4 was chosen deliberately to bias the larger B70 toward holding most of the 64 transformer layers, since the two GPUs have very different VRAM capacity (32GB vs 12GB) and vLLM's default PP layer split does not appear to be VRAM-aware.
Steps to Reproduce
Start the container with the configuration above.
Engine initializes successfully:
Both workers load their assigned weight shards without error.
Available KV cache memory: 0.58 GiB is reported (positive — confirms successful memory budgeting after switching to FP8 KV cache; an earlier attempt without --kv-cache-dtype fp8 reported Available KV cache memory: -1.02 GiB and failed at startup with ValueError: No available memory for the cache blocks).
GPU KV cache size: 10,922 tokens is reported.
Server starts and responds to GET /v1/models.
Send a POST /v1/chat/completions request. First request succeeds (200 OK, real generated tokens, confirmed via response body).
Send a second POST /v1/chat/completions request (a short prompt, max_tokens: 200).
Engine hangs. Logs show repeated:
INFO [shm_broadcast.py:705] No available shared memory broadcast block found in 60 seconds. This typically happens when some processes are hanging or doing some time-consuming work (e.g. compilation, weight/kv cache quantization).
four times in a row (4 minutes), followed by a fatal timeout:
RuntimeError: Worker failed with error 'level_zero backend failed with error: 20 (UR_RESULT_ERROR_DEVICE_LOST)', please check the stack trace above for the root cause
or, on a later attempt with the same configuration:
TimeoutError: RPC call to sample_tokens timed out.
vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue.
The client request that was in-flight returns 500 Internal Server Error. The entire engine is now dead and the container must be restarted; the API server itself stays up momentarily but all subsequent requests fail with the same EngineDeadError.
Re-tested from a clean restart with a single, minimal request ("hello", max_tokens: 20) to rule out request complexity or back-to-back timing as the cause. Same failure signature occurred on this single, simple request — ruling out "only fails on the second/heavier request" as the trigger. The failure is not reliably reproducible on the very first request after a restart, but is reliably reproducible within one or two requests of any kind.
Expected Behavior
The engine should continue serving subsequent requests reliably after a successful first request, given that GPU memory was already confirmed sufficient (Available KV cache memory positive) and the model loaded and ran correctly once.
Actual Behavior
The engine becomes unrecoverable after very few requests (often the very next one), failing either with a hard UR_RESULT_ERROR_DEVICE_LOST Level-Zero error or a shm_broadcast/RPC timeout that cascades into the same dead-engine state. Both failure modes occur specifically around the cross-worker communication / sampling step (sample_tokens RPC, determine_available_memory RPC in earlier variants), not during model loading or initial memory profiling.
Analysis / Why We Believe This Is a PP Cross-Device Communication Issue, Not a Memory or Config Problem
Memory was independently confirmed sufficient at startup (positive Available KV cache memory, both GPUs showed stable, non-overflowing memory usage in nvtop throughout — B70 at ~27GB/32GB, B580 at ~10–12GB/12GB).
The failure happens after a successful generation, mid-session, not at load time — ruling out a static OOM-at-startup explanation.
The error surfaces specifically inside vllm/distributed/device_communicators/shm_broadcast.py (acquire_read → TimeoutError) and the xccl distributed backend used for PP worker coordination — i.e., the failure is in the inter-process/inter-GPU communication layer itself, not in any single worker's local computation.
We separately confirmed (via lspci, /dev/dri/by-path, and torch.xpu.get_device_properties()) that device identification, PCI bus mapping, and render-node assignment were all correct and as expected — ruling out a device-misidentification explanation.
We attempted the following mitigations, none of which resolved the issue:
SYCL_UR_USE_LEVEL_ZERO_V2=0 (forcing the older non-V2 Unified Runtime Level-Zero backend) — no change, identical failure signature.
Reducing --max-model-len (16384 → 8192) and increasing --gpu-memory-utilization (0.80 → 0.90) — resolved a separate, earlier startup-time OOM issue (Available KV cache memory: -1.02 GiB), but did not affect this later in-session hang/crash.
Adding --kv-cache-dtype fp8 — also helped resolve the startup OOM, not the in-session crash.
--enforce-eager was set throughout all PP attempts (to avoid compounding instability with CUDA/SYCL graph capture) and did not prevent the crash.
Suspected Related Upstream Issue
We believe this may be the same underlying class of bug as other reports of Battlemage multi-GPU collective instability under vLLM, including reports of GP faults and xe BCS engine resets during TP/PP collective operations on matched pairs of Arc Pro B70 GPUs (i.e., this is not unique to mixed-capacity GPU pairs). If confirmed to be the same root cause, this suggests the issue lies in the Level-Zero / oneCCL collective communication layer for Battlemage rather than anything specific to asymmetric GPU configurations.
Additional Notes / Open Questions
We have not reproduced this on Intel's official llm-scaler-vllm image — all testing was done on a custom-built my-vllm-xpu:latest image, and we cannot rule out that the custom image's specific dependency versions (IPEX / oneCCL / Level-Zero runtime pinning) contribute to or are unrelated to this failure. We'd welcome guidance on whether this is reproducible on the official LLM-Scaler container.
We have not yet tested whether the same failure occurs with --tensor-parallel-size 2 instead of pipeline parallelism, nor whether it occurs on two identical Battlemage GPUs (we only have access to one B70 + one B580).
We have not tried the CCL_ATL_TRANSPORT, CCL_ZE_IPC_EXCHANGE, or ZE_FLAT_DEVICE_HIERARCHY environment variable combinations sometimes cited in similar reports, beyond SYCL_UR_USE_LEVEL_ZERO_V2=0.
Happy to provide full logs, nvtop captures, or run additional diagnostics on request — this hardware combination (B70 + B580) is unusual and may be a useful data point even if the root cause turns out to be the same as matched-pair B70 reports.
Severity / Impact
This makes pipeline parallelism unusable for sustained inference on this hardware combination — the engine can demonstrate that it can produce correct output once, but cannot be relied upon for more than one or two requests before requiring a manual container restart. For anyone trying to pool VRAM across multiple Battlemage GPUs to run models too large for a single card, this is currently a hard blocker to real usage, even though the underlying approach (uneven PP layer partitioning via VLLM_PP_LAYER_PARTITION) works exactly as documented for the memory-allocation side of the problem.
Summary
Running vllm serve with --pipeline-parallel-size 2 across two Intel Battlemage GPUs (Arc Pro B70 32GB + Arc B580 12GB) successfully loads the model and can serve at least one request, but the engine reliably becomes unstable shortly after — manifesting as either a hard UR_RESULT_ERROR_DEVICE_LOST crash or a silent shm_broadcast stall that times out and kills the engine (RPC call to sample_tokens timed out). The failure occurs during live token sampling / cross-worker communication, not during model loading or memory allocation.
Environment
CPU: Intel Core Ultra 9 285H
System RAM: 96GB
GPU 0: Intel Arc Pro B70 (Battlemage, BMG-G31, 32GB VRAM)
GPU 1: Intel Arc B580 (Battlemage, BMG-G21, 12GB VRAM)
iGPU: Intel Arc Pro 130T/140T (integrated, Arrow Lake-P) — present on the system but not exposed to torch.xpu in this container
Host OS: Unraid (Docker)
Container: Custom-built image (my-vllm-xpu:latest), vLLM version 0.23.1rc1.dev70+ga52205bcc
Driver/runtime: Intel xe kernel driver, oneAPI Level-Zero (Intel(R) oneAPI Unified Runtime over Level-Zero V2), driver_version 1.14.36300+8
Model: Qwen/Qwen3.6-27B-FP8 (64 hidden layers, hybrid Gated-DeltaNet/full-attention architecture)
Configuration Used
vllm serve /models/Qwen3.6-27B-FP8
--served-model-name Qwen3.6-27B
--pipeline-parallel-size 2
--tensor-parallel-size 1
--distributed-executor-backend mp
--dtype float16
--gpu-memory-utilization 0.90
--max-model-len 8192
--kv-cache-dtype fp8
--enforce-eager
--trust-remote-code
--port 5555
--host 0.0.0.0
Environment variables:
VLLM_WORKER_MULTIPROC_METHOD=spawn
VLLM_PP_LAYER_PARTITION=60,4
SYCL_UR_USE_LEVEL_ZERO_V2=0
Docker device passthrough:
--ipc=host --privileged --shm-size=10g --device /dev/dri:/dev/dri -v /dev/dri/by-path:/dev/dri/by-path
GPU-to-device mapping confirmed via torch.xpu.get_device_properties():
torch.xpu index 0 → Arc Pro B70 (31023MB reported)
torch.xpu index 1 → Arc B580 (11605MB reported)
VLLM_PP_LAYER_PARTITION=60,4 was chosen deliberately to bias the larger B70 toward holding most of the 64 transformer layers, since the two GPUs have very different VRAM capacity (32GB vs 12GB) and vLLM's default PP layer split does not appear to be VRAM-aware.
Steps to Reproduce
Start the container with the configuration above.
Engine initializes successfully:
Both workers load their assigned weight shards without error.
Available KV cache memory: 0.58 GiB is reported (positive — confirms successful memory budgeting after switching to FP8 KV cache; an earlier attempt without --kv-cache-dtype fp8 reported Available KV cache memory: -1.02 GiB and failed at startup with ValueError: No available memory for the cache blocks).
GPU KV cache size: 10,922 tokens is reported.
Server starts and responds to GET /v1/models.
Send a POST /v1/chat/completions request. First request succeeds (200 OK, real generated tokens, confirmed via response body).
Send a second POST /v1/chat/completions request (a short prompt, max_tokens: 200).
Engine hangs. Logs show repeated:
INFO [shm_broadcast.py:705] No available shared memory broadcast block found in 60 seconds. This typically happens when some processes are hanging or doing some time-consuming work (e.g. compilation, weight/kv cache quantization).
four times in a row (4 minutes), followed by a fatal timeout:
RuntimeError: Worker failed with error 'level_zero backend failed with error: 20 (UR_RESULT_ERROR_DEVICE_LOST)', please check the stack trace above for the root cause
or, on a later attempt with the same configuration:
TimeoutError: RPC call to sample_tokens timed out.
vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue.
The client request that was in-flight returns 500 Internal Server Error. The entire engine is now dead and the container must be restarted; the API server itself stays up momentarily but all subsequent requests fail with the same EngineDeadError.
Re-tested from a clean restart with a single, minimal request ("hello", max_tokens: 20) to rule out request complexity or back-to-back timing as the cause. Same failure signature occurred on this single, simple request — ruling out "only fails on the second/heavier request" as the trigger. The failure is not reliably reproducible on the very first request after a restart, but is reliably reproducible within one or two requests of any kind.
Expected Behavior
The engine should continue serving subsequent requests reliably after a successful first request, given that GPU memory was already confirmed sufficient (Available KV cache memory positive) and the model loaded and ran correctly once.
Actual Behavior
The engine becomes unrecoverable after very few requests (often the very next one), failing either with a hard UR_RESULT_ERROR_DEVICE_LOST Level-Zero error or a shm_broadcast/RPC timeout that cascades into the same dead-engine state. Both failure modes occur specifically around the cross-worker communication / sampling step (sample_tokens RPC, determine_available_memory RPC in earlier variants), not during model loading or initial memory profiling.
Analysis / Why We Believe This Is a PP Cross-Device Communication Issue, Not a Memory or Config Problem
Memory was independently confirmed sufficient at startup (positive Available KV cache memory, both GPUs showed stable, non-overflowing memory usage in nvtop throughout — B70 at ~27GB/32GB, B580 at ~10–12GB/12GB).
The failure happens after a successful generation, mid-session, not at load time — ruling out a static OOM-at-startup explanation.
The error surfaces specifically inside vllm/distributed/device_communicators/shm_broadcast.py (acquire_read → TimeoutError) and the xccl distributed backend used for PP worker coordination — i.e., the failure is in the inter-process/inter-GPU communication layer itself, not in any single worker's local computation.
We separately confirmed (via lspci, /dev/dri/by-path, and torch.xpu.get_device_properties()) that device identification, PCI bus mapping, and render-node assignment were all correct and as expected — ruling out a device-misidentification explanation.
We attempted the following mitigations, none of which resolved the issue:
SYCL_UR_USE_LEVEL_ZERO_V2=0 (forcing the older non-V2 Unified Runtime Level-Zero backend) — no change, identical failure signature.
Reducing --max-model-len (16384 → 8192) and increasing --gpu-memory-utilization (0.80 → 0.90) — resolved a separate, earlier startup-time OOM issue (Available KV cache memory: -1.02 GiB), but did not affect this later in-session hang/crash.
Adding --kv-cache-dtype fp8 — also helped resolve the startup OOM, not the in-session crash.
--enforce-eager was set throughout all PP attempts (to avoid compounding instability with CUDA/SYCL graph capture) and did not prevent the crash.
Suspected Related Upstream Issue
We believe this may be the same underlying class of bug as other reports of Battlemage multi-GPU collective instability under vLLM, including reports of GP faults and xe BCS engine resets during TP/PP collective operations on matched pairs of Arc Pro B70 GPUs (i.e., this is not unique to mixed-capacity GPU pairs). If confirmed to be the same root cause, this suggests the issue lies in the Level-Zero / oneCCL collective communication layer for Battlemage rather than anything specific to asymmetric GPU configurations.
Additional Notes / Open Questions
We have not reproduced this on Intel's official llm-scaler-vllm image — all testing was done on a custom-built my-vllm-xpu:latest image, and we cannot rule out that the custom image's specific dependency versions (IPEX / oneCCL / Level-Zero runtime pinning) contribute to or are unrelated to this failure. We'd welcome guidance on whether this is reproducible on the official LLM-Scaler container.
We have not yet tested whether the same failure occurs with --tensor-parallel-size 2 instead of pipeline parallelism, nor whether it occurs on two identical Battlemage GPUs (we only have access to one B70 + one B580).
We have not tried the CCL_ATL_TRANSPORT, CCL_ZE_IPC_EXCHANGE, or ZE_FLAT_DEVICE_HIERARCHY environment variable combinations sometimes cited in similar reports, beyond SYCL_UR_USE_LEVEL_ZERO_V2=0.
Happy to provide full logs, nvtop captures, or run additional diagnostics on request — this hardware combination (B70 + B580) is unusual and may be a useful data point even if the root cause turns out to be the same as matched-pair B70 reports.
Severity / Impact
This makes pipeline parallelism unusable for sustained inference on this hardware combination — the engine can demonstrate that it can produce correct output once, but cannot be relied upon for more than one or two requests before requiring a manual container restart. For anyone trying to pool VRAM across multiple Battlemage GPUs to run models too large for a single card, this is currently a hard blocker to real usage, even though the underlying approach (uneven PP layer partitioning via VLLM_PP_LAYER_PARTITION) works exactly as documented for the memory-allocation side of the problem.