Skip to content

[XPU] support HND layout#47975

Merged
jikunshang merged 3 commits into
vllm-project:mainfrom
zhenwei-intel:xpu_hnd
Jul 17, 2026
Merged

[XPU] support HND layout#47975
jikunshang merged 3 commits into
vllm-project:mainfrom
zhenwei-intel:xpu_hnd

Conversation

@zhenwei-intel

@zhenwei-intel zhenwei-intel commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Purpose

with PR #44455 and vllm-project/vllm-xpu-kernels#432

Now we can support HND layout in XPU, this unblocks PD disaggregation with heterogeneous TP.

Test Plan

VLLM_KV_CACHE_LAYOUT=HND ZE_AFFINITY_MASK=4 python examples/basic/offline_inference/generate.py --model Qwen/Qwen3-0.6B --enforce-eager

# PD disaggregation with heterogeneous TP
PREFILLER_TP_SIZE=2 DECODER_TP_SIZE=1 bash tests/v1/kv_connector/nixl_integration/run_xpu_disagg_accuracy_test.sh

Test Result

  • acc (limit 100) and perf test (1k/1k) of qwen3-0.6B qwen3-30b-a3b
--- Performance Results ---
Model                          Layout      req/s  out_tok/s    ttft_ms    tpot_ms
--------------------------------------------------------------------------------
qwen3-0.6b                     HND          0.96     985.43     197.74      14.39
qwen3-0.6b                     NHD          0.96     983.80     192.10      14.41
qwen3-30b-a3b                  HND          0.19     193.33     600.87      73.26
qwen3-30b-a3b                  NHD          0.20     199.79     598.49      70.91

--- Accuracy Results ---
Model                          Layout   Task       Filter             Metric          Value   Stderr
----------------------------------------------------------------------------------------------------
qwen3-0.6b                     HND      gsm8k      strict-match       exact_match      0.44   0.0499
qwen3-0.6b                     NHD      gsm8k      strict-match       exact_match      0.44   0.0499
qwen3-30b-a3b                  HND      gsm8k      strict-match       exact_match      0.98   0.0141
qwen3-30b-a3b                  NHD      gsm8k      strict-match       exact_match      0.98   0.0141

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
@zhenwei-intel
zhenwei-intel marked this pull request as ready for review July 17, 2026 04:59

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@zhenwei-intel

Copy link
Copy Markdown
Contributor Author

cc @zufangzhu @jikunshang @yma11 PTAL.

This reverts commit f1bfaf0.

Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
@jikunshang jikunshang added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 17, 2026
@jikunshang
jikunshang enabled auto-merge (squash) July 17, 2026 10:09
@jikunshang
jikunshang merged commit 8502958 into vllm-project:main Jul 17, 2026
81 checks passed
plasticchris pushed a commit to plasticchris/vllm that referenced this pull request Jul 20, 2026
Signed-off-by: zhenwei-intel <zhenwei.liu@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/build intel-gpu Related to Intel GPU ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants