feat: add Qwen3.8-27B vLLM Ascend deployment reference - #1
Open
MingDa23333 wants to merge 2 commits into
Open
Conversation
added 2 commits
August 17, 2026 16:23
Adds references/models/qwen3-8-27b.md generated from the official vLLM Ascend Model Tutorial (Qwen3.8-27B.html) using the repo's update_model_references.py logic, plus the corresponding model-index.md entry (8 snippets, 2 vllm serve, 2 API checks).
- Raise max-model-len to 262144 (256K), max-num-batched-tokens to 32768, max-num-seqs 16, gpu-memory-utilization 0.90 - Add missing flags: --enable-auto-tool-choice, --tool-call-parser qwen3_coder, --reasoning-parser qwen3 - Add tuned deployment snippets: 2 NPU (recommended, sufficient) and 4 NPU (verified in production)
MingDa23333
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Add the missing Qwen3.8-27B reference to the skill, generated from the official vLLM Ascend Model Tutorial, then aligned with the production-tuned parameters:
references/models/qwen3-8-27b.md— reference file:vllm servew8a8 TP2 / w8a8-mxfp8 TP1, functional verification) — kept verbatim as the authoritative starting point--tensor-parallel-size 2(2 cards sufficient),--max-model-len 262144(256K),--max-num-batched-tokens 32768,--max-num-seqs 16,--gpu-memory-utilization 0.90, plus--enable-auto-tool-choice,--tool-call-parser qwen3_coder,--reasoning-parser qwen3--tensor-parallel-size 4--enable-prefix-caching,--speculative-config(qwen3_5_mtp, 3 tokens),--compilation-config(FULL_DECODE_ONLY),--additional-config(enable_cpu_binding), envPYTORCH_NPU_ALLOC_CONF=expandable_segments:True,HCCL_BUFFSIZE=512,OMP_PROC_BIND=false,OMP_NUM_THREADS=1references/model-index.md— index row:| Qwen3.8-27B | references/models/qwen3-8-27b.md | ... | 10 | 4 | 2 |Source
https://docs.vllm.ai/projects/ascend/en/latest/tutorials/models/Qwen3.8-27B.html
Verification
Eco-Tech/Qwen3.8-27B-w8a8exist (API 200)qwen3.8-27b-2npu, vLLM Ascend v0.22.1rc1-custom, 4 NPU, 256K context) confirmed working via chat/completions and completions endpoints