Skip to content

feat: add MiniMax M3 + M2.7 support (Kunlun XPU) - #396

Open
NaphJohn wants to merge 3 commits into
baidu:v0.21.0-devfrom
NaphJohn:feat/minimax-m3-m27-support
Open

feat: add MiniMax M3 + M2.7 support (Kunlun XPU)#396
NaphJohn wants to merge 3 commits into
baidu:v0.21.0-devfrom
NaphJohn:feat/minimax-m3-m27-support

Conversation

@NaphJohn

Copy link
Copy Markdown
Contributor

Merge MiniMax M3 (ops branch) and M2.7 increment onto v0.21.0-dev.

M3: MSA prefill/decode attention, fused MoE + custom ops (minimax_m3_moe_gate / moe_swiglu_quant / moe_pre_sorted_quant / moe_ffn_pre_sorted_scale / minimax_m3_fused_norm_quant), prequantized linear forward, EAGLE3 spec decode, full+piecewise graph, rope fixes.

M2.7: fused MoE gate + QK RMSNorm bf16, thinking modes, eagle3 adapt.

19 new files (model defs, reasoning/tool parsers, spec-decode, indexer); 13 existing files 3-way merged. Requires kunlun_ops >= 0.1.205 (new symbols: matmul w_trans, swiglu_bias, moe_fused_gate, moe_sigmoid_group_topk_norm reordered args, qkrmsnorm).

Change-Id: I3c44a1c1dfd1f8c8fada86b06057fed901b188d3

PR Description

FIX #xxxx


Checklist (Required)

Before submitting this PR, please ensure that all the following items are completed:

  • All code changes pass the pre-commit checks.
  • Commits are signed off using git commit -s.
  • The PR title is properly classified (see below).

PR Type

Please prefix the PR title with one or more of the following labels to help reviewers quickly understand the nature of the change:

  • [Feature] – New features or enhancements (e.g. Attention, Communicator, Kernel, Worker, etc.)
  • [Bugfix] – Bug fixes
  • [CI/Build] – CI, build system, or infrastructure improvements
  • [Doc] – Documentation updates or fixes
  • [Misc] – Other changes that do not fit the above categories (use sparingly)

Note: If the PR spans multiple categories, include all relevant prefixes.


Detailed Checklist (Click to Expand)

Thank you for contributing to vLLM Kunlun! To help us maintain high code quality and streamline the review process, please ensure your PR meets the following requirements.

1. Code Quality

  • All linting and formatting checks pass (pre-commit).
  • The code is well-structured and sufficiently documented.
  • The change is designed with maintainability and readability in mind.

2. Testing

  • Relevant unit tests are added or updated.
  • Integration tests are included when applicable.
  • Existing tests continue to pass.

3. DCO Compliance

This project follows the Developer Certificate of Origin (DCO).

  • All commits include a Signed-off-by: line.
  • Use git commit -s to automatically add the sign-off.

4. Review Expectations

During the review process, maintainers may:

  • Request code refactoring or additional tests.
  • Ask for clarifications on design decisions.
  • Suggest performance, stability, or maintainability improvements.

We appreciate your patience and collaboration throughout the review process!

wanghangkai added 2 commits July 24, 2026 14:43
Merge MiniMax M3 (ops branch) and M2.7 increment onto v0.21.0-dev.

M3: MSA prefill/decode attention, fused MoE + custom ops
(minimax_m3_moe_gate / moe_swiglu_quant / moe_pre_sorted_quant /
moe_ffn_pre_sorted_scale / minimax_m3_fused_norm_quant), prequantized
linear forward, EAGLE3 spec decode, full+piecewise graph, rope fixes.

M2.7: fused MoE gate + QK RMSNorm bf16, thinking modes, eagle3 adapt.

19 new files (model defs, reasoning/tool parsers, spec-decode, indexer);
13 existing files 3-way merged. Requires kunlun_ops >= 0.1.205 (new
symbols: matmul w_trans, swiglu_bias, moe_fused_gate,
moe_sigmoid_group_topk_norm reordered args, qkrmsnorm).

Change-Id: I3c44a1c1dfd1f8c8fada86b06057fed901b188d3
__init__.py imports preload_xpu at module load and calls
preload_xpu_libraries() to dlopen XPU shared libs (xcudart shim,
xpuapi, bkcl, etc.). The file predates the feature base so the
diff-overlay integration dropped it, breaking 'import vllm_kunlun'.
Verified: M2.7 W8A8 serves and generates on P800 TP=8 with this fix.

Change-Id: I67f3f4a3767443ee2c202064d9b9a3f7e3491b3d

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR integrates MiniMax M3 + M2.7 support into the Kunlun XPU fork of vLLM, including EAGLE3 speculative decoding utilities, MiniMax model/parsing components, and a set of Kunlun-specific kernel/dispatch patches (MoE, attention, RoPE, sampling, structured output) intended to preserve CUDA-like call conventions while routing execution to kunlun_ops where appropriate.

Changes:

  • Add/merge MiniMax M2/M3 model support (model definition, reasoning parser, tool-call parser, EAGLE3 drafter updates).
  • Introduce EAGLE/EAGLE3 speculative decoding helpers plus fused Triton/Kunlun control kernels and slot-mapping utilities.
  • Extend Kunlun OOT plugin runtime patching for KV-cache binding, attention backends, quantized linear/MoE paths, structured-output masking, and XPU dependency preloading.

Reviewed changes

Copilot reviewed 31 out of 33 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
vllm/v1/worker/gpu/spec_decode/eagle/eagle3_utils.py Adds helpers to configure EAGLE3 auxiliary hidden-state layers from model/config.
vllm/v1/spec_decode/utils.py Adds EAGLE step/prepare Triton kernels and shared spec-decode utilities.
vllm/v1/sample/ops/topk_topp_sampler.py Adds a unified sampler module with CUDA/CPU/XPU/ROCm fast paths and fallbacks.
vllm/v1/sample/ops/logprobs.py Adds a compiled helper for batched threshold counting used by logprob/sampling flows.
vllm/utils/init.py Adds shared utility helpers and compile_unless_eager platform guard logic for Kunlun.
vllm/tool_parsers/minimax_m3_tool_parser.py Adds MiniMax M3 namespace-delimited tool-call parser with schema-aware conversion.
vllm/reasoning/minimax_m2_reasoning_parser.py Adds MiniMax M2 thinking-mode reasoning parser (enabled/disabled/adaptive).
vllm/model_executor/models/minimax_m2.py Adds the MiniMax M2 inference-only model with Kunlun-specific MoE gate fast path and EAGLE3 support.
vllm/model_executor/models/llama_eagle3.py Adds/adjusts EAGLE3 Llama draft model support, aux-state fusion, and weight-loading behavior.
vllm/model_executor/layers/mamba/linear_attn.py Optimizes MiniMaxText01 RMSNorm QK path and integrates with linear attention flow.
vllm_kunlun/v1/worker/utils.py Patches KVBlockZeroer and adds Kunlun-specific KV-cache binding for multi-cache-per-layer indexing.
vllm_kunlun/v1/structured_output/utils.py Switches structured-output grammar masking to Kunlun native op with xgrammar fallback.
vllm_kunlun/v1/spec_decode/eagle.py Provides Kunlun kernel adapters to replace Triton EAGLE control kernels.
vllm_kunlun/v1/spec_decode/init.py Adds Kunlun speculative decoding package marker/init.
vllm_kunlun/v1/sample/rejection_sampler.py Replaces rejection sampler Triton kernels with Kunlun ops while keeping upstream API semantics.
vllm_kunlun/v1/attention/backends/minimax_m3_indexer.py Adds a key-only index-cache backend for MiniMax-M3 MSA side cache allocation/binding.
vllm_kunlun/v1/attention/backends/kunlun_attn.py Extends Kunlun attention backend behavior (paged-vs-spec decode selection, metadata additions, error checks).
vllm_kunlun/quantization/kernels/scale_mm.py Updates int8 scaled matmul path and adds a torch fallback option plus prequantized input support.
vllm_kunlun/quantization/compressed_tensors/compressed_tensors_moe.py Adds/rewires fused gate + pre-quantized MoE pipelines and related env toggles for Kunlun.
vllm_kunlun/preload_xpu.py Adds XPU shared-library preload helper to stabilize kunlun_ops import dependency resolution.
vllm_kunlun/platforms/envs.py Adds Kunlun env toggles for new fused kernels and behavior switches.
vllm_kunlun/ops/unquantized_linear.py Patches unquantized linear apply to prefer Kunlun matmul op when eligible.
vllm_kunlun/ops/rotary_embedding/kunlun_rope.py Re-enables Kunlun RoPE kernel path with stricter contracts and cache normalization.
vllm_kunlun/ops/linear.py Adds prequantized forward entry points for Column/RowParallelLinear and adjusts loader support handling.
vllm_kunlun/ops/layernorm.py Adds fused Gemma add-RMSNorm + quant entry point and fused-weight caching.
vllm_kunlun/ops/_custom_ops.py Registers multiple new custom ops (fused norm+quant, swiglu_bias, fused gate, moe pre-sort+quant, etc.).
vllm_kunlun/models/llama_eagle3.py Adds Kunlun-specific EAGLE3 performance patches (fc_norm fusion, identity vocab mapping fast-path).
vllm_kunlun/init.py Updates plugin registration to preload libs, expand post-import patch hooks, and add weak_ref_tensor fallback.
ci.yml Updates CI profile (Python version, resources) and switches to compileall checks for vllm + vllm_kunlun.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +25 to +27
torch._check(x.shape[0] >= 1)
torch._check(x.shape[0] == values.shape[0])
return (x >= values).sum(-1)
Comment on lines +276 to +279
assert len(generators) != logits.shape[0], (
"xpu kernel topk_topp_sampler does not support batch-wise generators."
)
generator = torch.xpu.default_generators[logits.device.index]
def eagle_step_slot_mapping_metadata_kernel(
positions_ptr, # [batch_size] - current positions (1D view for M-RoPE)
block_table_ptr, # [batch_size, n_blocks_per_req]
block_table_stride, # stride for block_table dim 1
- minimax_m3.py: drop unused imports (math, MiniMaxText01RMSNormTP,
  maybe_remap_kv_scale_name) and unused local `d`
- multiproc_executor.py / gpu_model_runner.py: convert 4 lambda
  assignments to def (E731)

Change-Id: Iac22296035f22e460f153c3acc3b72d4bd66e78c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants