[Models] Support DeepSeek-V4 GGUF weights mapping and architecture fallback

[DjoserKhemSimeu]

Summary

This PR adds GGUF weight mapping and configuration parsing support for DeepSeek-V4 architectures (such as DeepSeek-V4-Flash checkpoints).
Related to : #114

Motivation & Problem

When serving a DeepSeek-V4 GGUF checkpoint (e.g., bartowski/DeepSeek-V4-Flash-0731-GGUF), model initialization fails during weight name resolution with the following error:

RuntimeError: Unknown gguf model_type: deepseek_v4

Root Causes

1. Model Type Resolution: In transformers.py, only deepseek_v2 and deepseek_v3 were mapped to the internal deepseek2 GGUF architecture mapping. deepseek_v4 was unhandled, causing a lookup failure in gguf.MODEL_ARCH_NAMES.
2. Config Parser Architecture Lookup: In config_parser.py, if config.model_type was not yet registered inside transformers' MODEL_FOR_CAUSAL_LM_MAPPING_NAMES and no custom adapter override was set, the parser failed with RuntimeError: Can't get gguf config for {config.model_type} instead of inspecting config.architectures already declared in the Hugging Face config.

Technical Changes

• GGUF Weights Adapter (vllm_gguf_plugin/weights_adapter/transformers.py):

• Included "deepseek_v4" alongside "deepseek_v3" and "deepseek_v2" to map to the "deepseek2" GGUF architecture.

• Set up MoE expert slicing (gate_proj, up_proj, down_proj, and e_score_correction_bias).

• Config Parser Fallback (vllm_gguf_plugin/config_parser.py):

• Added graceful fallback to config.architectures[0] when get_adapter_architecture(config) and MODEL_FOR_CAUSAL_LM_MAPPING_NAMES do not have an explicit entry for new or custom causal LM model types.

• Tests (tests/test_plugin.py):

• Added unit test test_gguf_config_parser_fallback_to_config_architectures to verify config parser fallback behavior for newly introduced model types with custom architectures.

Verification & Testing

• Verified tensor name mappings for DeepSeek-V4 MoE layer patterns (blk.{idx}.exp_probs_b.bias, blk.{idx}.ffn_{gate,up,down}_exps.weight).
• Verified GGUFConfigParser test suite passes and handles both canonical and custom architecture configs.

---

Signed-off-by: DjoserKhemSimeu [djoser.simeu@gmail.com](mailto:djoser.simeu@gmail.com)

[Isotr0py]

QQ: Does this work with Mega MoE?

[DjoserKhemSimeu]

QQ: Does this work with Mega MoE?

No, it does not use the fused Mega MoE kernel (DeepGEMM).

Here is exactly how DeepSeek-V4 GGUF execution currently works:

• Weight Handling: The adapter maps the GGUF weights to the deepseek2 MoE architecture. It uses split_stacked_experts to unbind the 3D stacked expert tensor into individual per-expert weights (specifically model.layers.{idx}.mlp.experts.{expert_id}.{gate,up,down}_proj and e_score_correction_bias).

• Execution Path: Computation is routed through GGUFLinearMethod using the plugin's dequantization and matmul kernels (such as fused_mul_mat_gguf or the standard vLLM MoE linear path).