Skip to content

[Support Matrix] Request coverage for DeepSeek-V4-Flash on RTX PRO 6000D with SGLang 0.5.16 #1595

Description

@Ray-ling7

What coverage are you requesting?

Model (HuggingFace ID):

deepseek-ai/DeepSeek-V4-Flash

Native FP4-expert checkpoint, architecture DeepseekV4ForCausalLM.

Hardware / System:

  • Product designation: NVIDIA RTX PRO 6000D
  • CUDA/NVML-reported device name: NVIDIA RTX 6000D
  • Compute capability: SM120
  • Runtime-visible memory: 83.05 GiB per GPU
  • AIC system definition: not yet upstreamed; naming to be confirmed

This is the Chinese-market RTX PRO 6000D version and should not be
treated as equivalent to the existing 96 GB rtx_pro_6000_server
system definition.

Backend / Framework:

SGLang

Backend Version (if specific):

SGLang 0.5.16

Mode

  • Aggregated (agg)
  • Disaggregated (disagg)
  • Both

Additional context

The existing DeepSeek-V4 SGLang collectors contain several API
assumptions specific to SGLang 0.5.14 and cannot run against SGLang
0.5.16 without compatibility updates.

The affected collection paths include:

  • DeepSeek-V4 CSA/HCA context and generation modules
  • MHC pre/post modules
  • MXFP4 MoE
  • FP8 block GEMM

Representative incompatibilities observed with SGLang 0.5.16:

  • ModelRunner.__init__() no longer accepts the legacy tp_rank
    arguments
  • Req.set_extend_input_len() is no longer available
  • ForwardBatch.init_new() requires
    return_hidden_states_before_norm
  • KV-cache profiling moved from ModelRunner into
    KVCacheConfigurator
  • the legacy _global_server_args access path is no longer available
  • MoE and FP8 kernel modules were relocated

I have implemented a local compatibility patch and validated the
collectors against the environment above.

Local validation completed:

  • 213 relevant collector tests passed
  • Ruff lint passed
  • Ruff formatting check passed
  • CSA and HCA context collection completed successfully
  • CSA and HCA generation collection completed successfully
  • MHC, MoE, and FP8 block GEMM probes completed successfully
  • repeated measurements were stable

The implementation can be split into collector-focused PRs:

  1. DeepSeek-V4 CSA/HCA context and generation compatibility
  2. MHC collector compatibility
  3. MXFP4 MoE collector compatibility
  4. SM120 FP8 block GEMM collector compatibility

I would also like to contribute the validated SGLang 0.5.16 collector
compatibility changes and unit tests in a follow-up PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions