--tokenizer-trust-remote-code cannot load DeepSeek-V3.2-Exp tokenizer (model_type=deepseek_v32) — no config fallback for an unregistered model_type without auto_map

[janbernloehr]

Summary

aiperf profile fails at Configure Profiling when --tokenizer points at a model whose model_type is not yet registered in the installed transformers AND whose config.json ships no auto_map. Concretely, deepseek-ai/DeepSeek-V3.2-Exp (model_type: "deepseek_v32") cannot be tokenized even with --tokenizer-trust-remote-code, because there is no remote config class to import and aiperf has no config-alias fallback. The serving engine (vLLM/SGLang) serves the model fine; only the aiperf client crashes, so no benchmark runs.

Environment

• aiperf 0.6.0post1 (behavior also present on current main)
• transformers 5.5.4 (resolved from aiperf's transformers>=4.56.0)
• model: deepseek-ai/DeepSeek-V3.2-Exp, architectures: ["DeepseekV32ForCausalLM"], model_type: "deepseek_v32", no auto_map in config.json
• GPUs: any (failure is at client tokenizer init, pre-traffic)

Steps to reproduce

pip install aiperf==0.6.0post1
# Any OpenAI-compatible server that serves DeepSeek-V3.2-Exp works; e.g. vLLM >= 0.11:
vllm serve deepseek-ai/DeepSeek-V3.2-Exp --trust-remote-code --port 8000 &
aiperf profile \
  --url localhost:8000 --model deepseek-ai/DeepSeek-V3.2-Exp \
  --tokenizer deepseek-ai/DeepSeek-V3.2-Exp --tokenizer-trust-remote-code \
  --endpoint-type completions --streaming \
  --synthetic-input-tokens-mean 1600 --output-tokens-mean 600 \
  --concurrency 1 --request-count 8

Expected behavior

aiperf loads the tokenizer (DeepSeek-V3.2 reuses the DeepSeek-V3 config schema and tokenizer) and runs the benchmark.

Actual behavior

WARNING  You are using a model of type `deepseek_v32` to instantiate a model of type ``. ... (configuration_utils.py)
ERROR    Failed to load tokenizer 'deepseek-ai/DeepSeek-V3.2-Exp'
ERROR    Failed to perform operation 'Configure Profiling'

--tokenizer-trust-remote-code is a no-op for this checkpoint because it has no auto_map, so there is no remote config class to import; AutoConfig falls back to an empty config for the unknown deepseek_v32 and the tokenizer load aborts.

Why aiperf cannot currently recover

src/aiperf/common/tokenizer.py implements alias resolution, trust_remote_code passthrough, a "Tokenizer class ... does not exist" upgrade hint, and a tiktoken builtin fallback — but it has no config-alias registry. An unregistered model_type with no auto_map therefore has no recovery path.

Suggested fix (matches the serving frameworks)

Register a small config alias before tokenizer load: subclass the closest existing transformers config and register it under the new model_type so AutoConfig.register's class.model_type == key consistency check passes. vLLM and SGLang already do exactly this:

• vLLM keeps a _CONFIG_REGISTRY in vllm/transformers_utils/config.py; vllm/transformers_utils/configs/granite4_vision.py documents the recommended lifecycle ("remove once transformers adds native support").
• SGLang registers class _DeepseekV32ConfigAlias(DeepseekV3Config): model_type = "deepseek_v32" via AutoConfig.register in python/sglang/srt/utils/hf_transformers/common.py.

Minimal sketch:

from transformers import AutoConfig, DeepseekV3Config

class _DeepseekV32ConfigAlias(DeepseekV3Config):
    model_type = "deepseek_v32"  # V3.2 reuses the V3 config schema

AutoConfig.register("deepseek_v32", _DeepseekV32ConfigAlias, exist_ok=True)

This should be temporary and removed once transformers registers deepseek_v32 natively (tracked upstream in huggingface/transformers#43037 / #42590).

Scope of the alias

Register an alias for deepseek_v32 only. Verified against transformers 5.5.4 CONFIG_MAPPING_NAMES:

• deepseek_v32: absent AND its checkpoint has no auto_map -> needs the alias (this bug).
• kimi_k2: also absent from transformers, but its checkpoint ships auto_map pointing at DeepseekV3Config, so --tokenizer-trust-remote-code already loads it — do NOT add an alias for it.
• deepseek_v3 / deepseek_v2: already present in transformers — no alias needed.

So the fix should be narrowly scoped to checkpoints that are both unregistered and lack auto_map.