You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
QPF016 and the setup at the end of each report take a model's context limit from max_position_embeddings alone. vLLM works it out in _get_and_verify_max_len and derive_max_model_len_and_key, and three differences matter:
It reads other keys too, such as n_positions, max_seq_len, and seq_length, takes the smallest, and lets model_max_length in the model config override them.
It takes the smaller of that and model_max_length in tokenizer_config.json.
It multiplies by the RoPE scaling factor, except for the su, longrope, llama3, yarn, deepseek_yarn, and deepseek_llama_scaling types, and except for Gemma 3. A linear factor of 4 on a 4,096-token model allows 16,384 tokens, so today QPF016 raises a false alarm at 8,192.
What to do
Add a helper in src/quantpreflight/rules.py that returns the limit from the config and the tokenizer config, following the three steps above. The checkpoint's tokenizer config is in meta.tokenizer_config.
The fix derives the effective vLLM context limit from the supported model config keys, model and tokenizer limits, and RoPE scaling rules, then uses the same value in QPF016 and setup planning. Tests cover configuration precedence, tokenizer caps, linear scaling, and the scaling exceptions.
Activity
mikemikimike commentedon Nov 6, 2026
Implemented in PR #19: #19
The fix derives the effective vLLM context limit from the supported model config keys, model and tokenizer limits, and RoPE scaling rules, then uses the same value in QPF016 and setup planning. Tests cover configuration precedence, tokenizer caps, linear scaling, and the scaling exceptions.
Validation:
python -m pytest -q(7556 passed)python -m pytest tests/test_rule_boundaries.py tests/test_planner.py -q(238 passed)