fix(Qwen3/GLM5.2): exclude frontend-undecodable padded vocab rows#698
fix(Qwen3/GLM5.2): exclude frontend-undecodable padded vocab rows#698Kokoro2336 wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc62f3e093
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| effective_vocab - 1, | ||
| config.vocab_size, | ||
| ); | ||
| config.selection_vocab = effective_vocab; |
There was a problem hiding this comment.
Apply the selectable vocab to Qwen3 target logits
When a Qwen3 tokenizer exposes fewer decodable IDs than the checkpoint vocab_size (the case this branch detects and logs here), the normal target path still projects and samples the full checkpoint vocab: LocalQwen3Lane::new sizes decode logits and SampleScratch with config.vocab_size, select_step_tokens samples those logits, and DFlash verify uses the same target logits. As a result, non-speculative Qwen3 requests and the verify bonus path can still choose the padded, frontend-undecodable rows; this new selection_vocab value only bounds the draft/Markov side unless the main prefill/decode/verify logits use it too.
Useful? React with 👍 / 👎.
bc62f3e to
fd11135
Compare
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
fd11135 to
7091ef0
Compare
Description
Fixes #680
Type of Change
Checklist
My code follows the style guidelines of this project (see
docs/conventions/coding-style.md).I have performed a self-review of my own code.
I have formatted my commits according to Commitizen conventions.
I have run the local test suite and all tests pass (see
CLAUDE.md).