Skip to content

fix(Qwen3/GLM5.2): exclude frontend-undecodable padded vocab rows#698

Open
Kokoro2336 wants to merge 4 commits into
openinfer-project:mainfrom
Kokoro2336:fix/issue-680
Open

fix(Qwen3/GLM5.2): exclude frontend-undecodable padded vocab rows#698
Kokoro2336 wants to merge 4 commits into
openinfer-project:mainfrom
Kokoro2336:fix/issue-680

Conversation

@Kokoro2336

Copy link
Copy Markdown
Contributor

Description

Fixes #680

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

Checklist

  • My code follows the style guidelines of this project (see docs/conventions/coding-style.md).

  • I have performed a self-review of my own code.

  • I have formatted my commits according to Commitizen conventions.

  • I have run the local test suite and all tests pass (see CLAUDE.md).

@Kokoro2336 Kokoro2336 changed the title Fix(Qwen3/GLM5.2): exclude frontend-undecodable padded vocab rows fix(Qwen3/GLM5.2): exclude frontend-undecodable padded vocab rows Jul 17, 2026
@Kokoro2336
Kokoro2336 marked this pull request as ready for review July 22, 2026 14:18

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bc62f3e093

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

effective_vocab - 1,
config.vocab_size,
);
config.selection_vocab = effective_vocab;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Apply the selectable vocab to Qwen3 target logits

When a Qwen3 tokenizer exposes fewer decodable IDs than the checkpoint vocab_size (the case this branch detects and logs here), the normal target path still projects and samples the full checkpoint vocab: LocalQwen3Lane::new sizes decode logits and SampleScratch with config.vocab_size, select_step_tokens samples those logits, and DFlash verify uses the same target logits. As a result, non-speculative Qwen3 requests and the verify bonus path can still choose the padded, frontend-undecodable rows; this new selection_vocab value only bounds the draft/Markov side unless the main prefill/decode/verify logits use it too.

Useful? React with 👍 / 👎.

Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Signed-off-by: Kokoro2336 <2529677678@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sampling: exclude frontend-undecodable padded vocab rows in Qwen3 and GLM5.2

1 participant