fix: correctly quantize linear-layer weights and embeddings for full-… - #16
Open
uppiy47g wants to merge 1 commit into
Open
fix: correctly quantize linear-layer weights and embeddings for full-…#16uppiy47g wants to merge 1 commit into
uppiy47g wants to merge 1 commit into
Conversation
…precision (BF16) GGUF sources - gguf_tensor.py: unpack() previously returned native F32/F16/BF16 tensors as-is regardless of the requested default_tensor_type. When the entire source GGUF is BF16 (no per-tensor quantization applied by llama.cpp), this caused linear weights (mlp.up_proj/gate_proj/ down_proj, lm_head.weight, q_proj, etc.) that should be packed to Q4_1 per the model config to instead be left as raw oversized BF16 tensors in the wrong layout, crashing the FLM runtime (access violation in VCRUNTIME140.dll) on load. Now unpack() only takes the float passthrough path when the requested default_tensor_type is not itself a quantized target (Q4_0/Q4_1/Q8_0); otherwise it dequantizes and feeds into the existing quantize+pack path. 1D tensors (e.g. rope_freqs) are excluded since block quantization only applies to 2D matmul weights. - gemma4.py: add _quantize_embedding_int8() and use it for token_embd.weight / per_layer_token_embd.weight so embedding tables are exported as int8 + per-32-group F32 scale, matching the format expected by the FLM runtime instead of raw BF16. Verified: BF16-source conversion now produces a model.q4nx that is byte-identical in size and has identical tensor dtypes/shapes (709/709 match) to FLM's official reference Q4NX package, and loads/runs correctly. Q4_K_M-source conversion (mixed precision) regression tested with no change in behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…precision (BF16) GGUF sources
gguf_tensor.py: unpack() previously returned native F32/F16/BF16 tensors as-is regardless of the requested default_tensor_type. When the entire source GGUF is BF16 (no per-tensor quantization applied by llama.cpp), this caused linear weights (mlp.up_proj/gate_proj/ down_proj, lm_head.weight, q_proj, etc.) that should be packed to Q4_1 per the model config to instead be left as raw oversized BF16 tensors in the wrong layout, crashing the FLM runtime (access violation in VCRUNTIME140.dll) on load. Now unpack() only takes the float passthrough path when the requested default_tensor_type is not itself a quantized target (Q4_0/Q4_1/Q8_0); otherwise it dequantizes and feeds into the existing quantize+pack path. 1D tensors (e.g. rope_freqs) are excluded since block quantization only applies to 2D matmul weights.
gemma4.py: add _quantize_embedding_int8() and use it for token_embd.weight / per_layer_token_embd.weight so embedding tables are exported as int8 + per-32-group F32 scale, matching the format expected by the FLM runtime instead of raw BF16.
Verified: BF16-source conversion now produces a model.q4nx that is byte-identical in size and has identical tensor dtypes/shapes (709/709 match) to FLM's official reference Q4NX package, and loads/runs correctly.