fix(weight_utils): name weight_type companions after the last weight only - #131
Open
KaigeGao1110 wants to merge 1 commit into
Open
KaigeGao1110 wants to merge 1 commit into
KaigeGao1110 wants to merge 1 commit into
Conversation
…only
The GGUF iterators built the synthetic companion name with
name.replace("weight", "weight_type"), which rewrites every occurrence.
A module whose own name contains "weight" therefore lost its name:
Qwen3.8-Flash-Next's hyper-connection projection
input_mix_weight_down.weight got the companion
input_mix_weight_type_down.weight_type, which no module owns, and loading
its GGUF failed with "no module or parameter named
hyper_connection_mixer.input_mix_weight_type_down".
Add gguf_weight_type_name(), which replaces only the last "weight", and use
it in both the LLM and the diffusion iterator. Names with a single "weight"
are unchanged. The tests cover the helper and drive both real iterators
over a written Q8_0 GGUF file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Kaige <a825075826@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
The GGUF weight iterators name the synthetic
weight_typecompanion withname.replace("weight", "weight_type"), which rewrites every occurrence. A module whose own name containsweighttherefore loses its name.Qwen3.8-Flash-Next's hyper-connection projection
input_mix_weight_down.weightgot the companioninput_mix_weight_type_down.weight_type. No module owns that name, so loading the GGUF failed withno module or parameter named hyper_connection_mixer.input_mix_weight_type_down.Changes
gguf_weight_type_name()inweight_utils.py. It replaces only the lastweight(the parameter name) and raises if the name has none.gguf_quant_weights_iterator_multi) and the diffusion iterator (weights_adapter/diffusion/base.py).Names with a single
weightare unchanged, so existing models map exactly as before.Test plan
tests/test_weight_type_names.py:weightnames,*_weight.weightnames and fusedw13_weight, and rejects names withoutweight;weightonly.With the two call sites reverted to
str.replace(the helper kept), the iterator tests fail:With the fix, all 8 tests pass.
Test results
I ran the full suite on one RTX PRO 6000 Blackwell (SM120) with vLLM nightly
2a02f6ef(0.28.1rc1.dev628) and PyTorch 2.13.0+cu130, with Hugging Face offline. I ran it on this branch and on unmodifiedmain(d4c1f0d), using the same_C_ggufbuild (csrc/is untouched):mainThe new tests pass (8 more passed). The 81 failures are the same test IDs on both trees, and none of them comes from this change:
test_kernels.py::test_moe:triton.runtime.errors.OutOfResources: out of resource: shared memory, because the Triton block size exceeds this GPU's limit.test_gguf_generation.py::test_modelsand 5 ×test_multimodal_gguf.py: the models cannot be downloaded offline (OSError,LocalEntryNotFoundError, andHFValidationErrorforrepo:quantreferences).test_plugin.py::test_register_sets_engine_args_for_gguf_model:HFValidationErrorfor the local path/tmp/model.gguf.pre-commit run --all-filespasses.Duplicate check: I searched open and closed PRs for
weight_type. Nothing touches companion naming.🤖 Generated with Claude Code