[ Don't Pull ] - Adds full support for converting Qwen3.6 35B A3B MoE variants - #18
Open
Atomic-Germ wants to merge 31 commits into
Open
[ Don't Pull ] - Adds full support for converting Qwen3.6 35B A3B MoE variants#18Atomic-Germ wants to merge 31 commits into
Atomic-Germ wants to merge 31 commits into
Conversation
This line had me so confused when trying to actually use the script.
I don't know about others, but I use more than one venv at different times. It'd be best practice also. AI DISCLOSURE: No it's a single line.
…precision (BF16) GGUF sources - gguf_tensor.py: unpack() previously returned native F32/F16/BF16 tensors as-is regardless of the requested default_tensor_type. When the entire source GGUF is BF16 (no per-tensor quantization applied by llama.cpp), this caused linear weights (mlp.up_proj/gate_proj/ down_proj, lm_head.weight, q_proj, etc.) that should be packed to Q4_1 per the model config to instead be left as raw oversized BF16 tensors in the wrong layout, crashing the FLM runtime (access violation in VCRUNTIME140.dll) on load. Now unpack() only takes the float passthrough path when the requested default_tensor_type is not itself a quantized target (Q4_0/Q4_1/Q8_0); otherwise it dequantizes and feeds into the existing quantize+pack path. 1D tensors (e.g. rope_freqs) are excluded since block quantization only applies to 2D matmul weights. - gemma4.py: add _quantize_embedding_int8() and use it for token_embd.weight / per_layer_token_embd.weight so embedding tables are exported as int8 + per-32-group F32 scale, matching the format expected by the FLM runtime instead of raw BF16. Verified: BF16-source conversion now produces a model.q4nx that is byte-identical in size and has identical tensor dtypes/shapes (709/709 match) to FLM's official reference Q4NX package, and loads/runs correctly. Q4_K_M-source conversion (mixed precision) regression tested with no change in behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Multi-file export with vision_weight.q4nx and audio_weight.q4nx where they belong - _pack_q4nx pad rows to 32 so numbers like 100 don't kill it - _export_q4nx_tensors will write an individual file such as in `-o dir/vision_weight.q4nx`
Easily used.
* extract -f string, fall back to repo name map * lfm prioritize q4_0, gpt-oss prefer q1_0 -> mxfp4 * use the maps
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AMD has been using some internal thing or manually piecing together models. No good.
This is entirely coded by AI and I didn't write a single line. For that reason, and because the devs never actually look at this repo anymore, this is not actually intended to be pulled.
I don't intend to go through any of the customary checks. Proof of it working is Atomic-Germ/Darwin-36B-Opus-NPU2 on hf or modelscope.
If someone else wants to write this same thing in a culturally acceptable manner, I'm sure everyone would appreciate it. If there was any chance of it being pulled. For the record I think being vehemently against AI to the point of refusing anything made with it regardless of quality is a short term fad people will feel really stupid about.
Y'all felt the same way about Photoshop. Photoshop was equated with no skill. Literally any tool like that is at first until people learn to be skilled in that context. It does in fact take skill to get anything Good from any tool. For that reason I will not go through all the hoops for anyone's PR comfort.
I'm not an AI-free safe space and if it's a problem, try not making your entire personality out of things you hate.
This will never be pulled, but it's here because it actually works.