Skip to content

[Bug] Checkpoint conversion fails for Qwen3.6-35B FP8: Tensor size mismatch in dequant_fp8_weight #402

Description

@quanrennsxsb

Before you start

  • I have checked the supported models and this checkpoint is not listed there. A listed checkpoint that fails is a bug; use a Bug report instead.
  • I have read the Roadmap and this model is not already planned there.
  • I have read the FAQ.
  • I have searched existing issues and found no duplicate.
  • This is not a GGUF checkpoint. GGUF support is on the Roadmap; please do not open an issue for it.

Hugging Face link

https://huggingface.co/lamianlbe/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-fp8

Is the model architecture already supported

Yes, but this checkpoint or quantization does not load

Is the quantization already supported

Yes, but this checkpoint's weight format does not load

What happens when you load it

[12:04:28] stdout/INFO  Traceback (most recent call last):
[12:04:28] stdout/INFO    File "<frozen runpy>", line 198, in _run_module_as_main
[12:04:28] stdout/INFO    File "<frozen runpy>", line 88, in _run_code
[12:04:28] stdout/INFO    File "C:\Users\quanren\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 138, in <module>
[12:04:28] stdout/INFO      raise SystemExit(main())
[12:04:28] stdout/INFO    File "C:\Users\quanren\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 134, in main
[12:04:28] stdout/INFO      return runner(args[1:])
[12:04:28] stdout/INFO    File "C:\Users\quanren\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\cli.py", line 50, in _run_checkpoint
[12:04:28] stdout/INFO      return main(argv, prog="ft checkpoint")
[12:04:28] stdout/INFO    File "C:\Users\quanren\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\checkpoint\__main__.py", line 53, in main
[12:04:28] stdout/INFO      index = convert_checkpoint(
[12:04:28] stdout/INFO    File "C:\Users\quanren\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\checkpoint\convert.py", line 211, in convert_checkpoint
[12:04:28] stdout/INFO      for name, tensor in count_bar(load_weight(model_path, torch.device("cpu"),
[12:04:28] stdout/INFO    File "C:\Users\quanren\AppData\Local\FreeToken\venv\Lib\site-packages\tqdm\std.py", line 1187, in __iter__
[12:04:28] stdout/INFO      for obj in iterable:
[12:04:28] stdout/INFO    File "python/freetoken/models/weight.py", line 251, in load_weight
[12:04:28] stdout/INFO    File "python/freetoken/models/qwen3_5_moe/weight.py", line 230, in iter_weights
[12:04:28] stdout/INFO    File "python/freetoken/models/qwen3_5_moe/weight.py", line 265, in freetoken.models.qwen3_5_moe.weight.iter_weights
[12:04:28] stdout/INFO    File "python/freetoken/models/qwen3_5_moe/weight.py", line 130, in freetoken.models.qwen3_5_moe.weight._load_maybe_quantized
[12:04:28] stdout/INFO    File "python/freetoken/models/qwen3_5_moe/weight.py", line 87, in freetoken.models.qwen3_5_moe.weight._dequant_fp8_weight
[12:04:28] stdout/INFO  RuntimeError: The size of tensor a (2048) must match the size of tensor b (16) at non-singleton dimension 1
[12:04:29] stdout/INFO  checkpoint Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-fp8 exited with code 1

Anything else

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions