Skip to content

Disable parallel quantization for mul1 codebook#239

Open
remichu-ai wants to merge 1 commit into
turboderp-org:devfrom
remichu-ai:fix/mul1-disable-parallel-quant
Open

Disable parallel quantization for mul1 codebook#239
remichu-ai wants to merge 1 commit into
turboderp-org:devfrom
remichu-ai:fix/mul1-disable-parallel-quant

Conversation

@remichu-ai

@remichu-ai remichu-ai commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Summary

convert.py --parallel_mode is currently unsafe with --codebook mul1. In a full GLM-5.2 MUL1 quant run it corrupts layer-0 EXL3 scale metadata (.suh tensors containing NaN/Inf), which then poisons layer-1 calibration Hessians and causes Cholesky failure.

This PR disables parallel_mode automatically when the selected codebook is mul1, with a warning, while leaving the existing parallel path unchanged for the default MCG codebook.

Validation

  • Syntax check: python -m py_compile exllamav3/conversion/convert_model.py
  • Reproduced failure with --codebook mul1 --parallel_mode: layer 0 produced NaN/Inf .suh; layer 1 Hessians became all-NaN and Cholesky failed.
  • Retested GLM-5.2 full 3.0bpw MUL1 quant with serial mode: cleanly passed layers 0, 1, and 2 and reached loading layer 3. Layer-0 saved tensors had no NaN/Inf floating tensors.

Notes

This is a conservative correctness guard. It avoids the known-bad combination without changing quantization kernels or the parallel path for other codebooks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant