Skip to content

fix(forge): honor quantize:false module overrides - #281

Open
shiftedx wants to merge 1 commit into
youssofal:mainfrom
shiftedx:fix/forge-preserve-unquantized-modules
Open

fix(forge): honor quantize:false module overrides#281
shiftedx wants to merge 1 commit into
youssofal:mainfrom
shiftedx:fix/forge-preserve-unquantized-modules

Conversation

@shiftedx

Copy link
Copy Markdown
Contributor

Summary

Forge's mixed-precision converter currently turns every matching module_overrides entry into a quantization-parameter dictionary. An entry such as:

{"suffix": "linear_attn.in_proj_a", "quantize": false}

therefore ignores the exclusion and silently falls through to the default 8-bit override values.

This change makes build_predicate return the literal False value expected by mlx_lm.utils.quantize_model when a matching override explicitly sets "quantize": false. The matched module stays at source precision; ordinary mixed-precision overrides, layer filtering, first-match precedence, and body fallback behavior are unchanged.

The converter docstring now records the exclusion contract, and the regression test verifies both the excluded path and an unmatched body path.

Why this matters

Hybrid recipes use source-precision exclusions for numerically sensitive projections while quantizing the rest of the trunk. Before this fix, those recipes completed without an error but produced q8 tensors for the modules that were intended to remain BF16, changing both model size and the requested quality/performance tradeoff.

Verification

  • uv run --frozen --extra dev ruff check mtplx/commands/forge_mixed_convert.py tests/test_forge_mixed_convert.py — passed
  • uv run --frozen --extra dev pytest -q tests/test_forge_mixed_convert.py — 10 passed
  • MTPLX_CONFIG=<isolated-empty-path> uv run --frozen --extra dev pytest -q tests/test_no_mlx_imports.py tests/test_public_cli.py tests/test_runtime_kpis.py — 268 passed
  • uv run --frozen --extra dev python -m build — sdist and wheel built
  • scripts/hygiene_scan.sh — passed
  • scripts/fresh_venv_smoke.sh — passed

A real Qwen 3.8 27B conversion was also structurally inspected: all 96 requested recurrent-projection exclusions remained BF16, none gained quantization scale/bias tensors or quantization-config entries, and the other 234 mixed-precision overrides were still applied.

@shiftedx
shiftedx requested a review from youssofal as a code owner August 17, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant