Skip to content

Add first-class Qwen3.8 DFlash2 backend - #309

Open
Deviad wants to merge 1 commit into
youssofal:mainfrom
Deviad:issue-308
Open

Add first-class Qwen3.8 DFlash2 backend#309
Deviad wants to merge 1 commit into
youssofal:mainfrom
Deviad:issue-308

Conversation

@Deviad

@Deviad Deviad commented Aug 21, 2026

Copy link
Copy Markdown

Summary

  • add a first-class MLX dflash2 backend for Qwen3.8-27B bundles
  • keep target and DFlash2 weights namespaced under one fail-closed artifact contract
  • route ask, quickstart, and serve through the official dflash==0.1.0 implementation, with explicit target-only AR support
  • support unquantized, 8-bit, and 4-bit draft loading, telemetry, local sidecars, Homebrew environments, and OpenAI-compatible serving
  • add focused artifact/runtime/server/exactness tests and user documentation; no llama.cpp dependency

Verification

  • uv lock --check
  • Ruff: new DFlash2 source and tests pass; added-line scan reports no new Ruff findings in modified legacy files
  • focused DFlash2 + related regression suite passes; real-model gates skip unless explicitly enabled
  • unchanged artifact suite: 97 tests pass
  • full server suite: 329 tests pass
  • Python wheel and sdist build successfully; the wheel contains both new runtime modules and the dflash2 extra
  • Homebrew-routed ask completed through DFlash2; quickstart --dry-run --json selected backend_id=dflash2; a live OpenAI-compatible server request returned HTTP 200
  • 1,024-token prompt / 64-token greedy exactness passed for unquantized, 8-bit, and 4-bit DFlash2 plus native MTP, all with the same committed-token SHA-256
  • selected 4-bit draft: 19.29747831754572 wall-clock tok/s versus 11.05228273050392 target-only AR and 9.804179514121502 native MTP on the same prompt and budget

Five existing tests/test_public_cli.py failures were reproduced unchanged on the base revision and are not regressions from this change.

Closes #308

@Deviad
Deviad requested a review from youssofal as a code owner August 21, 2026 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add first-class Qwen3.8 DFlash2 backend for MLX runtime

1 participant