fix(mlx_whisper): recast "auto"/blank language to None server-side - #8
Merged
Conversation
Clients connect to a socket and don't know which backend is behind it, so a uniform "auto" is the natural 'detect the language' request. But Whisper has no "auto" token — mlx_whisper.transcribe(language="auto") raises ValueError in its tokenizer, and language=None is how Whisper asks for auto-detect. Recast "auto"/blank -> None in the whisper backend (the one layer that knows its engine idiom and that the backend is whisper), rather than pushing the quirk onto every backend-agnostic client. Localized to the whisper backend on purpose: parakeet ignores language, and nemotron accepts "auto" as a first-class prompt key (None already maps to its "auto" default) — so a server-generic recast would needlessly couple the sentinel to nemotron's default. Net effect: a uniform client "auto" (or None) means auto-detect on all three backends. Adds parametrized whisper tests (auto/AUTO/' auto '/''/None -> None; real codes pass through); refreshes the three-way language contract docstring. Full suite 293 passed, 2 skipped; ruff clean.
Post-ship follow-up note in the nemotron plan's workspace (below the review marker, contract hash untouched): records that the three-way language contract was refined by PR #8 — mlx_whisper now normalises 'auto'/blank to None before forwarding (Whisper has no 'auto' token). CHANGELOG entry deferred to the next patch release.
Address two Minor deep-review findings (discoverability, no behavior change): - server.py: comment at the open_stream call naming the accepted language sentinels (None/'auto'/ISO code) and each backend's treatment, so a future backend author sees the contract at the call site. - mlx_whisper.py: module docstring now states it recasts 'auto'/blank->None and points to nemotron.py's full three-way contract table.
Bump version 0.3.0 -> 0.3.1 and add the CHANGELOG [0.3.1] 'Fixed' entry for the mlx_whisper 'auto'/blank -> None recast (PR #8). Footer link added; dev-plan Findings note updated to record the fix shipped in 0.3.1. Wheel METADATA stays PyPI-clean at 0.3.1 (no mlx-audio/direct-URL in Requires-Dist). Git tag v0.3.1 + GitHub release are cut after merge.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes
"auto"a safe, uniform "detect the language" sentinel across all STT backends by recasting it server-side, instead of asking every client to special-case it.Why
A client connects to a socket and is backend-agnostic — it can't see whether
nemotron,parakeet, orwhisperis behind it. So pushing "sendNone, not"auto"" onto clients is the wrong layer. Today:"auto"(before)NoneValueError: Unsupported language: auto→transcript.failedDEFAULT_NEMOTRON_LANGUAGE="auto"→ LIDOnly Whisper breaks:
mlx_whisper.transcribe's tokenizer accepts only real codes/names, andlanguage=Noneis how Whisper itself requests auto-detection.Change
Recast
"auto"/blank →Nonein the whisper backend (_normalize_language), at the one layer that knows both its engine's idiom and that the backend is whisper. Real codes ("en","es-ES") pass through unchanged.Localized, not server-generic, on purpose: parakeet ignores
languageand nemotron treats"auto"as a first-class token (and already mapsNone→ its"auto"default). A blanket server-level"auto"→Nonewould couple the sentinel to nemotron's default — a later change toDEFAULT_NEMOTRON_LANGUAGEwould silently change what"auto"means there. Translating per-backend avoids that.Net effect: a uniform client
"auto"(orNone) means "auto-detect" on every backend, with no client-side branching.Tests
Parametrized whisper backend cases:
auto/AUTO/" auto "/""/None→None;en/es-ESpass through. Three-way language-contract docstring innemotron.pyrefreshed.Full suite: 293 passed, 2 skipped;
ruff format+ruff checkclean.Downstream note
This lets a language-configurable client (e.g. koda's
STT_WS_LANGUAGE) send the literal"auto"and have it work uniformly — the client no longer needs to map its "auto" sentinel to omitting the field.Release
This PR also bumps the version to 0.3.1 and adds the CHANGELOG
[0.3.1]"Fixed" entry for this change (wheel METADATA verified PyPI-clean at 0.3.1). The git tagv0.3.1+ GitHub release are cut after merge.