speech addons: enable per-arch ggml CPU variants on desktop linux-arm64 (whispercpp, bci, tts-ggml)#3272
Merged
Merged
Conversation
Contributor
Review StatusCurrent Status: ✅ APPROVED |
GustavoA1604
requested changes
Jul 14, 2026
GustavoA1604
left a comment
Contributor
There was a problem hiding this comment.
Also put concise entry in CHANGELOG.md under [unreleased] for the packages
41ee84c to
3a8b3ee
Compare
Contributor
Author
|
Reworked per review guidance:
|
3a8b3ee to
26d35e4
Compare
…speech 2026-07-14) Desktop linux-arm64 prebuilds compiled ggml at the plain armv8-a baseline: the ARM dotprod/fp16/i8mm kernels were compiled out and both f16 and quantized models ran on the slowest generic paths (Android already builds GGML_CPU_ALL_VARIANTS). The build-flags change lives in the ggml-speech vcpkg port (2026-07-14, qvac-registry-vcpkg PR #249): per-arch runtime-scored CPU MODULE variants (SIGILL-safe on any aarch64 host), static libc++ in the modules, flat .so staging in lib/, SVE tiers excluded (2-4x slower than NEON on the VL=128 cores in the desktop fleet). This PR consumes the new port version in bci-whispercpp, transcription-whispercpp and tts-ggml, and makes the two whisper addons load the dynamically-loaded backends on linux-arm64: - WhisperModel/BCIModel: the ensureBackendsLoadedAndroid gate (now ensureBackendsLoaded) widens from __ANDROID__ to __ANDROID__ || (__linux__ && __aarch64__) -- with GGML_BACKEND_DL=ON no backend is statically registered and the addon must call ggml_backend_load_all_from_path(backendsDir/BACKENDS_SUBDIR) before whisper_init. Kept off linux-x64, which stays on static backends. tts-ggml already threads backends_dir unconditionally through tts-cpp EngineOptions. Verified end-to-end via overlay-port benchmark runs (ubuntu-24.04-arm, mean RTF): whisper base f16 0.3316 -> 0.0970 / q8_0 0.1270 -> 0.0629, small f16 1.3450 -> 0.3426 (run 29333221534); chatterbox q4 2.70 -> 2.28 (run 29333223845); bci green (run 29333225816). All other platforms and Android unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
26d35e4 to
728e30c
Compare
Contributor
Author
|
Per review: consumption now flows through the -cpp ports — tetherto/qvac-registry-vcpkg#249 bumps parakeet-cpp/whisper-cpp/tts-cpp to require ggml-speech 2026-07-14, and this PR just bumps the whisper-cpp override (1.9.1#3) / tts-cpp constraint (2026-07-13#2). Direct ggml-speech deps removed. Also rebased onto main (CHANGELOG conflict with #3265 resolved). |
Contributor
QVAC E2E —
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the transcription-parakeet linux-arm64 CPU enablement (#3263) to bci-whispercpp, transcription-whispercpp and tts-ggml.
Problem
Desktop linux-arm64 prebuilds compiled ggml at the plain armv8-a baseline — the ARM dotprod/fp16/i8mm kernels were compiled out, leaving f16 and quantized models on the slowest generic paths (Android already builds
GGML_CPU_ALL_VARIANTS).Changes
The build-flags change lives in the ggml-speech vcpkg port (2026-07-14, qvac-registry-vcpkg#249) — per review guidance, a change for every consumer of the port belongs in the port, not in triplets. The port builds per-arch runtime-scored CPU MODULE variants (SIGILL-safe), statically links libc++ into the modules, stages them flat in
lib/(where the addons' existing staging glob already looks), and excludes the SVE tiers (2–4× slower than NEON on VL=128 cores — A/B linked in the port PR).This PR:
whisper-cppoverride1.9.1#1 → 1.9.1#3(transcription-whispercpp, bci-whispercpp) andtts-cpp >= 2026-07-13#2(tts-ggml) — whose ports now requireggml-speech >= 2026-07-14(per review: no direct ggml-speech dependency in the packages).ensureBackendsLoadedAndroid→ensureBackendsLoaded) widens from__ANDROID__to__ANDROID__ || (__linux__ && __aarch64__)— withGGML_BACKEND_DL=ONno backend is statically registered, and the addon must callggml_backend_load_all_from_path(backendsDir/BACKENDS_SUBDIR)beforewhisper_init. Gated to linux-arm64 only per review (linux-x64 stays on static backends). tts-ggml needs no addon change — it already threadsbackends_dirunconditionally through tts-cppEngineOptions.[Unreleased]for all three packages.Verification (ubuntu-24.04-arm, mean RTF, lower is better)
End-to-end benchmark runs with this change set (port content via overlay):
All other platforms and Android unchanged/green in the same runs.
Merge order: qvac-registry-vcpkg#249 first (version resolution needs ggml-speech 2026-07-14 in the registry). Independent of #3263.
🤖 Generated with Claude Code