QVAC-22367 fix(parakeet): window the offline encoder for long audio#101
Merged
freddy311082 merged 2 commits intoJul 22, 2026
Merged
Conversation
Review StatusCurrent Status: ✅ APPROVED |
GustavoA1604
requested changes
Jul 22, 2026
Author
|
/review |
Rebased the long-form windowed-encoder work onto master's engines/parakeet layout (post-reorg) to resolve the PR conflicts. Combines the original fix, the pos_emb hard-ceiling change, and the review fixups into one commit on the new layout. Bounds offline-transcription memory on long audio: transcribe_samples / transcribe_samples_stream used to run the conformer encoder over the whole input in a single graph (O(T_enc^2) self-attention, ~100 GB / SIGKILL on a ~90 min file). The mel is now computed once (so per-feature CMVN stays global, matching single-pass) and the encoder is slid over it in overlapping windows, with the shared context trimmed at the seams; inputs that fit one window keep the bit-identical single-pass path. - pos_emb_max_len is a hard ceiling on the per-window frame count (resolve_long_form_window_frames); auto window = min(pos_emb_max_len, 3750). - Pure planner + seam trim + bounds-checked append in long_form.h, covered model-free by test-long-form-windows (planner, resolver, stitch, append). - append_committed_frames fails loud (rc -6) instead of reading past the buffer; n_enc_frames advances only on a successful append. - test-long-form parity test varies per-repetition gain (fixture-gated). New EngineOptions knobs long_form_window_frames / long_form_context_frames (0 = auto, < 0 = disable).
Zbig9000
force-pushed
the
QVAC-22367-parakeet-long-audio
branch
from
July 22, 2026 14:41
e5f2255 to
e58ff4b
Compare
GustavoA1604
approved these changes
Jul 22, 2026
freddy311082
approved these changes
Jul 22, 2026
Zbig9000
marked this pull request as ready for review
July 22, 2026 16:47
Zbig9000
added a commit
to Zbig9000/qvac-registry-vcpkg
that referenced
this pull request
Jul 22, 2026
…rto tetherto#101) Repoint the port from the fork branch to the merged upstream commit now that tetherto/qvac-ext-lib-whisper.cpp#101 has landed: - REPO -> tetherto/qvac-ext-lib-whisper.cpp - REF -> 88b690c0 (master; PR tetherto#101, long-form windowed encoder on engines/parakeet) - HEAD_REF -> master - SHA512 recomputed for the upstream tarball; versions git-tree updated. SOURCE_PATH stays engines/parakeet. version-date/port-version unchanged (2026-07-21#0); ggml-speech floor unchanged (>= 2026-07-15).
Zbig9000
added a commit
to Zbig9000/qvac
that referenced
this pull request
Jul 22, 2026
…e at merged-upstream port Follow the registry PR after it was repointed to the merged upstream commit (tetherto/qvac-ext-lib-whisper.cpp#101, engines/parakeet). default-registry baseline -> 5a4aa57, whose parakeet-cpp 2026-07-21#0 port now builds from the merged upstream master (88b690c0) instead of the fork branch. Still resolving from the fork registry until the registry PR merges; once it does, switch default-registry back to tetherto and keep the version>= / overrides pin.
GustavoA1604
pushed a commit
to tetherto/qvac-registry-vcpkg
that referenced
this pull request
Jul 22, 2026
…der) (#254) * QVAC-22367 bump[parakeet-cpp]: 2026-07-21#0 (long-audio windowed encoder) Points parakeet-cpp at the QVAC-22367 fix that bounds offline-transcription memory on long audio: the conformer encoder now slides over the input in overlapping windows instead of running the whole file through a single O(T_enc^2) attention graph (which OOMed on multi-hour audio). ggml-speech floor is unchanged (>= 2026-07-15); this is a source-only bump. WIP until the upstream source PR merges: portfile REPO/REF/HEAD_REF point at Zbig9000/qvac-ext-lib-whisper.cpp @ 963b28db (branch QVAC-22367-parakeet-long-audio); SHA512 is that fork tarball's hash. master has since moved parakeet-cpp/ -> engines/parakeet/ (reorg PR #95); this pin stays on the pre-reorg ecac5bb7 line so the bugfix does not also adopt the layout migration. On merge, repoint REPO -> tetherto, REF -> the merge commit, HEAD_REF -> master, adjust SOURCE_PATH if it lands on the reorged tree, and recompute SHA512 + the versions git-tree. * QVAC-22367 review[parakeet-cpp]: repoint to review-fixed source + drop ticket refs Address review on the addon/source PRs: - Repoint REF to the review-fixed fork commit e5f22556 (global CMVN, model-free seam tests, fail-loud append) and update SHA512 + the versions git-tree. - Drop the ticket identifier from the portfile header and vcpkg.json description. Still WIP until the upstream source PR merges: REPO/REF/HEAD_REF point at the Zbig9000 fork branch; on merge, repoint REPO -> tetherto, REF -> the merge commit, HEAD_REF -> master, adjust SOURCE_PATH if it lands on the reorged engines/parakeet tree, and recompute SHA512 + git-tree. * QVAC-22367 review[parakeet-cpp]: repoint to engines/parakeet fork commit Follow the ext-lib PR after it was rebased onto master's engines/parakeet layout (reorg): REF -> fork commit e58ff4b0, SOURCE_PATH -> engines/parakeet, updated SHA512 + versions git-tree, and the subfolder references in the portfile/vcpkg.json. Still WIP (fork REPO/REF/HEAD_REF) until the upstream source PR merges; on merge repoint REPO -> tetherto, REF -> the merge commit, HEAD_REF -> master, recompute SHA512 + git-tree. * QVAC-22367 bump[parakeet-cpp]: 2026-07-21#0 -> merged upstream (tetherto #101) Repoint the port from the fork branch to the merged upstream commit now that tetherto/qvac-ext-lib-whisper.cpp#101 has landed: - REPO -> tetherto/qvac-ext-lib-whisper.cpp - REF -> 88b690c0 (master; PR #101, long-form windowed encoder on engines/parakeet) - HEAD_REF -> master - SHA512 recomputed for the upstream tarball; versions git-tree updated. SOURCE_PATH stays engines/parakeet. version-date/port-version unchanged (2026-07-21#0); ggml-speech floor unchanged (>= 2026-07-15).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
QVAC-22367 — Parakeet long-audio OOM fix (engine)
Problem
Engine::transcribe_samples/transcribe_samples_streamrun the FastConformerencoder over the entire input in a single graph. Self-attention is O(T_enc²), so
long audio builds enormous score tensors — a ~90 min file needs ~100 GB and the
process is SIGKILLed. The 0.3.3 ONNX long-form windowing was never re-implemented
for the GGML batch path.
Fix
When the input would exceed the per-window encoder-frame ceiling (auto:
min(pos_emb_max_len, 3750)), slide the encoder over the audio in overlappingwindows, trim the shared context at the interior seams, and concatenate the
committed frames into the same
[T_enc, d_model]buffer the decoder alreadyconsumes — bounded peak memory, single decode, no mid-word cuts. Inputs that fit
one window keep the bit-identical single-pass path, so short/typical files are
unaffected. New
EngineOptions.long_form_window_frames/long_form_context_frames(0 = auto, <0 = disable) make it tunable/testable.Tests
test-long-form-windows— model-free planner unit test (centre coverage, noseam gaps/duplicates, bounded window size incl. a 90-min case). Runs on every
checkout.
test-long-form— fixture-gated windowed-vs-single-pass parity on CTC/TDT.Built all 101 targets and
ctestgreen locally (CPU); the model-gated paritytest runs on CI where the GGUF fixtures are staged.
Base / reorg note (draft)
This branch is based on
ecac5bb7(the commit the shippedparakeet-cpp2026-07-13#2registry port pins) to keep the bugfix isolated and fullyvalidated against the currently-shipping tree.
masterhas since movedparakeet-cpp/→engines/parakeet/(reorg PR #95); the change needs replayingonto
engines/parakeet/(a mechanical relocation) before it can merge cleanly.Opening as draft pending that rebase / a decision on the preferred base.
Companion PRs
qvac-registry-vcpkg): QVAC-22367 bump[parakeet-cpp]: 2026-07-21#0 (long-audio windowed encoder) qvac-registry-vcpkg#254qvac): QVAC-22367 fix[transcription-parakeet]: bound long-audio memory via windowed encoder qvac#3379