Repository navigation
Conversation
#322) New "transcribeWithVocabulary" command, unreachable from any Electron code path. Runs the exact same asr.transcribe() call "transcribe" already makes, then layers FluidAudio's CTC keyword-spotting rescore pass on top as a side-channel - not a different transcription class. Every API call here was verified against the vendored FluidAudio source first (CustomVocabularyContext, CtcKeywordSpotter, VocabularyRescorer, AudioConverter - see the #322 issue comment for the full research), then confirmed by actually compiling it. Vocabulary is hardcoded (a few proper nouns, two spoken-command phrases) - this spike answers one question, the ANE latency cost of the extra pass, and nothing about where a real vocabulary would come from. Never executed - this session's sandbox cannot run transcription-helper at all, same constraint as #253's Swift piece. docs/testing/vocabulary-boosting-spike-322.md has the manual steps and what to look for. Claude-Session: https://claude.ai/code/session_0144sxo5CJiyk99vaR7sgjuj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A compile-verified spike for #322 (keyword boosting), not a shipped feature - opened as its own PR so it's reviewable and mergeable independently, but this is research/spike work, not something to hold up other work on.
Why this exists
Posted findings on #322 after reading the vendored FluidAudio source: the issue's own cost estimate was too high on both counts (the second model is a documented ~97.5 MB, not 100-400 MB; boosting composes as a side-channel pass over the existing
AsrManager.transcribe()call, not an engine swap). The one thing that research couldn't answer - the ANE latency cost of the extra CTC pass on the actual M3 target - needed real code and real hardware, hence this spike.What's built
transcribeWithVocabulary, a new command ontranscription-helper's stdio protocol - not called from any Electron code path. Same request shape as"transcribe", runs the exact sameasr.transcribe()call unmodified, then layers FluidAudio's CTC keyword-spotting rescore pass on top against a hardcoded vocabulary (a few proper nouns, two spoken-command phrases - not a real vocabulary source, that's separate unresolved product work). Every FluidAudio API call was verified against source first, then confirmed by compiling.Verification
Compile-verified only.
swift build --package-path native/transcription-helper -c releasesucceeds. This session's sandbox cannot executetranscription-helperat all (confirmed separately, same constraint as #253's Swift piece), so nothing here has actually run.docs/testing/vocabulary-boosting-spike-322.mdhas manual steps: build, feed it a WAV via raw stdio JSON, compare itsmsagainst a plain"transcribe"request for the same clip to isolate what the boosting pass adds, and check whether it fits the sub-1s dictation budget (ADR-0002/ADR-0003).Please run it before drawing any conclusions from it - I'd genuinely like to know the latency number before this goes any further (into a real feature, or shelved).
🤖 Generated with Claude Code