Repository navigation
Drag-and-drop file transcription with a queue (#253) - #412
Merged
Merged
Conversation
Reads straight from disk instead of the base64-over-stdio path "transcribe" uses for live dictation - the file already exists and could be long, so there is no reason to round-trip it through JSON. Hands the URL straight to the file-URL overload of AsrManager.transcribe, which resamples via AVAudioFile (whatever format AVFoundation can open - WAV, AIFF, CAF, M4A/AAC, MP3) and auto-chunks past ASRConfig.default's 30s streaming threshold on its own. No ffmpeg, no manual chunking work needed on our side - confirmed by reading FluidAudio's own source, not guessed. Shares the #252 language-hint parsing with "transcribe" via a small languageHint() helper. Compile-verified only (swift build, twice) - I have no way to execute -test this from this session; every attempt to run transcription-helper here gets killed within seconds, same as attempts by the live running app's own supervisor. Needs a real run before this ships.
…es (#253) Sends a path field instead of base64 wav. The live-dictation REQUEST_TIMEOUT_MS (30s) makes no sense for a dropped file that could run many minutes, so request() now takes an optional per-call timeout and transcribeFile uses its own generous one (30 minutes).
Pure, adapter-injected module - one file at a time (the resident helper only ever runs one transcription regardless), a failure never blocks the rest of the queue, retry/remove/clearFinished for the UI layer to call. onChange fires a full snapshot on every transition so main.js can just push the latest array to the renderer rather than diffing.
#253) IPC surface: add (drop), chooseFiles (dialog.showOpenDialog, audio filter + All Files), getQueue, retry, remove, clearFinished, copy, save. Save writes a .txt next to the source file - an explicit action, so re-saving over an earlier save is expected, not guarded against. preload exposes webUtils.getPathForFile as openstream.files - the renderer's only way to resolve a dropped File to a real path now that Electron 32+ dropped the File.path augmentation.
Drop zone (dashed glass panel, active state on dragover) plus a Choose files… button for the file-picker path, both landing on the same fileTranscription.add() entry point. A queue card below shows each job's status pill, a live elapsed-time readout while it's transcribing, and per-status actions - Copy/Save once done, Retry on failure, Remove for anything not actively running. Verified visually with a stubbed window.openstream against the real compiled bundle (a screenshot harness, not the shipped app) - matches the existing Liquid design system with no new visual bugs.
# Conflicts: # electron/main.js # electron/preload.js # src/App.tsx # src/nav.ts # src/openstreamBridge.d.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #253 (the core: drop-a-file, a queue, save/copy output). Dock/menu-bar drop and a Finder Services entry are the issue's own "bonus" items - not in this PR. Tying the queue into a persistent results list waits on #136 (recording history), which doesn't exist yet.
The finding that shaped this
Read FluidAudio's source (
native/transcription-helper/.build/checkouts/FluidAudio) before building anything, rather than assuming the issue's own "handle formats ffmpeg/whisper accept" framing was still accurate post-#326. It isn't:AsrManager.transcribe(_ url: URL, ...)(the file-URL overload, distinct from the base64-buffer path live dictation uses) already resamples viaAVAudioFile/AVAudioConverter- whatever format AVFoundation can open (WAV, AIFF, CAF, M4A/AAC, MP3, FLAC), no ffmpeg needed.ASRConfig.defaulthasstreamingEnabled: trueand astreamingThresholdof 480,000 samples (30s at 16kHz) - anything longer already gets FluidAudio's own disk-backed chunked processing, automatically. No manual chunking needed either.So the whole file-handling half of the issue's "Build" list turned out to already be solved by the engine - the actual work is a new entry point onto it.
What's built
transcribeFileSwift command - reads straight from disk (apathfield, not base64), hands the URL to the file-URLtranscribe()overload. Shares the Multiple languages + auto-detect #252 language-hint parsing with the livetranscribecommand.transcriptionHelper.js:transcribeFile(path, language), with its own 30-minute request timeout (request()now takes a per-call override) - the 30s live-dictation ceiling makes no sense for a file that could run many minutes.fileTranscriptionQueue.js: a pure, adapter-injected FIFO coordinator - one file at a time, a failure never blocks the rest of the queue,retry/remove/clearFinished, full-snapshotonChangefor the UI to just re-render from.file-transcription:*IPC (add, chooseFiles viadialog.showOpenDialog, get/retry/remove/clearFinished/copy/save), pluswebUtils.getPathForFileexposed asopenstream.files- Electron 32+ dropped theFile.pathaugmentation, this is the replacement for resolving a droppedFileto a real path.add()entry point; a queue card with per-job status pill, a live elapsed-time readout, and Copy/Save/Retry/Remove depending on status.Judgement calls to veto
transcription-helperprocess, which only ever runs one transcription at a time (its command loop is a blocking read-one/reply-one). A long file transcribing can transiently delay a concurrent push-to-talk. The alternative - a second resident Parakeet instance - doubles the memory Idle model unload to cut resident RAM #257 exists to cut down, for a genuinely rare overlap (actively batch-transcribing a file and live-dictating in the same window). Flagged loudly in the manual-check doc as the one behaviour that needs a real look..txtnext to the source; clicking it twice overwrites the first save. It's an explicit user action each time, not a silent background write - treated it like any app's Save button, not a delete-adjacent action needing a confirm.What I could not verify
The Swift
transcribeFilecommand is compile-verified only (swift build, twice, clean) - not execution-tested. Every attempt to runtranscription-helperin this session got killed by the sandbox within seconds, including what looked like the live app's own restart attempts after an unrelated incident earlier in the session. The manual-check doc (docs/testing/file-transcription-253.md) leads with this and has a full checklist - please run it (afternpm run build:transcription-helper, since the checked-in binary predates this change) before trusting the feature.Everything else - the queue coordinator, the timeout plumbing, the IPC surface, the UI - is either unit-tested or was verified against the real compiled renderer bundle with a stubbed
window.openstream(a screenshot harness, not the shipped app - included as a mockup check, not a substitute for the manual pass).Tests
fileTranscriptionQueue.test.js(12),transcriptionHelper.test.js(+3 fortranscribeFile). Full suite (372) + typecheck + build green.🤖 Generated with Claude Code