Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ A local-first voice dictation app for macOS. The user holds a key, speaks, and t
One complete act of speaking and having the resulting text placed at the cursor. Begins when the user presses the push-to-talk key and ends when the text lands.
_Avoid_: Utterance, recording, session

**File transcription**:
A second, distinct mode (#253): the user drops or picks an existing audio file and gets a transcript back, rather than speaking live at the cursor. Shares the transcription engine and its resident model with Dictation, but nothing else - no cursor, no cleanup rules, no break placement, no push-to-talk. One file transcribes at a time; a long one can transiently delay a concurrent Dictation, since both use the same resident model.
_Avoid_: Batch transcription, offline transcription (accurate but not the term used here), import

**Dictation latency budget**:
The time from the end of speech to the text arriving at the cursor, which the product commits to keeping under one second. It ends where the user can see it end, so the cost of placing the text is inside the budget rather than outside it.
_Avoid_: Response time, turnaround
Expand Down
56 changes: 56 additions & 0 deletions docs/testing/file-transcription-253.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Manual check: drop-a-file transcription (#253)

CI covers the queue coordinator (`fileTranscriptionQueue.test.js`) and the
`transcribeFile` request/reply shape (`transcriptionHelper.test.js`) against
fakes. What it can't drive: the real Swift command, drag-and-drop, the file
picker, or a real audio file actually decoding.

**The Swift side (`transcribeFile` in `native/transcription-helper`) is
compile-verified only** - it was never executed in the session that wrote it;
every attempt to run `transcription-helper` there got killed by the sandbox
within seconds. Run `npm run build:transcription-helper` before testing any
of this - the checked-in `resources/bin/transcription-helper` predates the
change.

## Steps

1. **Basic drop.** Open the **Files** tab. Drag a short `.wav` or `.m4a`
file (a Voice Memo works) onto the drop zone - it highlights on
dragover, and the file appears in the queue as "Transcribing…" then
"Done" with a transcript.
2. **The file picker.** Click **Choose files…**, pick 2-3 files at once
(multi-select) in different formats - `.wav`, `.m4a`, `.mp3` if you
have one handy. All three should queue and process one at a time, in
the order picked - confirm only one row ever shows "Transcribing…" at
once.
3. **A long file.** Drop something over ~30s (FluidAudio's own streaming
threshold) - a longer Voice Memo or an exported meeting recording.
Confirm it still produces a full transcript, not just the first ~30s -
this is what confirms the file-URL `transcribe()` overload is really
auto-chunking rather than silently truncating.
4. **A bad file.** Drop something that isn't audio (a `.txt`, a `.jpg`).
The row should land on "Failed" with a readable error, not crash the
queue or the app - and the next queued file (if any) should still run.
5. **Copy / Save.** On a done job, click **Copy** - paste somewhere to
confirm the clipboard has the transcript. Click **Save** - a `.txt`
with the same base name should appear next to the source file; open it
to confirm the text matches. Save again - it overwrites without asking
(deliberate, see the #253 PR).
6. **Retry / Remove.** On a failed job, **Retry** re-queues it. On any
job that isn't currently transcribing, **Remove** drops it from the
list. Confirm **Remove** is unavailable (or a no-op) on the row that's
actively transcribing.
7. **Clear finished.** With a mix of done/failed/queued jobs, **Clear
finished** should drop the done and failed rows and leave anything
still queued or transcribing.
8. **Concurrency with live dictation - the known tradeoff.** Drop a longish
file, and while it's transcribing, hold push-to-talk and dictate
normally. Confirm what actually happens: does the live dictation wait
behind the file job (expected, per the #253 PR), or does something worse
happen (a crash, a lost dictation, the overlay hanging)? This is the one
behaviour the PR flagged as accepted-but-unverified.
9. **No persistence across restart.** Quit and relaunch with jobs still in
the queue - it should come back empty. This is deliberate for v1 (no
backing store yet - see #253's PR for the "ties into #136" follow-up),
confirm it's not silently losing an in-flight job's *file* (the source
file on disk is never touched, only the in-app queue state resets).
132 changes: 132 additions & 0 deletions electron/fileTranscriptionQueue.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
// #253: drop-a-file transcription. A small FIFO queue over the same
// resident transcription-helper live dictation uses (see transcribeFile()
// in transcriptionHelper.js) - one job at a time, since the helper only
// ever runs one transcription at a time regardless (it reads one
// newline-delimited request, blocks until the model replies, reads the
// next). A long file job can delay a concurrent live dictation; that is a
// known, accepted tradeoff for now (see the #253 PR), not something this
// module tries to work around - a second resident model would double the
// memory #257 exists to cut down.
//
// Pure and adapter-injected, same ports-and-adapters shape as
// dictationCoordinator.js: `transcription` is asserted, `onChange` is
// optional and fires with a full snapshot after every state change so a
// caller (main.js) can just re-render from the latest array rather than
// diffing.

function createFileTranscriptionQueue(options = {}) {
const { transcription, language = { get: () => "auto" }, onChange = () => {} } = options;

if (!transcription || typeof transcription.transcribeFile !== "function") {
throw new Error("createFileTranscriptionQueue requires transcription.transcribeFile()");
}

let jobs = [];
let nextId = 1;
let processing = false;

function snapshot() {
return jobs.map((job) => ({ ...job }));
}

function notify() {
try {
onChange(snapshot());
} catch {
// A listener must never break the queue.
}
}

function baseName(filePath) {
const parts = String(filePath).split("/");
return parts[parts.length - 1] || filePath;
}

// Runs until the queue is drained. Only one call is ever "live" at a
// time - the `processing` guard means a second addFiles() while a job is
// in flight just returns, and the in-flight call's own loop picks up
// whatever got added.
async function drain() {
if (processing) return;
processing = true;
try {
let job;
while ((job = jobs.find((candidate) => candidate.status === "queued"))) {
job.status = "transcribing";
job.startedAt = Date.now();
notify();
try {
job.text = await transcription.transcribeFile(job.path, language.get());
job.status = "done";
} catch (error) {
job.status = "failed";
job.error = error instanceof Error ? error.message : String(error);
}
job.finishedAt = Date.now();
notify();
}
} finally {
processing = false;
}
}

function addFiles(paths) {
const added = [];
for (const filePath of Array.isArray(paths) ? paths : []) {
if (typeof filePath !== "string" || !filePath) continue;
const job = {
id: String(nextId++),
path: filePath,
name: baseName(filePath),
status: "queued",
text: null,
error: null,
queuedAt: Date.now(),
startedAt: null,
finishedAt: null,
};
jobs.push(job);
added.push(job.id);
}
if (added.length > 0) {
notify();
void drain();
}
return added;
}

function retry(id) {
const job = jobs.find((candidate) => candidate.id === id);
if (!job || job.status !== "failed") return false;
job.status = "queued";
job.text = null;
job.error = null;
job.startedAt = null;
job.finishedAt = null;
notify();
void drain();
return true;
}

function remove(id) {
const index = jobs.findIndex((candidate) => candidate.id === id);
if (index === -1 || jobs[index].status === "transcribing") return false;
jobs.splice(index, 1);
notify();
return true;
}

function clearFinished() {
const before = jobs.length;
jobs = jobs.filter((job) => job.status === "queued" || job.status === "transcribing");
if (jobs.length !== before) notify();
}

function getJobs() {
return snapshot();
}

return { addFiles, retry, remove, clearFinished, getJobs };
}

module.exports = { createFileTranscriptionQueue };
Loading
Loading