Replace realtime Scribe streaming with post-recording batch transcription

.env.example

@@ -1,2 +1,2 @@
-# Required for realtime transcript token generation.
+# Required for post-recording batch transcription (main process only).
 ELEVENLABS_API_KEY=

AGENTS.md

@@ -241,7 +241,7 @@ Default to:
 - Very long timelines with dense camera-overlay keyframes can produce extremely nested ffmpeg filter expressions that exceed parser limits and fail render; the overlay graph needs to stay bounded for long sessions.
 - Heavy H.264 screen exports (high resolution, high quality, dense UI detail) can stress QuickTime playback more than NLEs; explicit yuv420p and a compatible profile/level—and avoiding unnecessary output resolution—improves default macOS playback.
 - `@electron/packager` dependency pruning can miss transitive packages under `pnpm`’s symlinked `node_modules` layout (fresh CI installs surface this); the packaging smoke harness avoids relying on that prune path.
-- The Scribe live-transcription `AudioWorklet` (`audio-processor`) must load as a plain browser worklet script; compiling it with the main-process TypeScript target (e.g. CommonJS/NodeNext) breaks loading (`exports` undefined) and silently disables transcription-driven features.
+- Transcription is a batch pass over the finalized recording (main-process `transcription-service` extracts audio via ffmpeg stream copy, then calls ElevenLabs batch Scribe with word timestamps); it must never run before the finalized files and the recovery checkpoint are on disk, and its failure/timeout must fall back to full-duration sections instead of blocking or losing the take. The realtime Scribe WebSocket/AudioWorklet pipeline was removed because long silences and session limits killed the live socket mid-recording; take flows are serialized (`stopRecordingInFlight` guards Record) so overlapping post-processing cannot race on the shared recovery checkpoint.
 - `src/renderer/styles/main.css` is Tailwind build output (`build:styles`, minified); full `check` rebuilds styles via nested steps, so tracking a non-matching formatted copy causes perpetual diffs—prefer generating locally and not treating it as hand-edited source.
 - Recording durability depends on streaming `MediaRecorder` chunks directly to disk via IPC temp `.part` files (renamed on finalize) instead of buffering in the renderer; screen and camera are independent recorder pipelines, so stop/save/finalize must tolerate one failing without blocking the other, and recovery normalization must require the screen file but tolerate a missing camera file so a partial failure does not discard an otherwise-recoverable take. Partial WebM files report `duration=Infinity`; resolve real duration by seeking a probe `<video>` element to `1e101` so the browser computes duration from the seekable range, and tolerate unreadable probes without wedging the whole recovery flow.
 - Premiere Pro FCP7 xmeml `Crop` must be emitted as `<effecttype>filter</effecttype>` + `<effectcategory>Matte</effectcategory>`; mis-classifying it as a motion effect makes Premiere silently drop the effect, leaving the camera uncropped and PiP placement drifting off the intended corner. Crop runs before the fixed Motion effect in Premiere's pipeline, which shapes how authored PiP geometry translates into Motion center/scale + Crop percentages.

docs/production/feature-inventory.md

@@ -78,23 +78,16 @@ Acceptance criteria:
 
 ## C. Transcript And Trim
 
-### C1. Realtime transcript
+### C1. Batch transcript
 
-- During recording, app sends PCM chunks to Scribe realtime websocket and displays partial + committed text.
+- After recording stops and the take's files are finalized and checkpointed for recovery, the app batch-transcribes the finalized mic audio (audio-only file, or audio extracted from the camera file via ffmpeg stream copy) with ElevenLabs Scribe in the main process, then groups word timestamps into speech segments.
 
 Acceptance criteria:
 
-- Committed transcript segments store `start/end/text`.
-- Non-speech annotations (e.g. bracketed cues) are stripped from user-visible transcript and segment content.
-
-### C2. Segment editing
-
-- User can select transcript segments and toggle deletion with keyboard shortcuts.
-
-Acceptance criteria:
-
-- Deleted segments are excluded from trim input.
-- Badge reflects active vs removed count.
+- No transcription network call happens before the finalized files and the recovery take are on disk; a transcription failure or timeout never loses or blocks the recording (falls back to full-duration sections with a visible warning).
+- Speech segments store `start/end/text`, with word timestamps mapped into recording time using the per-file recorder start offsets.
+- Non-speech annotations (e.g. bracketed cues) are stripped from segment content.
+- Recordings without mic audio skip transcription entirely (no network call).
 
 ### C3. Section computation
 

docs/production/runbook.md

@@ -3,7 +3,7 @@
 ## Environment
 
 - Copy `.env.example` to `.env`.
-- Set `ELEVENLABS_API_KEY` for realtime transcription token generation.
+- Set `ELEVENLABS_API_KEY` for post-recording batch transcription (used only in the main process; never exposed to the renderer).
 
 ## Local Verification
 

docs/production/target-architecture.md

@@ -22,6 +22,7 @@ src/
       render-service.js
       sections-service.js
       scribe-service.js
+      transcription-service.js
       media-service.js
     infra/
       file-system.js
@@ -33,6 +34,7 @@ src/
     features/
       transcript/
         transcript-utils.js
+        batch-transcript.js
       timeline/
         section-utils.js
         keyframe-utils.js
@@ -59,7 +61,7 @@ flowchart LR
   mainIpc --> recoveryService["services/recovery-service.js"]
   mainIpc --> renderService["services/render-service.js"]
   mainIpc --> sectionsService["services/sections-service.js"]
-  mainIpc --> scribeService["services/scribe-service.js"]
+  mainIpc --> transcriptionService["services/transcription-service.js"]
   projectService --> sharedDomain["shared/domain/*.js"]
   recoveryService --> sharedDomain
   renderService --> sharedDomain

src/main.ts

@@ -23,7 +23,7 @@ import { renderComposite } from './main/services/render-service';
 import { exportPremiereProject } from './main/services/premiere-export-service';
 import { computeSections } from './main/services/sections-service';
 import { generatePreview } from './main/services/preview-render-service';
-import { getScribeToken } from './main/services/scribe-service';
+import { transcribeRecordingFile } from './main/services/transcription-service';
 import * as proxyService from './main/services/proxy-service';
 import * as recordingService from './main/services/recording-service';
 
@@ -49,7 +49,7 @@ registerIpcHandlers({
   exportPremiereProject,
   computeSections,
   generatePreview,
-  getScribeToken,
+  transcribeRecordingFile,
   proxyService,
   recordingService,
   setPendingDisplayMediaSource

src/main/ipc/register-handlers.ts

@@ -19,6 +19,7 @@ import type { renderComposite } from '../services/render-service';
 import type { exportPremiereProject } from '../services/premiere-export-service';
 import type { computeSections } from '../services/sections-service';
 import type { generatePreview } from '../services/preview-render-service';
+import type { transcribeRecordingFile } from '../services/transcription-service';
 import type * as proxyServiceModule from '../services/proxy-service';
 import type * as recordingServiceModule from '../services/recording-service';
 
@@ -28,6 +29,9 @@ type RenderComposite = typeof renderComposite;
 type ExportPremiereProject = typeof exportPremiereProject;
 type ComputeSections = typeof computeSections;
 type GeneratePreview = typeof generatePreview;
+type TranscribeRecordingFile = (
+  opts: Parameters<typeof transcribeRecordingFile>[0]
+) => ReturnType<typeof transcribeRecordingFile>;
 type ProxyService = typeof proxyServiceModule;
 type RecordingService = typeof recordingServiceModule;
 
@@ -51,7 +55,7 @@ export function registerIpcHandlers({
   exportPremiereProject,
   computeSections,
   generatePreview,
-  getScribeToken,
+  transcribeRecordingFile,
   proxyService,
   recordingService,
   setPendingDisplayMediaSource
@@ -69,7 +73,7 @@ export function registerIpcHandlers({
   exportPremiereProject: ExportPremiereProject;
   computeSections: ComputeSections;
   generatePreview: GeneratePreview;
-  getScribeToken: () => Promise<string>;
+  transcribeRecordingFile: TranscribeRecordingFile;
   proxyService: ProxyService;
   recordingService: RecordingService;
   setPendingDisplayMediaSource: (sourceId: string | null) => void;
@@ -331,11 +335,20 @@ export function registerIpcHandlers({
     return filePaths[0];
   });
 
-  ipcMain.handle('get-scribe-token', async () => {
+  ipcMain.handle('transcription:transcribe', async (_event, opts: unknown) => {
+    const payload = (opts || {}) as { sourcePath?: unknown; languageCode?: unknown };
+    if (typeof payload.sourcePath !== 'string' || !payload.sourcePath.trim()) {
+      throw new Error('transcription:transcribe requires a sourcePath');
+    }
     try {
-      return await getScribeToken();
+      return await transcribeRecordingFile({
+        sourcePath: payload.sourcePath,
+        ...(typeof payload.languageCode === 'string' && payload.languageCode
+          ? { languageCode: payload.languageCode }
+          : {})
+      });
     } catch (error) {
-      console.error('Failed to get Scribe token:', error);
+      console.error('Batch transcription failed:', error);
       throw error;
     }
   });

src/main/services/scribe-service.ts

@@ -6,12 +6,4 @@ function getRequiredEnv(name: string): string {
   return value;
 }
 
-async function getScribeToken(): Promise<string> {
-  const apiKey = getRequiredEnv('ELEVENLABS_API_KEY');
-  const { ElevenLabsClient } = await import('@elevenlabs/elevenlabs-js');
-  const client = new ElevenLabsClient({ apiKey });
-  const response = await client.tokens.singleUse.create('realtime_scribe');
-  return response.token;
-}
-
-export { getRequiredEnv, getScribeToken };
+export { getRequiredEnv };

src/main/services/transcription-service.ts

@@ -0,0 +1,150 @@
+import crypto from 'crypto';
+import fs from 'fs';
+import os from 'os';
+import path from 'path';
+
+import ffmpegStatic from 'ffmpeg-static';
+
+import { runFfmpeg } from './ffmpeg-runner';
+import { getRequiredEnv } from './scribe-service';
+
+export interface TranscriptionWord {
+  text: string;
+  start?: number;
+  end?: number;
+  type?: string;
+}
+
+export interface TranscribeRecordingResult {
+  words: TranscriptionWord[];
+  languageCode?: string;
+}
+
+export interface TranscribeRecordingOptions {
+  /** Finalized recording file that carries the mic audio (audio-only or camera webm). */
+  sourcePath: string;
+  languageCode?: string;
+}
+
+export interface TranscriptionDeps {
+  runFfmpegImpl?: typeof runFfmpeg;
+  ffmpegPath?: string;
+  /**
+   * Performs the batch speech-to-text request. Defaults to the ElevenLabs SDK;
+   * injectable so tests never touch the network.
+   */
+  convertImpl?: (request: {
+    modelId: string;
+    file: { path: string };
+    timestampsGranularity: string;
+    tagAudioEvents: boolean;
+    diarize: boolean;
+    languageCode?: string;
+  }) => Promise<unknown>;
+}
+
+async function defaultConvertImpl(request: {
+  modelId: string;
+  file: { path: string };
+  timestampsGranularity: string;
+  tagAudioEvents: boolean;
+  diarize: boolean;
+  languageCode?: string;
+}): Promise<unknown> {
+  const apiKey = getRequiredEnv('ELEVENLABS_API_KEY');
+  const { ElevenLabsClient } = await import('@elevenlabs/elevenlabs-js');
+  const client = new ElevenLabsClient({ apiKey });
+  return client.speechToText.convert(request as never);
+}
+
+function normalizeWords(rawWords: unknown): TranscriptionWord[] {
+  if (!Array.isArray(rawWords)) return [];
+  const words: TranscriptionWord[] = [];
+  for (const raw of rawWords) {
+    if (!raw || typeof raw !== 'object') continue;
+    const candidate = raw as { text?: unknown; start?: unknown; end?: unknown; type?: unknown };
+    if (typeof candidate.text !== 'string') continue;
+    const word: TranscriptionWord = { text: candidate.text };
+    if (typeof candidate.start === 'number' && Number.isFinite(candidate.start)) {
+      word.start = candidate.start;
+    }
+    if (typeof candidate.end === 'number' && Number.isFinite(candidate.end)) {
+      word.end = candidate.end;
+    }
+    if (typeof candidate.type === 'string') {
+      word.type = candidate.type;
+    }
+    words.push(word);
+  }
+  return words;
+}
+
+/**
+ * Transcribes a finalized recording file with batch Scribe.
+ *
+ * The source recording is never modified: audio is stream-copied (no
+ * re-encode) into a temp file with ffmpeg first, both to drop video payload
+ * from the upload and to rewrite the container headers MediaRecorder leaves
+ * incomplete (partial WebM metadata). The temp file is always removed, and
+ * every failure surfaces as a rejection for the caller to handle — this
+ * service must never affect the durability of the recording itself.
+ */
+export async function transcribeRecordingFile(
+  { sourcePath, languageCode = 'eng' }: TranscribeRecordingOptions,
+  deps: TranscriptionDeps = {}
+): Promise<TranscribeRecordingResult> {
+  const runFfmpegImpl = deps.runFfmpegImpl || runFfmpeg;
+  const convertImpl = deps.convertImpl || defaultConvertImpl;
+
+  // Fail fast before doing any work when the key is absent, so the renderer
+  // gets a clear, actionable error instead of a late SDK failure.
+  getRequiredEnv('ELEVENLABS_API_KEY');
+
+  if (typeof sourcePath !== 'string' || !sourcePath.trim()) {
+    throw new Error('Transcription source path is required');
+  }
+  let stats: fs.Stats;
+  try {
+    stats = fs.statSync(sourcePath);
+  } catch {
+    throw new Error(`Transcription source not found: ${sourcePath}`);
+  }
+  if (!stats.isFile() || stats.size === 0) {
+    throw new Error(`Transcription source is empty: ${sourcePath}`);
+  }
+
+  const tempAudioPath = path.join(
+    os.tmpdir(),
+    `loop-transcribe-${crypto.randomBytes(6).toString('hex')}.webm`
+  );
+
+  try {
+    await runFfmpegImpl({
+      ffmpegPath: deps.ffmpegPath ?? ffmpegStatic ?? undefined,
+      args: ['-hide_banner', '-nostdin', '-i', sourcePath, '-vn', '-c:a', 'copy', '-y', tempAudioPath]
+    });
+
+    const response = (await convertImpl({
+      modelId: 'scribe_v2',
+      file: { path: tempAudioPath },
+      timestampsGranularity: 'word',
+      tagAudioEvents: false,
+      diarize: false,
+      languageCode
+    })) as { words?: unknown; languageCode?: unknown } | null;
+
+    const result: TranscribeRecordingResult = {
+      words: normalizeWords(response?.words)
+    };
+    if (typeof response?.languageCode === 'string') {
+      result.languageCode = response.languageCode;
+    }
+    return result;
+  } finally {
+    try {
+      fs.rmSync(tempAudioPath, { force: true });
+    } catch {
+      // Best-effort temp cleanup; never mask the primary error.
+    }
+  }
+}

src/preload.ts

@@ -56,7 +56,7 @@ const electronApi: ElectronApi = {
   importFile: (sourcePath, projectFolder) =>
     ipcRenderer.invoke('import-file', sourcePath, projectFolder),
   pickImageFile: () => ipcRenderer.invoke('pick-image-file'),
-  getScribeToken: () => ipcRenderer.invoke('get-scribe-token'),
+  transcribeRecording: (opts) => ipcRenderer.invoke('transcription:transcribe', opts),
   generateProxy: (opts) => ipcRenderer.invoke('proxy:generate', opts),
   onProxyProgress: (listener) => {
     if (typeof listener !== 'function') return () => {};