Skip to content

Replace realtime Scribe streaming with post-recording batch transcription#14

Open
tadaspetra wants to merge 1 commit into
mainfrom
batch-transcription
Open

Replace realtime Scribe streaming with post-recording batch transcription#14
tadaspetra wants to merge 1 commit into
mainfrom
batch-transcription

Conversation

@tadaspetra

Copy link
Copy Markdown
Owner

What changed

Transcription is no longer a realtime WebSocket session during recording — it is now a batch pass over the finalized recording file, run after Stop.

  • New main-process transcription-service: extracts audio from the finalized take via ffmpeg stream copy (source file untouched; remux also fixes MediaRecorder's incomplete WebM headers), uploads to ElevenLabs batch Scribe (scribe_v2, word-level timestamps). The API key stays in the main process — the renderer no longer receives a Scribe token.
  • New batch-transcript renderer helper: groups word timestamps into speech segments using the same 1.5s silence threshold the old VAD used, so auto-cut boundaries stay familiar.
  • Stop flow reordered around durability: finalized files + recovery checkpoint (.pending-recording.json) are on disk before any network call. Transcription failure or timeout falls back to full-duration sections with a visible warning — it can only ever cost the auto-cut, never footage.
  • Timestamps anchored to per-file recorder start offsets (audioStartOffsetMs/cameraStartOffsetMs) instead of the old wall-clock scribeAudioOffset estimate.
  • Record is blocked while the previous take is still post-processing (visible notice), so overlapping take flows cannot race on the shared recovery checkpoint or the stop guard.
  • Removed dead realtime code: WebSocket/AudioWorklet/PCM pipeline, audio-processor.ts, scribe-status.ts, live segment-editing keyboard shortcuts, scribe token IPC (net −215 lines).

Why

The realtime Scribe session was killed server-side during long pauses (idle/session limits), silently ending transcription for the rest of the take. Batch transcription of the whole file removes that failure mode entirely, improves accuracy, and deletes the reconnect/offset-estimation complexity.

Behavior tradeoffs a reviewer should know

  • No live transcript while recording; the panel shows a passive note and transcription progress appears after Stop.
  • The Backspace-to-delete-segment shortcut during recording is gone (segments don't exist until after Stop).
  • Stop now waits on the batch transcription (bounded: 2–10 min timeout scaled to take length) before entering the editor; the take is already durable on disk during that wait.

Tests

  • tests/unit/batch-transcript.test.ts — segment grouping, offsets, annotation stripping, timeout bounds (15 tests)
  • tests/unit/transcription-service.test.ts — ffmpeg args, temp-file cleanup on success and every failure path, missing key/file/empty-file errors, response normalization (10 tests)
  • tests/unit/register-handlers.test.ts — new transcription:transcribe handler delegation + payload validation (3 tests)
  • media-cleanup and scribe-service suites updated; obsolete scribe-status tests removed with the module

pnpm run check passes (lint, typecheck, unit/integration, Electron smoke, packaging smoke). Docs updated: feature-inventory, runbook, target-architecture, .env.example, AGENTS.md learned facts.

Residual risk: real-network transcription path is exercised only manually; suggested spot-check is a short take with a deliberate 30s pause to confirm auto-cut boundaries.

🤖 Generated with Claude Code

…tion

The realtime WebSocket session died during long silences (idle/session
limits), killing the transcript for the rest of the take. Transcription is
now a batch pass over the finalized recording, ordered so footage durability
never depends on the network:

- Stop flow saves the recovery checkpoint before any transcription call;
  failure or timeout falls back to full-duration sections with a visible
  warning instead of blocking or losing the take.
- New main-process transcription-service extracts audio from the finalized
  file via ffmpeg stream copy and calls batch Scribe (scribe_v2, word
  timestamps); the API key never reaches the renderer.
- Word timestamps map into recording time via per-file recorder start
  offsets instead of the old wall-clock estimate.
- Record is blocked while the previous take is still post-processing so
  overlapping take flows cannot race on the shared recovery checkpoint.
- Removed the WebSocket/AudioWorklet/PCM pipeline, live segment editing UI,
  scribe token IPC, and scribe-status module (net -215 lines).

Tests: segment builder + timeout helper (15), transcription service temp-file
cleanup and failure paths (10), IPC handler validation (3); media-cleanup and
register-handlers suites updated. pnpm run check passes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant