A transparent, always-on-top, frameless live caption & translation overlay for your desktop — built with Tauri v2 + React.
It listens to your system audio (whatever is playing — YouTube, a meeting, a movie) and shows YouTube-style live captions at the bottom of your screen:
- EN — live English captions (streaming, ~100–200 ms latency) via Vosk
- JA — Japanese speech → English translation (Vosk recognition + Ollama translation)
- ES — Spanish speech → English translation (Vosk recognition + Ollama translation)
The widget is draggable, remembers its position and size, spans the full display width by default, and stays out of your way with a click-through-friendly transparent design.
- 🪟 Frameless, transparent, always-on-top overlay widget
- ⚡ Real-time streaming captions (Vosk processes 250 ms chunks in <10 ms)
- 🌐 Japanese / Spanish → English translation via Ollama (local or ollama.com with an API key)
- 📥 One-click speech-model download (no manual setup)
- 🔴 LIVE toggle — show only the current spoken sentence, hide the translation history
- 🖱️ Drag anywhere, resize freely — size & position persist across launches
- 🎨 Settings panel: font, font scale, opacity, widget width & heights, with a Reset button
- 🖥️ Cross-platform: Linux (PulseAudio/PipeWire), Windows (WASAPI loopback), macOS (CoreAudio via a loopback driver)
vid_translate/
├── index.html # Single HTML page Tauri loads (mounts #root)
├── package.json # Frontend deps (React 19, Vite 7, @tauri-apps/api) & scripts
├── vite.config.js # Vite dev-server config for Tauri (port 1420)
├── README.md # This file
├── HANDOFF.md # Developer handoff notes / design rationale
├── public/
│ └── vite.svg # Favicon
├── src/ # ── React frontend ──
│ ├── main.jsx # React entry point (mounts <App />)
│ ├── App.jsx # Entire UI: modes, start/stop, LIVE toggle, settings,
│ │ # window sizing/position persistence, backend events
│ └── App.css # All styling: overlay bar, buttons, settings, animations
├── dist/ # Vite build output (generated — embedded in release builds)
└── src-tauri/ # ── Rust backend ──
├── Cargo.toml # Rust deps: tauri 2, vosk, ureq, zip, dirs, serde, tokio,
│ # wasapi (Windows-only), cpal (macOS-only)
├── Cargo.lock # Pinned dependency versions
├── build.rs # Per-platform link paths + rpaths for the Vosk library
├── tauri.conf.json # Window config (frameless, transparent, always-on-top), dev URL
├── tauri.linux.conf.json # Linux-only overlay: bundles libvosk.so into the package
├── tauri.windows.conf.json # Windows-only overlay: bundles the DLLs as resources
├── tauri.macos.conf.json # macOS-only overlay: libvosk.dylib → Contents/Frameworks
├── Info.plist # macOS: merged into the bundle (mic usage description)
├── entitlements.plist # macOS: used only when signing with a real identity
├── capabilities/
│ └── default.json # Tauri v2 permissions (drag, resize, reposition, close…)
├── src/
│ ├── main.rs # Executable entry point → calls vid_translate_lib::run()
│ ├── lib.rs # The heart: Tauri commands (start/stop_listening,
│ │ # download_vosk_model, pull_model), model paths,
│ │ # EN pipeline, JA/ES translation pipeline (Ollama),
│ │ # TranscriptionEvent / StatusEvent emission
│ ├── recognizer.rs # Vosk streaming recognizer wrapper (Partial/Final/Silent)
│ └── audio/
│ ├── mod.rs # SAMPLE_RATE = 16000, cfg-switch between platforms
│ ├── linux.rs # Linux system-audio capture (parec / PulseAudio)
│ ├── windows.rs # Windows system-audio capture (WASAPI loopback)
│ └── macos.rs # macOS capture (CoreAudio loopback device + resampler)
├── vendor/
│ ├── linux-x86_64/
│ │ └── libvosk.so # Vosk shared library for Linux builds
│ └── macos/ # libvosk.dylib — fetched, not committed (see scripts/)
├── libvosk.dll # ┐
├── libvosk.lib # │ Vosk + MinGW runtime libraries
├── libgcc_s_seh-1.dll # │ vendored for Windows builds
├── libstdc++-6.dll # │ (bundled as resources)
├── libwinpthread-1.dll # ┘
├── icons/ # App icons for every platform (ico, icns, PNGs)
├── gen/schemas/ # Generated capability JSON schemas (do not edit)
└── target/ # Cargo build output, incl. release bundles (generated)
The mode button in the widget bar cycles through the three modes (click it while stopped: EN → JA → ES → EN …). Each mode is a different pipeline under the hood:
🇬🇧 EN — Live English captions
- System audio is captured at 16 kHz mono (PulseAudio on Linux, WASAPI loopback on Windows).
- 250 ms chunks are streamed into the Vosk English model (
vosk-model-small-en-us-0.15). - Vosk emits partial results (the sentence being spoken right now, updating live) and final results (completed utterances).
- Captions appear instantly in the overlay — no translation step, no network, fully offline.
Latency: ~100–200 ms end-to-end — the "YouTube captions" feel.
🇯🇵 JA — Japanese speech → English translation
- Same audio capture, but streamed into the Vosk Japanese model (
vosk-model-small-ja-0.22). - Live Japanese text is shown at the bottom of the widget as it's spoken (the dimmed line).
- When Vosk finalizes an utterance, it's sent to a background worker that calls Ollama with a translation prompt.
- The English translation streams in word-by-word and is added to a scrolling history above the live line.
Ollama options (in ⚙ Settings):
- Local — leave the API key empty; the app talks to
http://localhost:11434. Pull a model first (e.g.ollama pull gemma3:27b) or use the in-app pull with progress. - Cloud — set an ollama.com API key to use hosted models instead.
LIVE toggle: press LIVE to hide the history and show only the current spoken Japanese, big and centered — useful when you just want to shadow speech.
🇪🇸 ES — Spanish speech → English translation
Identical to JA mode, but uses the Vosk Spanish model (vosk-model-small-es-0.42) for recognition. Live Spanish appears at the bottom; finalized sentences are translated to English via Ollama and pushed into the history. The LIVE toggle works the same way.
📥 Speech models — auto-download
The first time you start a mode whose model is missing, the widget shows a Download button. One click fetches the model from alphacephei.com, shows progress, and extracts it to:
~/.local/share/vid_translate/ (Linux)
%LOCALAPPDATA%\vid_translate\ (Windows)
~/Library/Application Support/vid_translate/ (macOS)
├── vosk-model # English
├── vosk-model-ja # Japanese
└── vosk-model-es # Spanish
No manual steps needed.
| Requirement | Notes |
|---|---|
| Node.js ≥ 18 + npm | Frontend tooling |
| Rust (stable) + Cargo | Install via rustup |
| Tauri v2 system deps | Linux: webkit2gtk-4.1, libappindicator, etc. — see Tauri prerequisites. macOS: Xcode Command Line Tools + CMake (brew install cmake) |
| libvosk (macOS only) | bash scripts/fetch-libvosk-macos.sh once before the first build |
| A loopback driver (macOS only) | BlackHole or similar — see the macOS section below |
| Ollama (optional) | Only needed for JA/ES translation — ollama.com/download |
git clone <repo-url>
cd vid_translate
npm install
npm run tauri dev- First compile takes 5–15 minutes (builds the Vosk bindings). Later builds are fast.
- The frontend hot-reloads; Rust changes trigger a rebuild.
cd vid_translate
export APPIMAGE_EXTRACT_AND_RUN=1
export NO_STRIP=1
npx tauri build --bundles appimageWhy the environment variables?
APPIMAGE_EXTRACT_AND_RUN=1— the bundler's own tools (linuxdeploy,appimagetool) are themselves AppImages that need FUSE to mount. This flag makes them self-extract and run directly, so the build works on systems without (working) FUSE.NO_STRIP=1— the bundler normally strips binaries, butstripcorrupts the prebuiltlibvosk.soon some systems (e.g. Fedora's binutils). This skips stripping.
Output lands in:
src-tauri/target/release/bundle/appimage/vid_translate_0.0.1_amd64.AppImage
First run — make it executable:
chmod +x src-tauri/target/release/bundle/appimage/vid_translate_0.0.1_amd64.AppImage
./src-tauri/target/release/bundle/appimage/vid_translate_0.0.1_amd64.AppImage
libvosk.sois bundled inside the AppImage (viatauri.linux.conf.json+ rpath magic inbuild.rs) — no system-wide Vosk install needed.
On a Windows machine with Rust + Node installed:
cd vid_translate
npx tauri buildThis produces an .msi / NSIS installer under:
src-tauri\target\release\bundle\
The required DLLs (libvosk.dll, libgcc_s_seh-1.dll, libstdc++-6.dll, libwinpthread-1.dll) are vendored in src-tauri/ and bundled automatically as resources (see tauri.windows.conf.json). Audio capture uses WASAPI loopback, so it hears whatever the system is playing.
cd vid_translate
bash scripts/fetch-libvosk-macos.sh # once — downloads libvosk.dylib into src-tauri/vendor/macos/
npx tauri build --bundles applibvosk.dylib is copied into VidTranslate.app/Contents/Frameworks (via tauri.macos.conf.json) and found at runtime through the @executable_path/../Frameworks rpath embedded by build.rs — no Homebrew or system-wide Vosk install needed. The fetch script pulls Vosk's universal2 build (x86_64 + arm64) and hard-fails if the arm64 slice is missing.
Requires macOS 11 (Big Sur) or later — Apple Silicon does not exist below 11.0, so that is the floor for both architectures.
Releases ship two DMGs, split by architecture, not by chip generation:
| Download | Runs on |
|---|---|
VidTranslate_<version>_aarch64.dmg |
All Apple Silicon Macs — M1, M2, M3, M4, including Pro/Max/Ultra |
VidTranslate_<version>_x86_64.dmg |
Intel Macs |
There is no universal binary because ct2rs CMake-builds CTranslate2 for the host arch only, so CI runs one job on Apple Silicon and one on Intel. Both are baseline builds (no -mcpu=native), so the Apple Silicon DMG is not tied to the chip it was built on.
Release builds are ad-hoc signed, not notarized, so Gatekeeper blocks the first launch. Right-click the app → Open, or:
xattr -dr com.apple.quarantine /Applications/vid_translate.appLinux and Windows can tap the system output mix directly. macOS offers no such API to ordinary apps, so VidTranslate captures from a virtual loopback device instead — a free driver that presents whatever is played into it as a recordable input:
- Install BlackHole 2ch (or Loopback, VB-Cable, Soundflower…).
- Open Audio MIDI Setup → + → Create Multi-Output Device, and tick both BlackHole 2ch and your speakers/headphones.
- Set that Multi-Output Device as your Mac's sound output. You keep hearing audio, and BlackHole gets a copy.
- Start VidTranslate — it auto-detects BlackHole and captures from it.
If no loopback device is installed, the app shows a setup screen with a link to BlackHole and a Use microphone button, which falls back to capturing the default input instead. macOS will ask for microphone permission on the first capture either way — a loopback device is an input device as far as the OS is concerned.
Terminal-only setup (optional) — no GUI needed for either step:
brew install blackhole-2ch # asks for your admin password (it's a .pkg installer)
sudo killall coreaudiod # installer says "reboot required" — restarting coreaudiod sufficesThe Multi-Output Device can also be created programmatically instead of via Audio MIDI Setup — it is just a "stacked" CoreAudio aggregate device (AudioHardwareCreateAggregateDevice); see the macOS notes in HANDOFF.md for the exact recipe.
Good to know:
-
While a Multi-Output Device is the system output, the keyboard volume keys are disabled — macOS cannot control an aggregate device's volume. Adjust volume in the app that's playing, or switch output back when you're done captioning.
-
Undoing it all: switch output back to your speakers (System Settings → Sound), delete the Multi-Output Device in Audio MIDI Setup (select it → –), then remove the driver:
sudo rm -rf /Library/Audio/Plug-Ins/HAL/BlackHole2ch.driver && sudo killall coreaudiodThat single folder is the entire install — BlackHole ships no kernel extensions, launch agents, or background processes.
Open with the ⚙ button. Everything persists in localStorage:
| Setting | Default | Notes |
|---|---|---|
| Ollama API key | (empty) | Empty = local Ollama at localhost:11434 |
| Ollama model | gemma3:27b |
Any model Ollama can run/pull |
| Font / Font scale | System UI / 1.0× | |
| Opacity | 0.78 | Overlay background transparency |
| Width | (full display) | Set a px value for a narrower widget (also makes it draggable on both axes) |
| EN height / JA-ES height | 90 / 280 px | Per-mode widget heights |
| Reset | Restores UI defaults but keeps your Ollama key & model |
Manually resizing the window with the mouse updates and saves these presets automatically. The widget also remembers where you last placed it and reopens there.
Whisper is a batch encoder-decoder — it always processes a 30-second window, adding multi-second lag. Vosk is a streaming CTC model: partial words appear as they're spoken, giving true live-caption latency. For JA/ES, translation quality comes from Ollama instead, keeping recognition streaming and only translating finalized sentences.