Skip to content

Rewrite Local Wisper primarily in BAML - #8

Merged
none23 merged 30 commits into
mainfrom
t3code/experiment-with-baml-rewrite
Aug 15, 2026
Merged

Rewrite Local Wisper primarily in BAML#8
none23 merged 30 commits into
mainfrom
t3code/experiment-with-baml-rewrite

Conversation

@none23

@none23 none23 commented Aug 14, 2026

Copy link
Copy Markdown
Owner

Summary

Replace the Python application with a compiled BAML application for local Parakeet speech-to-text. The existing lw command and Sway keybinding workflow remain compatible. The installed app runs without Python or the BAML CLI and automatically chooses CUDA or CPU inference.

Changes

  • Run nvidia/parakeet-tdt-0.6b-v3 through native ONNX inference instead of Python, NeMo, or Faster Whisper.
  • Prefer FP16 CUDA inference and fall back to INT8 CPU inference when CUDA is unavailable. Keep one warm model process and one verified model cache per user to avoid duplicate RAM or VRAM use.
  • Preserve the lw, preload, sway-start, sway-stop, and sway-cancel commands, the existing Sway wrapper contract, and clipboard or focused-window transcript delivery.
  • Preserve deterministic glossary and spoken-number cleanup, with optional OpenAI cleanup for longer transcripts.
  • Treat speech-to-text as a system-level feature rather than an editor feature, and remove the Neovim plugin.
  • Remove user-facing backend, model, device, and quantization choices in favor of one automatic configuration.
  • Add automated BAML, Rust, and shell quality checks for contributors and pull requests.

@none23 none23 changed the title Rewrite Local Wisper with BAML and Rust Rewrite Local Wisper primarily in BAML Aug 15, 2026
@none23
none23 force-pushed the t3code/experiment-with-baml-rewrite branch from 45686e3 to a3b9642 Compare August 15, 2026 06:54
@none23
none23 merged commit 155c5ef into main Aug 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant