Real-time speech recognition for Roman Urdu, English, and code-switched Urdu-English. Speak naturally, get Urdu written in Latin letters — not Urdu script, not translated to English. Runs entirely on your own CPU with faster-whisper: no API key, no internet, no per-minute billing, nothing leaves your machine.
Drop it into any Python project with two callbacks, or run it as a standalone CLI.
| The problem it solves | Why Roman Urdu is genuinely hard |
| Install | Two commands |
| Run it standalone | CLI in 10 seconds |
| Use it in your project | The integration guide |
| API reference | Every method and option |
| Recipes | Copy-paste integrations |
| For AI agents | Prompt for Claude / GPT / Cursor |
| Configuration | Tuning table |
| How it works | Architecture |
| Benchmarks | Real measured numbers |
| Troubleshooting | Hard-won fixes |
| Credits |
Most speech-to-text either gives you English or gives you Urdu script. Neither is what a Pakistani developer actually types.
You say: "aaj main aap se baat kar raha hoon, this is mixed speech"
Urdu STT → میں آپ سے بات کر رہا ہوں wrong alphabet
English STT → "I am talking to you today" it translated you
This module → "Aaj main aap se baat kar raha hoon, this is mixed speech."
Whisper picks its output script from the language token you give it. Ask for Urdu and you get Urdu script. Ask for English and it translates instead of transcribing — silently destroying your actual words.
The trick this module uses: force language="en" and feed Roman Urdu prose as
initial_prompt. The decoder treats that prompt as "text so far" and keeps writing in the
same script — spelling Urdu phonetically in Latin letters while leaving your English alone.
Transliterating Urdu script afterwards does not work as an alternative. Urdu script omits short vowels, so
ممکن ہےcan only ever be recovered asmmkn hy, nevermumkin hai. The information isn't in the text. This was tested and discarded.
| Roman Urdu + English | Handles code-switching mid-sentence, no language flag needed |
| Truly offline | Nothing leaves the machine. Works on a plane |
| Zero cost | No key, no quota, no per-minute billing |
| Live drafts | Grey text updates while you speak, exact line replaces it when you pause |
| Drop-in module | Two callbacks and you're integrated |
| Hallucination guards | Whisper's silence-inventions and prompt-echo loops are filtered |
git clone https://github.com/DeveloperSarim/roman-urdu-speech-to-text.git
cd roman-urdu-speech-to-text
pip install -r requirements.txtFull setup with a virtualenv (recommended)
python3.11 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txtDependencies are faster-whisper, sounddevice, webrtcvad-wheels, numpy, and
pyobjc-framework-AVFoundation (macOS diagnostics only). No PyTorch — this uses
CTranslate2, which is far lighter on CPU.
Models download themselves on first run (~600 MB) into ~/.cache/huggingface.
Requirements: Python 3.9+ (built and tested on 3.11), a working microphone, ~1 GB disk for models, ~1 GB free RAM.
Platform note: developed and tested on macOS 15 (Intel). Every dependency is cross-platform, so Linux and Windows should work — but they're untested, and the macOS-specific fixes in Troubleshooting won't apply there.
python transcribe.pySpeak. Grey text appears while you talk, the accurate line lands when you pause, and
everything is appended to transcript.txt. Ctrl+C to stop — it waits for a phrase that's
still decoding so your last sentence isn't lost.
Mic not working? python mic_check.py tests three independent capture backends and prints
a plain verdict. See Troubleshooting.
Grab transcribe.py — that's the whole module. One file, no package to install, no
__init__.py needed.
curl -O https://raw.githubusercontent.com/DeveloperSarim/roman-urdu-speech-to-text/main/transcribe.py
pip install faster-whisper sounddevice webrtcvad-wheels numpyyour-project/
├── transcribe.py ← this file
└── your_app.py
from transcribe import Transcriber
with Transcriber(on_final=lambda text: print("Heard:", text)) as t:
t.wait()That's it. on_final fires once per completed phrase with clean, accurate text.
start() returns immediately and everything runs on background threads, so your app's own
loop keeps going:
from transcribe import Transcriber
commands = []
t = Transcriber(on_final=commands.append)
t.start() # returns as soon as models are loaded
while my_app_is_running():
if commands:
handle(commands.pop(0))
do_other_work()
t.stop()def draft(text): print(f"\r\033[2K {text}", end="", flush=True) # grey, overwrite
def final(text): print(f"\r\033[2K> {text}") # committed line
with Transcriber(on_final=final, on_draft=draft) as t:
t.wait()on_draft fires repeatedly with a rough guess while you're still speaking and is meant to
be overwritten in place, not appended. Skip it if you only want finished text —
live_preview=False also frees a whole model's worth of RAM and CPU.
t = Transcriber()
print(t.transcribe_file("meeting.wav")) # no microphone involvedAccepts any format ffmpeg can read, and any sample rate — it resamples internally.
Nothing is ever printed. Every result reaches you through the callbacks.
| Callback | Fires | Give it |
|---|---|---|
on_final(text) |
Once per phrase, after you pause | The text you actually want |
on_draft(text) |
Repeatedly while you speak | A live preview to overwrite in place |
Callbacks run on a background thread. If your handler touches a GUI or a non-thread-safe
object, push the text onto a queue.Queue and consume it from your main thread.
| Method | Description |
|---|---|
start() |
Load models, open mic, begin. Blocks a few seconds while loading, then returns self. Everything after runs in background threads |
stop(drain=True) |
Stop capturing. With drain=True waits up to 15 s for a phrase mid-decode so it isn't lost |
wait() |
Block until stop() or Ctrl+C. For CLI-style apps |
transcribe_file(path) |
One-shot decode of an audio file. Returns a string. No mic needed |
with Transcriber(...) as t: |
Context manager — calls start() and stop() for you |
from transcribe import list_microphones, DEFAULT_SEED
list_microphones() # [(0, 'MacBook Air Microphone', 48000)]Pass a device index as Transcriber(device=2) to pick a specific mic.
Voice commands → your own handler
import sys
from transcribe import Transcriber
def route(text):
low = text.lower()
if "whatsapp" in low: open_whatsapp() # your own function
elif "chatgpt" in low: open_chatgpt() # your own function
elif "band karo" in low: sys.exit()
else: print("Unknown:", text)
with Transcriber(on_final=route) as t:
t.wait()Feed speech straight into an LLM
import anthropic
from transcribe import Transcriber
client = anthropic.Anthropic()
def ask(text):
reply = client.messages.create(
model="claude-sonnet-5",
max_tokens=512,
messages=[{"role": "user", "content": text}],
)
print(reply.content[0].text)
with Transcriber(on_final=ask) as t:
t.wait()Stream to a browser over WebSocket
import asyncio, json, websockets
from transcribe import Transcriber
loop = asyncio.get_event_loop()
clients = set()
def broadcast(kind):
def send(text):
msg = json.dumps({"type": kind, "text": text})
for ws in list(clients):
asyncio.run_coroutine_threadsafe(ws.send(msg), loop)
return send
async def handler(ws):
clients.add(ws)
try:
await ws.wait_closed()
finally:
clients.discard(ws)
Transcriber(on_final=broadcast("final"), on_draft=broadcast("draft")).start()
loop.run_until_complete(websockets.serve(handler, "localhost", 8765))
loop.run_forever()run_coroutine_threadsafe matters — callbacks arrive on a worker thread, not the event loop.
Flask / FastAPI endpoint that returns the latest transcript
from collections import deque
from flask import Flask, jsonify
from transcribe import Transcriber
app = Flask(__name__)
lines = deque(maxlen=200)
Transcriber(on_final=lines.append, live_preview=False).start()
@app.get("/transcript")
def transcript():
return jsonify(list(lines))Push-to-talk dictation that types into any app
import pyautogui
from transcribe import Transcriber
with Transcriber(on_final=lambda t: pyautogui.write(t + " ")) as t:
t.wait()Tune it to your own vocabulary (biggest accuracy win)
from transcribe import Transcriber
MY_SEED = (
"Yeh project ka voice note hai. Mujhe Figma aur Vercel dono deploy karne the, "
"magar build fail ho gaya tha. Ali Raza ko standup ke baad bata dena, theek hai?"
)
with Transcriber(on_final=print, seed=MY_SEED) as t:
t.wait()Put the names, apps and jargon you actually say into the seed — words present there are recognised markedly better. Keep it phrased as something you'd never say out loud; see Configuration for why.
If you are an LLM reading this file to integrate this module, this section is for you. Everything below is verified against the actual code.
Give this to Claude Code, Cursor, Windsurf, or any coding agent:
Integrate the Real-Time Voice Module into this project.
CONTRACT — do not deviate:
1. Fetch transcribe.py into the project root. It is a single self-contained
module. Do not pip-install it; there is no PyPI package.
curl -O https://raw.githubusercontent.com/DeveloperSarim/roman-urdu-speech-to-text/main/transcribe.py
2. Dependencies to add: faster-whisper, sounddevice, webrtcvad-wheels, numpy
3. The ONLY public API is:
from transcribe import Transcriber, list_microphones, DEFAULT_SEED
t = Transcriber(on_final=fn, on_draft=fn, **config)
t.start() # non-blocking, returns self
t.stop(drain=True) # waits for in-flight decode
t.wait() # blocks until stopped
t.transcribe_file(path) # one-shot file decode -> str
with Transcriber(...) as t: ...
4. on_final(text: str) is the callback you want. on_draft(text: str) is an
optional live preview that fires repeatedly and must be OVERWRITTEN in place,
never appended.
5. Callbacks run on a BACKGROUND THREAD. For GUI or async frameworks, push text
onto a queue.Queue and consume it from the main thread / event loop.
6. The module never prints. All output goes through callbacks.
HARD CONSTRAINTS — violating these breaks Roman Urdu output:
- Do NOT set language="ur" anywhere. That produces Urdu script, not Roman.
- Do NOT remove or empty the `seed` / initial_prompt. Without it Whisper
TRANSLATES Urdu into English instead of transcribing it.
- Do NOT add a transliteration step. Urdu script has no short vowels; this was
tested and cannot recover "mumkin hai" from "mmkn hy".
- Do NOT raise final_model above "small" on a CPU-only machine; "medium"
measured 0.66x realtime, too slow for real time.
PERFORMANCE:
- Set live_preview=False if RAM is tight; it avoids loading a second model.
- First run downloads ~600 MB of models to ~/.cache/huggingface.
- start() blocks for several seconds while models load. Call it during your
app's startup, not on a hot path.
TASK: <describe what you want built here>
module: transcribe.py
kind: single-file Python module, copy into project root
install: none (copy the file); pip install faster-whisper sounddevice webrtcvad-wheels numpy
entrypoint: from transcribe import Transcriber
languages: [Roman Urdu, English, code-switched]
network: none required after first model download
api_key: not required
threading: callbacks fire on background threads; not thread-affine safe for GUI
public_api:
Transcriber:
constructor: (on_final=None, on_draft=None, **config)
methods: [start, stop, wait, transcribe_file]
context_manager: true
functions: [list_microphones, DEFAULT_SEED]
config_defaults:
final_model: small
draft_model: base
live_preview: true
seed: DEFAULT_SEED
silence_ms: 500
preview_ms: 700
min_speech_ms: 300
max_phrase_s: 15
vad_level: 2
min_level: 0.003
final_beam: 5
device: null
context_lines: 3
forbidden:
- setting language to "ur"
- removing the seed / initial_prompt
- post-hoc Urdu-to-Roman transliterationEvery option is a constructor keyword: Transcriber(silence_ms=400, vad_level=3).
| Option | Default | What to do with it |
|---|---|---|
seed |
DEFAULT_SEED |
The biggest accuracy lever. See below |
final_model |
"small" |
"medium" is more accurate but ran at 0.66× realtime — too slow. "base" is ~3× faster and clearly worse (kholng, firme) |
draft_model |
"base" |
Only used when live_preview=True |
live_preview |
True |
False drops grey drafts, frees a model's RAM and CPU. Turn this off first if things feel slow |
final_beam |
5 |
1 is ~25% faster and slightly worse (mein for main). 2–5 measured about equal |
silence_ms |
500 |
Pause that ends a phrase. Lower = snappier but chops mid-sentence pauses |
preview_ms |
700 |
How often the grey draft refreshes |
min_speech_ms |
300 |
Blips shorter than this are discarded, so coughs don't become lines |
vad_level |
2 |
0–3. Raise to 3 if background noise keeps triggering it |
min_level |
0.003 |
Near-silent clips are dropped unheard — Whisper hallucinates on them |
max_phrase_s |
15 |
Hard cap per phrase. Also sets the decode window, so lowering it speeds decoding |
device |
None |
Mic index from list_microphones() |
context_lines |
3 |
How many of your own recent lines feed back as context |
The seed is prose, not a word list — measured. Sentences teach spelling in context
(Mujhe/dunga instead of mujhi/doonga); loose words can't. Put your own names, apps
and jargon in it.
One rule: phrase it as something you would never say out loud. Whisper echoes its prompt back when audio is unclear, and the module drops anything matching the seed — so if the seed resembles your normal speech, your real words get dropped with it.
flowchart LR
A["🎙️ Mic<br/>48 kHz"] --> B["Downsample<br/>16 kHz"]
B --> C["WebRTC VAD<br/>30 ms frames"]
C --> D["Phrase builder<br/>300 ms pre-roll<br/>ends on 500 ms silence"]
D -->|"while speaking"| E["Draft decode<br/>base · beam 1"]
D -->|"on pause"| F["Final decode<br/>small · beam 5"]
E --> G["on_draft()"]
F --> H["Guards"]
H --> I["on_final()"]
I -.->|"last 3 lines<br/>as context"| F
Why whole phrases, not fixed slices. Whisper is dramatically more accurate given a complete utterance than an arbitrary 1-second chunk. VAD finds the natural boundaries, and a 300 ms pre-roll buffer means your first syllable isn't clipped off.
Why two models. small is accurate but slow; base runs ~3× faster. The fast one
gives you something on screen while you're still talking, the accurate one overwrites it
when you pause. Drafts are skipped while a final is decoding — on 2 physical cores they
fight for CPU and both get slower.
The guards. Whisper writes its own prompt back out when audio is unclear, so the seed
appeared verbatim in transcripts — and because finalized lines feed back as context, one
echo compounded into Haan bhai. Haan bhai. Haan bhai. Haan bhai. Three filters stop it,
and anything they catch is also kept out of the rolling context so it can't snowball:
| Guard | Catches |
|---|---|
echoes() |
Output is a 30+ char chunk of the seed |
looks_looped() |
Half or more of the words are repeats |
| duplicate check | Identical to the previous line |
min_level |
Near-silent audio, before it's ever decoded |
Measured on the weakest reasonable target: Intel Core i3-1000NG4 @ 1.1 GHz, 4 cores, 8 GB RAM (MacBook Air 2020, no GPU). Anything newer will be faster.
| Model | Speed | Verdict |
|---|---|---|
tiny |
7.1× realtime | Too inaccurate |
base |
8.0× realtime | Used for live drafts |
small |
2.7× realtime | Used for final lines |
medium |
0.66× realtime | Too slow for real time |
chunk_length pinned to max_phrase_s instead of Whisper's default 30 s — verified
byte-identical output on four test clips for ~25% less time. Whisper pads every clip to a
fixed window, so a 3-second phrase otherwise costs the same as a 30-second one.
On the 8 GB test machine with Chrome and an IDE open, swap hit 7.07 GB of 8 GB and load average hit 119. The same audio decoded in anywhere from 6 s to 89 s. No beam size or model choice competes with that. Close heavy apps, or set
live_preview=Falseso only one model is resident.
Run python mic_check.py first. It tests three independent capture backends and tells you
which works.
All three backends fail — CoreAudio is wedged (macOS)
This is the one that costs hours. The signature is the same FourCC error 'stop' surfacing
through every API:
| Backend | Symptom |
|---|---|
| sounddevice / PortAudio | AUHAL ... Audio Hardware Not Running, PaErrorCode -9986 |
| AVAudioEngine (Apple's own) | Code=1937010544 — that number is 'stop' in FourCC |
| ffmpeg / AVFoundation | Hangs forever; -t never fires because no samples arrive |
The mic still enumerates and permission still reads AUTHORIZED. Everything looks fine, nothing works. When even Apple's own AVAudioEngine can't start the input unit, it isn't your code or your permissions:
sudo killall coreaudiod # all audio stops ~2s, macOS restarts it automaticallyThen re-run mic_check.py. If it still fails, reboot.
No permission prompt ever appears
Don't run it from an IDE's built-in terminal. A terminal inside VS Code / Cursor / Antigravity is a child of the IDE's plugin helper process, which has no audio-input entitlement — macOS never even shows a prompt. Use Terminal.app.
Otherwise: System Settings → Privacy & Security → Microphone, enable your terminal, then quit it completely (Cmd+Q) and reopen. Permissions are only read at launch.
"Could not open the microphone" / 16 kHz refused
Built-in Mac mics often refuse a 16 kHz stream outright while accepting 44.1/48 kHz. The module already handles this — it captures at the device's native rate and downsamples. If you still see the error, list devices and pass one explicitly:
from transcribe import list_microphones, Transcriber
print(list_microphones())
Transcriber(device=1, on_final=print).start()It writes lines when I'm not speaking
Whisper invents phrases like "Thank you." on near-silence. Common junk is filtered
already. If more gets through, raise min_level (e.g. 0.006) and vad_level=3.
A line I really said got dropped
The echo guard thought it was the seed talking. Reword seed so it shares less phrasing
with your normal speech.
It's slow
In order of impact: close Chrome and other RAM hogs → live_preview=False →
final_beam=1 → final_model="base". See the RAM warning in Benchmarks.
Built by DeveloperSarim
Standing on faster-whisper · CTranslate2 · OpenAI Whisper · WebRTC VAD · sounddevice
If this saved you time, a ⭐ costs nothing.
MIT © DeveloperSarim — see LICENSE.
Use it, ship it, sell it. Just keep the copyright notice.