Speaker Transcriber LiteRT

A fully local Python pipeline for speaker diarization followed by word-timestamped speech transcription.

The pipeline decodes audio or video with PyAV, normalizes it to mono 16 kHz, runs speaker diarization first, transcribes with Parakeet-TDT-v3 second, aligns every word to one speaker, repairs likely boundary drift, reconstructs readable speaker turns, and exports six transcript formats.

Pipeline

Audio or video
    ↓
PyAV decode and mono 16 kHz normalization
    ↓
Pyannote ONNX segmentation
    ↓
WeSpeaker LiteRT embeddings and speaker clustering
    ↓
Parakeet-TDT-v3 LiteRT transcription
    ↓
Word-to-speaker alignment
    ↓
Sentence-aware boundary repair
    ↓
Sentence and utterance reconstruction
    ↓
TXT · JSON · Markdown · SRT · VTT · RTTM

Project status

Version 0.1.0 is an alpha-quality implementation. The orchestration, alignment, repair, reconstruction, exporters, batch CLI, and unit-testable model interfaces are implemented.

Real inference is implemented for the five-second, multi-signature Parakeet model published by litert-community. The runtime calls its encode and decode signatures through the LiteRT Python API and reconstructs word timestamps from TDT token-duration emissions.

Requirements

• Python 3.10–3.12
• Local CPU inference, with optional GPU acceleration
• No cloud service
• No external ffmpeg executable
• Sufficient memory and disk space for the selected models

Installation

git clone https://github.com/jcolozzi/speaker-transcriber-litert.git
cd speaker-transcriber-litert

python -m venv .venv

Activate the environment:

# Windows PowerShell
.venv\Scripts\Activate.ps1

# macOS/Linux
source .venv/bin/activate

Install:

python -m pip install --upgrade pip
pip install -e .

For development:

pip install -e ".[dev]"

Model files

Place these files in models/:

models/
├── pyannote_seg30.onnx
├── wespeaker_emb_fp16.tflite
├── parakeet_tdt_0.6b_v3_5s_i8.tflite
└── tokenizer.model

Download them with the Hugging Face CLI:

hf download litert-community/parakeet-tdt-0.6b-v3 \
  parakeet_tdt_0.6b_v3_5s_i8.tflite --local-dir models
hf download litert-community/Speaker-Diarization-LiteRT \
  pyannote_seg30.onnx wespeaker_emb_fp16.tflite --local-dir models
hf download spybyscript/parakeet-tdt-litert \
  tokenizer.model --local-dir models

See MODEL_SETUP.md for model contracts and licenses.

Run the environment check before processing media:

speaker-transcriber doctor

Command-line usage

Transcribe one file:

speaker-transcriber transcribe meeting.mp4

The transcribe command is optional for the common case:

speaker-transcriber meeting.mp4

Select formats and rename speakers:

speaker-transcriber meeting.mp4 \
  --formats txt,json,md,srt,vtt,rttm \
  --speaker-name SPEAKER_00=John \
  --speaker-name SPEAKER_01=Guest

Process a directory:

speaker-transcriber transcribe recordings/ --recursive

Continue after failed files by default, or stop immediately:

speaker-transcriber recordings/ --recursive --fail-fast

Use more permissive boundary repair:

speaker-transcriber meeting.wav --repair-mode aggressive

Write detailed intermediate diagnostics:

speaker-transcriber meeting.wav --debug

Desktop GUI

Launch the modern CustomTkinter interface with either command:

speaker-transcriber gui
speaker-transcriber-gui

The GUI provides media/model/output pickers, selectable export formats, CPU thread settings, an adaptive light/dark theme, and live pipeline stage updates. Inference runs on a worker thread so the window remains responsive.

GPU acceleration

auto is the default accelerator mode. It asks LiteRT to use the GPU for supported operations and permits CPU execution for unsupported operations. If the GPU runtime cannot initialize, the app logs the reason and automatically uses the CPU interpreter instead.

On Windows, install Microsoft's DirectX Shader Compiler runtime once:

powershell -ExecutionPolicy Bypass -File scripts/setup_windows_gpu.ps1

Then use the default behavior, explicitly request the GPU-preferred path, or force CPU-only execution:

speaker-transcriber meeting.wav --accelerator auto
speaker-transcriber meeting.wav --accelerator gpu
speaker-transcriber meeting.wav --accelerator cpu

The startup log reports GPU, GPU with CPU fallback, or CPU independently for Parakeet and WeSpeaker. The GUI exposes the same choices.

With the current LiteRT Windows WebGPU runtime, the linked quantized Parakeet graph cannot be safely compiled, so the app keeps Parakeet on CPU and accelerates WeSpeaker on the GPU. This is an automatic per-model fallback, not a failed GPU setup. Other platforms are attempted normally.

Programmatic usage

from speaker_transcriber import (
    PipelineConfig,
    SpeakerTranscriptionPipeline,
)

config = PipelineConfig(
    model_dir="models",
    output_dir="output",
    temp_dir="temp",
)

pipeline = SpeakerTranscriptionPipeline(config)

result = pipeline.process(
    "meeting.mp4",
    speaker_names={
        "SPEAKER_00": "John",
        "SPEAKER_01": "Guest",
    },
)

print(result.transcript.utterances)

for path in result.exports.paths():
    print(path)

Batch processing:

batch = pipeline.process_batch(
    ["meeting-1.mp4", "meeting-2.wav"],
    continue_on_error=True,
)

print(batch.succeeded, batch.failed)

Output formats

Format	Purpose
TXT	Readable timestamped speaker transcript
JSON	Complete words, segments, sentences, utterances, metadata, and timings
Markdown	Human-readable transcript for documentation
SRT	Subtitle captions with comma millisecond timestamps
VTT	Web captions with period millisecond timestamps
RTTM	Speaker diarization timeline

Boundary repair

The default conservative repair engine evaluates short speaker spans using:

• surrounding speaker continuity
• pauses
• sentence and clause punctuation
• phrase length and duration
• protected conversational responses
• word confidence
• optional diarization confidence

Every accepted repair can be inspected through the audit records or debug/boundary_repair.json.

Testing

The lightweight unit suite does not require model binaries:

pip install -r requirements-test.txt
pytest

An end-to-end test can be reproduced with the four-minute public mock-interview fixture documented in SAMPLE_TEST.md. Model binaries and downloaded media stay outside version control.

The PyAV integration test runs when PyAV and SoundFile are installed.

Repository structure

speaker_transcriber/
├── alignment.py
├── audio.py
├── cli.py
├── config.py
├── diarization.py
├── exporters.py
├── logging_config.py
├── models.py
├── pipeline.py
├── repair.py
├── sentence_builder.py
├── transcription.py
├── utils.py
└── validation.py

Known limitations

• The published Parakeet model accepts fixed five-second windows. Longer audio is processed with overlapping chunks and timestamp-aware deduplication.
• GPU support depends on LiteRT's operator and platform coverage. Mixed GPU/CPU execution and full CPU fallback are supported by the app.
• Overlapping speech receives one speaker per transcribed word; simultaneous multi-speaker word output is not represented.
• Speaker names are aliases supplied by the user, not biometric identification.
• The repair engine is heuristic and should not be treated as forensic proof of speaker identity.

License

MIT