Reusable fine-tuning and evaluation infrastructure for Akan-language speech systems. The active ASR target is an expressive Akan recognizer: stronger word transcription plus punctuation, pause, emphasis, and stretch capture where the data supports it.
The goal is commercial-grade speech infrastructure for domains such as health, ecommerce, customer support, family care, and local voice agents. This repo is intentionally separate from the hackathon app so experiments, datasets, checkpoints, and Modal jobs do not bloat product code.
- Audit Waxal Akan ASR and freeze a reproducible, speaker-balanced benchmark.
- Normalize Akan text consistently and inspect data quality before training.
- Evaluate existing baselines:
teckedd/whisper_small-waxal_akan-asr-v1- MMS ASR variants
- any strong public Whisper/Akan checkpoint we validate
- Publish the experimental Round 2 model with its failed-promotion limitations.
- Build ASR Milestone 1: correction capture, punctuation restoration, and expressive prosody tags.
- Build commercially usable Asante Twi TTS after ASR Milestone 1 has a clean feedback loop.
- Run Modal only for durable gated jobs; one container maximum and scale-to-zero.
configs/ Training and dataset configs.
data/ Local raw/processed data and manifests; gitignored.
docs/ Pipeline notes, dataset cards, Modal usage, eval protocol.
modal_jobs/ Modal GPU entrypoints.
src/ Reusable Python package.
scripts/ CLI scripts for prep, smoke tests, and publishing.
evals/ Small local samples and generated reports.
tests/ Unit tests for normalization and manifest logic.
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
python scripts/smoke_test.py
pytestRun the Gradio control console:
python app.pyOpen http://127.0.0.1:7862.
The UI can:
- build a fixed Waxal Akan sample pack,
- download the sample audio locally,
- dry-run model evaluation,
- cache a Hugging Face ASR model,
- run a small ASR eval,
- transcribe microphone/upload audio with three published checkpoints,
- render WER/CER reports,
- operate the ordered TTS corpus, smoke, overfit, pilot, and full-run gates.
Default workflow is local and CPU-safe. Modal GPU jobs should only run after:
- the dataset manifest validates,
- a small eval sample is ready,
- the target baseline is recorded,
- the expected output path and Hugging Face repo are known.
The immediate baseline is teckedd/whisper_small-waxal_akan-asr-v1. On this repo's frozen 99-row, 33-speaker Waxal benchmark it scores 33.62% WER / 12.37% CER with a 30.91%-36.79% bootstrap WER interval. No forced language and the checkpoint's stored Yoruba prompt produce identical predictions; English prompting is invalid for this model. The first candidate therefore uses no forced language token.
The first evidence-driven continuation run improved the complete 1,123-row validation split from 32.69% to 31.45% WER at step 300. Step 400 regressed to 31.83%. On all 1,522 held-out Waxal test rows, the selected checkpoint improves 33.84% to 32.77% WER and 12.74% to 12.47% CER. A 5,000-sample paired bootstrap gives 99.86% probability of improvement and a -1.90 to -0.33 point 95% WER-difference interval. The experimental candidate remains gated because two outputs entered severe repetition loops and Ghanaian listening review is pending.
Round 2 reached 32.84% WER / 11.79% CER on the immutable 1,522-row Waxal test, compared with 34.32% for the original and 33.66% for the continuation under the same harness. It is published as teckedd/whisper-small-waxal-round2-specaug-v1, but is explicitly experimental because one repetition collapse and the speaker-4430 regression failed the promotion gate. The next ASR milestone is not another blind run: it adds correction capture, punctuation restoration, and expressive tags while keeping WER evaluation separate.
The first broader Waxal+GhanaNLP training attempt, v0.2, is published for traceability at teckedd/serendepify-gsl-asr-ak-waxal-gnlp-whisper-small-balanced-fullft-v0.2, but it is not promoted. It trained successfully, but worsened dev WER from 44.32% to 52.46%, likely due to small-set overfitting across 4.8 effective epochs. See ASR v0.2 Result.
The v0.3 broader low-learning-rate frozen-encoder retry is published for traceability at teckedd/serendepify-gsl-asr-ak-waxal-gnlp-whisper-small-broad-lowlr-freezeenc-fullft-v0.3, but it is not promoted. It barely moved mixed dev WER from 45.44% to 45.18%, regressed Waxal from 34.54% to 35.84%, and left GhanaNLP unusable at 96.92% WER. See ASR v0.3 Result.
The v0.4 Waxal-only low-learning-rate frozen-encoder run is published for traceability at teckedd/serendepify-gsl-asr-ak-waxal-whisper-small-only-lowlr-freezeenc-fullft-v0.4, but it is not promoted. On its 1,024-row Waxal dev slice, the base Round 2 model scored 31.88% WER and the v0.4 checkpoint regressed to 34.09% WER, so the regression gate stopped the run at checkpoint 200. See ASR v0.4 Result.
The v0.5 GhanaNLP-only diagnostic is published for traceability at teckedd/serendepify-gsl-asr-ak-gnlp-whisper-small-only-lowlr-freezeenc-fullft-v0.5, but it is not promoted. It improved GhanaNLP validation WER from 113.97% to 87.55%, proving the dataset has learnable signal, but it remains far from usable and regressed Waxal from 32.59% to 35.75% on the regression slice. See ASR v0.5 Result.
The follow-up GhanaNLP manifest audit keeps 6,704 / 11,830 rows as first-pass clean candidates and identifies 3,088 too-short clips, 2,162 suspicious word/audio-speed rows, and 2,038 duplicate-transcript rows. GhanaNLP should now be used as cleaned adaptation data with Waxal replay, not as raw single-corpus fine-tuning. See GhanaNLP manifest audit v0.5.
The next planned trainable input is v0.6: serendepify-gsl-asr-ak-waxal-gnlpclean-whisper-small-replay-fullft-v0.6. It uses Waxal as replay/regression evidence, only clean-candidate GhanaNLP rows for adaptation, and excludes GhanaNLP test rows from the manifest. See ASR v0.6 manifest plan.
- Roadmap and handoff
- Artifact versioning
- ASR pipeline loop
- ASR Milestone 1: Expressive Akan Recognition
- ASR research spine
- ASR v0.2 result
- ASR v0.3 result
- ASR v0.4 result
- ASR v0.5 result
- GhanaNLP manifest audit v0.5
- ASR v0.6 manifest plan
- Asante Twi TTS research and execution record
- Contamination-safe ASR Round 2 specification
- Dataset plan
- Preprocessing strategy
- Experiment plan
- Evaluation protocol
- Frozen benchmark
- Modal usage
- Old Waxal notebook findings