Add Xiaomi MiMo-V2.5-ASR runner (closes #150)

[AlexZhangji]

Summary

Adds a runner for XiaomiMiMo/MiMo-V2.5-ASR, an end-to-end multilingual ASR model jointly optimized for Mandarin Chinese and English. Strong support for Chinese dialects (Wu / Cantonese / Hokkien / Sichuanese), code-switching, song lyrics, noisy / multi-speaker audio. Architecture: 1.2B audio tokenizer (RVQ 8-layer, 25 Hz) + 7B Qwen2 LM decoder.

Closes #150.

Setup

MiMo's package is not pip-installable; it must be cloned and added to PYTHONPATH. Full instructions in mimo/README.md. Headline:

git clone https://github.com/XiaomiMiMo/MiMo-V2.5-ASR.git ../MiMo-V2.5-ASR
cd ../MiMo-V2.5-ASR && pip install -r requirements.txt && pip install flash-attn==2.7.4.post1 && cd -
hf download XiaomiMiMo/MiMo-V2.5-ASR        --local-dir ./models/MiMo-V2.5-ASR
hf download XiaomiMiMo/MiMo-Audio-Tokenizer --local-dir ./models/MiMo-Audio-Tokenizer
export MIMO_REPO_PATH=$(realpath ../MiMo-V2.5-ASR)
bash run_mimo.sh

flash_attn_shim.py is a small SDPA-based fallback the runner auto-injects only when real flash_attn import fails (Windows native, exotic torch+CUDA combos). Linux + flash-attn hosts use the canonical path unchanged.

Result on Earnings22 (test, full set)

Dataset	Samples	Audio	WER	RTFx
earnings22	2737	5.43 h	13.75%	6.81

Hardware: NVIDIA RTX 4090 (24 GB VRAM, bf16 inference). MiMo's 8B bf16 weights (~16 GB) fit comfortably; A100-80GB (the leaderboard's reference HW) is fully supported.

Hyperparameters held identical to the runner's defaults — no per-dataset tuning. The runner accepts --dataset for any of ami / earnings22 / gigaspeech / librispeech / spgispeech / tedlium / voxpopuli with the same hyperparameters.

Remaining datasets

Submitting Earnings22 only for now. Per the README's "request a maintainer to run" policy, happy to either run the remaining 7 ESB short-form datasets myself or defer to maintainer evaluation — let me know which is preferred.

Notes on Earnings22 ground-truth quality

A spot-check of MiMo's 10 worst-WER samples (all <4 s utterances) found audio-to-reference misalignment in 10/10 cases, independently confirmed by re-transcribing each clip with Gemini 3.1 Pro. Example:

GT: okay (1.98 s clip)
MiMo: 2 maybe 2.5
Gemini: two maybe two and a half

This is consistent with #153 and the cleaned variant published by Artificial Analysis (Earnings22-Cleaned-AA), which reports an average 5.6 p.p. WER reduction across all models on the cleaned set. The 13.75% reported here is on the standard (uncleaned) leaderboard set for direct comparability with existing entries.

Test plan

• Smoke-tested 5 samples (WER 5.19% on this slice)
• Full Earnings22 test split (2737 samples, 13.75% WER, 6.81 RTFx)
• Manifest format matches eval_utils.write_manifest exactly (audio_filepath / duration / time / text / pred_text)
• Resume verified (interrupt + restart produces same final manifest)
• Hyperparameters identical for any --dataset choice
• Maintainer reproducibility check on A100-80GB (deferred)

[GLCLAP]

This test has a minor issue; the audio sampling rate needs to be converted. The fixed code is here: AlexZhangji#1

[Deep-unlearning]

Thanks for the submission! I ran the model on the 8 short-form English datasets (A100-80GB). Results below:

XiaomiMiMo/MiMo-V2.5-ASR — Average WER 8.22%

Dataset	WER %	RTFx
AMI	19.05	3.5
Earnings22	13.57	5.2
GigaSpeech	10.43	4.5
LibriSpeech Clean	2.17	5.6
LibriSpeech Other	6.83	5.4
SPGISpeech	3.28	5.9
TEDLIUM	3.20	4.5
VoxPopuli	7.20	6.5

Note the low RTFx — MimoAudio.asr_sft() only accepts one sample per call, so inference is sequential (no batching), as flagged in your README.

Do these numbers look right to you?

[GLCLAP]

The audio sampling rate needs to be converted. The default sampling rate for mimo is 24000. Earnings22 WER should be around 11% . The fixed code is AlexZhangji@1b16f74

[AlexZhangji]

Thanks for the submission! I ran the model on the 8 short-form English datasets (A100-80GB). Results below:

XiaomiMiMo/MiMo-V2.5-ASR — Average WER 8.22%

Dataset WER % RTFx
AMI 19.05 3.5
Earnings22 13.57 5.2
GigaSpeech 10.43 4.5
LibriSpeech Clean 2.17 5.6
LibriSpeech Other 6.83 5.4
SPGISpeech 3.28 5.9
TEDLIUM 3.20 4.5
VoxPopuli 7.20 6.5
Note the low RTFx — MimoAudio.asr_sft() only accepts one sample per call, so inference is sequential (no batching), as flagged in your README.

Do these numbers look right to you?

this looks good to me!