Skip to content

πŸ”„ [倄理中][wt-audio8-asr] feat: Add Audio8-ASR-0.1B community model port (audio8_asr)Β #336

Description

@gqf2008

Goal

Add native audio.cpp support for Audio8/Audio8-ASR-0.1B (HF: Audio8/Audio8-ASR-0.1B) as community family audio8_asr.

Audio8-ASR-0.1B is a compact multilingual ASR model (en/zh/yue/ja/ko/fr/de): Qwen3-ASR-0.6B audio encoder + 4-layer MLP tower adapter + 8-layer Qwen2-style decoder (hidden 512, tied embeddings, vocab 151936). 324M params total, 648 MB bf16. Self-reported Open ASR Leaderboard 7-split mean WER 7.03% (EN), WenetSpeech net CER 7.98 (ZH). Offline, ≀30 s, greedy.

Scope

  • Python reference environment + golden parity outputs (torch CPU, transformers, trust_remote_code)
  • GGUF converter tools/community_models/convert_audio8_asr.py (safetensors β†’ GGUF, metadata-embedded configs)
  • C++ community family src/community_models/audio8_asr/ (loader/assets/session/prompt/tower adapter/decoder)
  • Reuse qwen3_asr audio encoder + Whisper 128-mel frontend (configs are bit-identical)
  • Decoder via shared QwenCausalDecoder with use_qk_norm=false (Qwen2 style), tied lm_head
  • model_specs/audio8_asr.json β€” offline-only modes, local-conversion-only download (CC-BY-NC-4.0, follow mms_forced_aligner precedent: no redistribution)
  • Registration: CMakeLists audiocpp_add_model, loader catalog sync check, README supported-model table, docs/community_models/audio8_asr.md
  • Unit tests + parity evidence vs Python reference

Out of scope (phase 1)

  • Hotword logit boosting (needs decode-time logit-bias hook; later phase)
  • Word timestamps, streaming mode
  • GGUF redistribution (license is CC-BY-NC-4.0 β€” conversion is local-only)

Acceptance criteria

  • audiocpp_cli --list-loaders --json includes audio8_asr; python3 tools/check_loader_catalog_sync.py passes
  • Converted GGUF loads and transcribes a sample wav; output matches Python reference (greedy, 30 s window) exactly or with documented tolerance
  • Build + unit tests green locally (macOS, CPU/Metal backend)
  • Parity/build/run commands documented in the PR

References

  • HF model card: https://huggingface.co/Audio8/Audio8-ASR-0.1B (CC-BY-NC-4.0)
  • Architecture notes: whisper 128-mel frontend β†’ Qwen3-ASR encoder (896/18L/14H/ffn3584) β†’ 4Γ— pre-LN residual MLP tower (1024β†’4096, GELU) β†’ avg-pool merge_factor 4 β†’ LN+Linear(1024β†’512) β†’ set_rows at <|audio|> (151646); prompt <|user|><|begin_of_audio|><|audio|>Γ—N<|end_of_audio|>…<|assistant|>, N=((samples/160+1)//2)//4
  • Related but non-overlapping: PR feat: Supporting Audio8_TTS modelsΒ #333 (audio8_tts, TTS only)
  • NC-license precedent: model_specs/mms_forced_aligner.json (local-only conversion)

Metadata

Metadata

Assignees

No one assigned

    Labels

    new modelRequest for new model support

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions