Release 0.5 (WIP)

- Add reusable framework modules for upcoming 0.5 models, including DramaBox, RVC, Confucius4-TTS, Kroko ASR, and related encoder/vocoder paths
- Refactor DramaBox, RVC, and Confucius4-TTS toward spec v1 backed model loading and shared framework components
- Add framework support for Zipformer, Gemma decoder, Wav2Vec2-BERT, RMVPE, BigVGAN extensions, attention/norm/activation modules, and text normalization
- Update model specs and path-test coverage for migrated/new models
- Update WebUI model handling so default downloads use GGUF packages from model specs while existing local tensor installs remain loadable
- Add WebUI catalog/download tests and model parameter entries for newly exposed models

Author: 0xShug0

CMakeLists.txt

@@ -253,6 +253,7 @@ add_library(engine_core OBJECT
     src/framework/text/chinese_normalization.cpp
     src/framework/text/chunking.cpp
     src/framework/text/subtitle.cpp
+    src/framework/text/text_normalization.cpp
     src/framework/text/unicode_normalization.cpp
     src/framework/io/yaml.cpp
     src/framework/io/binary.cpp
@@ -272,12 +273,15 @@ add_library(engine_core OBJECT
     src/framework/modules/structural_modules.cpp
     src/framework/modules/asr_helpers.cpp
     src/framework/modules/conv_modules.cpp
+    src/framework/modules/zipformer_modules.cpp
     src/framework/modules/speech_encoders/whisper_embedding.cpp
     src/framework/modules/speech_encoders/whisper_frontend.cpp
     src/framework/modules/speech_encoders/campplus_encoder.cpp
+    src/framework/modules/pitch_extractors/rmvpe_pitch_extractor.cpp
     src/framework/modules/vocoders/bigvgan_vocoder.cpp
     src/framework/modules/vocoders/hift_vocoder.cpp
     src/framework/modules/speech_encoders/hubert_encoder.cpp
+    src/framework/modules/speech_encoders/wav2vec2_bert_encoder.cpp
     src/framework/modules/speech_encoders/wavlm_encoder.cpp
     src/framework/modules/speech_encoders/sanm.cpp
     src/framework/modules/optimizations/fast_conv_modules.cpp
@@ -293,6 +297,7 @@ add_library(engine_core OBJECT
     src/framework/modules/attention/relative_attention.cpp
     src/framework/modules/attention/scaled_dot_product_attention.cpp
     src/framework/modules/attention/grouped_query_attention.cpp
+    src/framework/modules/attention/projected_grouped_self_attention.cpp
     src/framework/modules/attention/streaming_frame_attention_cache.cpp
     src/framework/modules/transformers/qwen_decoder.cpp
     src/framework/modules/transformers/qwen_causal_decoder.cpp
@@ -304,6 +309,7 @@ add_library(engine_core OBJECT
     src/framework/modules/positional_modules.cpp
     src/framework/modules/text_encoders/t5_base_encoder.cpp
     src/framework/modules/text_encoders/t5_gemma_encoder.cpp
+    src/framework/modules/transformers/gemma_decoder.cpp
     src/framework/modules/optimizations/fast_kv_modules.cpp
     src/framework/tokenizers/hf_tokenizer_json.cpp
     src/framework/tokenizers/llama_bpe.cpp
@@ -681,12 +687,11 @@ audiocpp_add_model(dramabox
         src/models/dramabox/gemma_tokenizer.cpp
         src/models/dramabox/gemma3_encoder.cpp
         src/models/dramabox/latent_state.cpp
-        src/models/dramabox/loader.cpp
         src/models/dramabox/prompt_connector.cpp
         src/models/dramabox/session.cpp
         src/models/dramabox/vocoder.cpp
     INCLUDES
-        engine/models/dramabox/loader.h
+        engine/models/dramabox/session.h
     LOADERS
         engine::models::dramabox::make_dramabox_loader
 )
@@ -1001,9 +1006,8 @@ audiocpp_add_model(rvc
         src/models/rvc/session.cpp
         src/models/rvc/synthesizer.cpp
         src/models/rvc/voice_model.cpp
-        src/models/rvc/loader.cpp
     INCLUDES
-        engine/models/rvc/loader.h
+        engine/models/rvc/session.h
     LOADERS
         engine::models::rvc::make_rvc_loader
 )
@@ -1497,6 +1501,13 @@ if (ENGINE_BUILD_TESTS)
         COMMAND unicode_normalization_test
     )
 
+    add_engine_unittest(text_normalization_test tests/unittests/test_text_normalization.cpp)
+
+    add_test(
+        NAME text_normalization_test
+        COMMAND text_normalization_test
+    )
+
     add_engine_unittest(outetts_generation_budget_test tests/unittests/test_outetts_generation_budget.cpp)
     target_include_directories(outetts_generation_budget_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/tests/unittests)
 

README.md

@@ -56,10 +56,12 @@ Runtime tags: safetensors is the default model loading path. `GGUF 16/Q8/Q4` mea
 
 | Family | Task | Lang | Variants | Runtime |
 |---|---|---|---|---|
-| **ace_step** | Music, Edit | 50+ langs | ACE-Step 1.5 Turbo and Base with acestep-5Hz-lm-1.7B | GGUF 16/Q8 |
+| **ace_step** | Music, Edit | 50+ langs | ACE-Step 1.5 Turbo and Base with acestep-5Hz-lm-1.7B | GGUF 16 |
 | **bs_roformer** | Sep | lang agnostic | BS-RoFormer vocal separation checkpoints | GGUF Q8 |
 | **chatterbox** | TTS, Clone, VC| ar, da, de, el, en, es, fi, fr, hi, it, ko, ms, nl, no, pl, pt, sv, sw, tr | Chatterbox with 0.5B backbone | GGUF 16/Q8 |
+| **confucius4_tts** | Clone | zh, en, ja, ko, de, fr, es, id, it, th, pt, ru, ms, vi | Confucius4-TTS multilingual voice cloning | GGUF F32, Stream |
 | **citrinet_asr** | ASR | en | Citrinet-256 | GGUF Q8 |
+| **dramabox** | TTS, Clone | en | DramaBox expressive TTS and voice cloning | GGUF Q8 |
 | **fish_audio** | TTS, Clone, Ctrl | auto, en, zh | Fish Audio S2 Pro | GGUF 16/Q8 |
 | **fun_asr_nano** | ASR | auto, zh, en, ja | Fun-ASR-Nano-2512 | GGUF 16/Q8 |
 | **heartmula** | Music | zh, en, ja, ko, es | HeartMuLa-oss-3B with HeartCodec-oss | GGUF 16/Q8 |
@@ -77,6 +79,7 @@ Runtime tags: safetensors is the default model loading path. `GGUF 16/Q8/Q4` mea
 | **qwen3_asr** | ASR | zh, en, yue, ar, de, fr, es, pt, id, it, ko, ru, th, vi, ja, tr, hi, ms, nl, sv, da, fi, pl, cs, fil, fa, el, ro, hu, mk | Qwen3-ASR-0.6B, Qwen3-ASR-1.7B-hf | GGUF 16/Q8 |
 | **qwen3_forced_aligner** | Align | zh, yue, en, de, es, fr, it, pt, ru, ko, ja | Qwen3-ForcedAligner-0.6B | GGUF 16/Q8 |
 | **qwen3_tts** | TTS, Clone, Design, Ctrl | zh, en, fr, de, it, ja, ko, pt, ru, es | Qwen3-TTS-12Hz-0.6B-Base, Qwen3-TTS-12Hz-1.7B-Base, Qwen3-TTS-12Hz-1.7B-CustomVoice, Qwen3-TTS-12Hz-1.7B-VoiceDesign | GGUF 16/Q8 |
+| **rvc** | VC | lang agnostic | RVC F16 GGUF with packaged v1/v2 voices and optional retrieval blending | GGUF 16 |
 | **seed_vc** | VC | lang agnostic | SeedVC XLS-R + HiFT, SeedVC Whisper-small + BigVGAN | GGUF 16/Q8 |
 | **silero_vad** | VAD | lang agnostic | Silero VAD | Bundled, Stream |
 | **sortformer_diar** | Diar | en | Sortformer-4spk-v1 | - |
@@ -626,7 +629,7 @@ For long-form TTS tests, each run uses the same 6,026-character, 1,028-word inpu
 
 ## Runtime Memory Options
 
-Some models expose memory-saver session options such as `ace_step.mem_saver=true`, `heartmula.mem_saver=true`, `stable_audio.mem_saver=true`, `omnivoice.mem_saver=true`, and `voxcpm2.mem_saver=true`. These options keep the default output path unchanged while reducing graph workspace VRAM or releasing staged graph/cache state after request phases; later requests may rebuild released graphs.
+Some models expose memory-saver session options such as `ace_step.mem_saver=true`, `dramabox.mem_saver=true`, `heartmula.mem_saver=true`, `stable_audio.mem_saver=true`, `omnivoice.mem_saver=true`, and `voxcpm2.mem_saver=true`. These options keep the default output path unchanged while reducing graph workspace VRAM or releasing staged graph/cache state after request phases; later requests may rebuild released graphs.
 
 ## Precision/Quantization Support
 

docs/asr.md

@@ -9,6 +9,7 @@
 | Higgs Audio STT | `higgs_audio_stt` | offline, streaming | [Higgs Audio STT](#higgs-audio-stt) |
 | Hviske ASR | `hviske_asr` | offline | [Hviske ASR](#hviske-asr) |
 | Nemotron ASR | `nemotron_asr` | offline, streaming | [Nemotron ASR](#nemotron-asr) |
+| Parakeet-TDT | `parakeet_tdt` | offline, streaming | [Parakeet-TDT](#parakeet-tdt) |
 | VibeVoice ASR | `vibevoice_asr` | offline | [VibeVoice ASR](#vibevoice-asr) |
 | Voxtral Realtime | `voxtral_realtime` | offline, streaming | [Voxtral Realtime](#voxtral-realtime) |
 
@@ -87,26 +88,23 @@ joiner, greedy search, and modified beam search natively without ONNX Runtime.
 Blank penalty, natural-text hotwords, and opt-in endpoint segmentation are
 available as request options. Public free packages
 are available for German, English, Spanish, French, Italian, Hebrew, Dutch,
-Portuguese, Swedish, and Turkish. Download the matching free Kroko Community
-`.data` package and convert it before use:
+Portuguese, Swedish, and Turkish. The model manager defaults to the standalone
+English Q8_0 GGUF package:
 
 ```powershell
-python .\tools\community_models\convert_kroko_onnx.py `
-  .\models\Kroko-ASR\Kroko-EN-Community-128-L-Streaming-001.data `
-  .\models\Kroko-ASR\Kroko-EN-Community-128-L-Native `
-  --overwrite
+python .\tools\model_manager.py install kroko_asr_community_q8_0 --models-root .\models --overwrite
 ```
 
 ```powershell
 .\build\windows-cuda-release\bin\audiocpp_cli.exe `
   --task asr --mode streaming --family kroko_asr `
-  --model .\models\Kroko-ASR\Kroko-EN-Community-128-L-Native `
+  --model .\models\Kroko-ASR-GGUF\kroko-en-community-64-l-q8_0.gguf `
   --backend cuda --audio .\speech.wav --language en `
   --text-out .\transcript.txt --words-out .\words.json
 ```
 
-Converted safetensors and standalone Q8 GGUF are supported in offline and
-stateful streaming modes. Partial transcripts and word timestamps are exposed.
+Standalone Q8 GGUF is supported in offline and stateful streaming modes. Partial
+transcripts and word timestamps are exposed.
 See [Kroko Community ASR](community_models/kroko_asr.md) for package selection,
 conversion, GGUF, decoding options, parity, performance, and limitation details.
 
@@ -250,6 +248,23 @@ audiocpp_cli --task asr --family nemotron_asr --model models/nemotron-3.5-asr-st
 | `--text-out` | TXT path | not set | Transcript output. The transcript is also printed to stdout. |
 | `--session-option nemotron_asr.mem_saver=true|false` | bool | `false` | Release the offline encoder graph after each offline request. |
 
+## Parakeet-TDT
+
+Parakeet-TDT is a FastConformer-TDT ASR model for multilingual offline,
+long-form, and buffered-streaming transcription. The model manager defaults to
+the standalone Q8_0 GGUF package.
+
+```bash
+python3 tools/model_manager.py install parakeet_tdt_q8_0 --models-root models
+audiocpp_cli --task asr --family parakeet_tdt \
+  --model models/Parakeet-TDT-0.6B-v3-GGUF/parakeet-tdt-0.6b-v3-q8_0.gguf \
+  --backend cuda --audio speech_16k.wav --text-out transcript.txt
+```
+
+Use `parakeet_tdt_f16` for the F16 GGUF variant. See
+[Parakeet-TDT 0.6B v3](community_models/parakeet_tdt.md) for long-form,
+streaming, conversion, options, validation, and performance details.
+
 ## VibeVoice ASR
 
 VibeVoice ASR is an offline ASR model with greedy, sampling, and beam-search decode paths. It can return transcription text and structured segment/speaker-turn output when the model produces timestamps.

docs/audio_tools.md

@@ -3,6 +3,7 @@
 | Model | Family | Task(s) | Quick Start |
 |---|---|---|---|
 | MioCodec | `miocodec` | `vc`, `s2s` | [MioCodec](#miocodec) |
+| RVC | `rvc` | `vc` | [RVC](#rvc) |
 | Seed-VC | `seed_vc` | `vc`, `svc` | [Seed-VC](#seed-vc) |
 | VeVo2 | `vevo2` | TTS, SVC, VC, editing | [VeVo2](#vevo2) |
 | HTDemucs | `htdemucs` | `sep` | [HTDemucs](#htdemucs) |
@@ -51,6 +52,62 @@ audiocpp_cli --task s2s --family miocodec --model models/MioCodec-25Hz-44.1kHz-v
 | `--out` | WAV path | required | Output audio path. |
 | `--session-option miocodec.weight_type=<type>` | `native`, `f32`, `f16`, `bf16`, `q8_0` | `native` | Model weight type when supported by each component. |
 
+## RVC
+
+RVC is an offline retrieval-based voice-conversion model. The audio.cpp package
+uses native HuBERT content features, RMVPE pitch extraction, optional IVF
+retrieval blending, and packaged v1/v2 voices. The F16 GGUF package is
+self-contained and embeds the package spec plus packaged retrieval sidecars.
+
+| Field | Value |
+|---|---|
+| Family | `rvc` |
+| GGUF model | `models/RVC-GGUF/rvc-f16.gguf` |
+| Task | `vc` |
+| Modes | `offline` |
+| Input | Source speech WAV through `--audio` |
+| Packaged voices | `default`, `manthos`, `chocola`, `fraise` |
+| Output | Single converted WAV through `--out` |
+
+Packaged voice without retrieval:
+
+```bash
+audiocpp_cli --task vc --family rvc --model models/RVC-GGUF/rvc-f16.gguf --backend cuda --audio source.wav --out converted.wav --request-option voice_id=default
+```
+
+Packaged voice with retrieval blending:
+
+```bash
+audiocpp_cli --task vc --family rvc --model models/RVC-GGUF/rvc-f16.gguf --backend cuda --audio source.wav --out converted.wav --request-option voice_id=default --request-option retrieval_blend=0.5
+```
+
+User RVC checkpoint with a matching FAISS retrieval index:
+
+```bash
+audiocpp_cli --task vc --family rvc --model models/RVC-GGUF/rvc-f16.gguf --backend cuda --audio source.wav --out converted.wav --request-option voice_model_path=/path/to/voice.pth --request-option retrieval_index_path=/path/to/voice.index --request-option retrieval_blend=0.5
+```
+
+| Option | Values | Default | Meaning |
+|---|---|---:|---|
+| `--request-option voice_id=<id>` | `default`, `manthos`, `chocola`, `fraise` | `default` | Packaged RVC voice id. Ignored when `voice_model_path` is set. |
+| `--request-option voice_model_path=<path>` | `.pth` or `.pt` path | unset | User RVC checkpoint path. |
+| `--request-option pitch_extractor=rmvpe` | `rmvpe` | `rmvpe` | Pitch extractor for F0-enabled voices. |
+| `--request-option pitch_path=<path>` | CSV path | unset | Optional F0 override file with `time,Hz` rows sorted by time. |
+| `--request-option retrieval_index_path=<path>` | FAISS `.index` path | unset | User retrieval index used when `retrieval_blend` is greater than 0 for a user checkpoint. |
+| `--request-option retrieval_blend=<rate>` | `0.0` to `1.0` | `0.0` | IVF retrieval feature blend rate. `0` disables retrieval. |
+| `--request-option semitone_shift=<n>` | integer | `0` | Semitone pitch shift before synthesis. |
+| `--request-option pitch_filter_radius=<n>` | integer >= 0 | `3` | Median filter radius for F0 smoothing; values greater than 2 enable filtering. |
+| `--request-option output_sample_rate=<hz>` | integer >= 0 | `0` | Output sample rate; `0` keeps the selected voice model sample rate. |
+| `--request-option rms_mix_rate=<rate>` | float | `0.25` | RMS envelope mix rate after conversion. |
+| `--request-option unvoiced_protection=<rate>` | `0.0` to `1.0` | `0.33` | Unvoiced consonant protection strength. |
+| `--request-option speaker_id=<n>` | integer >= 0 | `0` | Speaker embedding id for multi-speaker checkpoints. |
+| `--request-option audio_pad_duration_sec=<sec>` | integer >= 1 | `1` | Long-audio chunk pad duration. |
+| `--request-option split_query_sec=<sec>` | integer >= 1 | `5` | Quiet-point query window for long-audio splitting. |
+| `--request-option split_center_sec=<sec>` | integer >= 1 | `30` | Long-audio split center stride. |
+| `--request-option split_threshold_sec=<sec>` | integer >= 1 | `32` | Input duration before quiet-point splitting is used. |
+| `--session-option rvc.weight_type=<type>` | `native`, `f32`, `f16`, `bf16`, `q8_0` | `f32` | Tensor storage type for native RVC, HuBERT, and RMVPE weights. |
+| `--session-option rvc.voice_cache_slots=<n>` | integer >= 0 | `4` | User voice model cache slots. Set `0` to disable caching. |
+
 ## Seed-VC
 
 Seed-VC provides voice conversion and singing voice conversion routes. See [Seed-VC](models/seed_vc.md) for the full route manual.

docs/community_models/inflect_v2.md

@@ -1,5 +1,8 @@
 # Inflect Micro v2 and Nano v2
 
+Default model-manager downloads use the published GGUF package when available;
+the original source/conversion instructions below remain valid for manual use.
+
 `inflect_v2` provides native GGML inference for the English
 [Inflect Micro v2](https://huggingface.co/owensong/Inflect-Micro-v2) and
 [Inflect Nano v2](https://huggingface.co/owensong/Inflect-Nano-v2) TTS models.

docs/community_models/kroko_asr.md

@@ -1,5 +1,8 @@
 # Kroko Community ASR
 
+Default model-manager downloads use the published GGUF package when available;
+the original source/conversion instructions below remain valid for manual use.
+
 `kroko_asr` is a native audio.cpp port of the free Kroko Community
 Zipformer2/RNN-T models. The Kaldi-compatible filterbank, streaming
 Conv2dSubsampling/ConvNeXt encoder, 19-layer Zipformer2, stateless predictor,

docs/community_models/parakeet_tdt.md

@@ -1,5 +1,8 @@
 # Parakeet-TDT 0.6B v3
 
+Default model-manager downloads use the published GGUF package when available;
+the original source/conversion instructions below remain valid for manual use.
+
 FastConformer-TDT ASR port of NVIDIA's [`nvidia/parakeet-tdt-0.6b-v3`](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3)
 (0.6B params, 25 European languages with auto language detection). The default
 package uses the repository's Transformers-compatible `model.safetensors`

docs/gguf.md

@@ -56,7 +56,7 @@ Status labels:
 
 | Family | Package-spec refactor | Safetensors tested after refactor | `orig` GGUF tested | 16-bit GGUF tested | `q8_0` GGUF tested |
 |---|---|---|---|---|---|
-| `ace_step` | Done | Pass | --- | Pass (drift) | Pass (drift) |
+| `ace_step` | Done | Pass | --- | Pass (drift) | No (planner sampling can fail) |
 | `bs_roformer` | Done | Pass | --- | --- | Pass |
 | `chatterbox` | Done | Pass | --- | Pass (ASR match, drift) | Pass (ASR match, drift) |
 | `citrinet_asr` | Done | Pass | --- | --- | Pass |
@@ -88,6 +88,7 @@ Status labels:
 | `qwen3_tts` base | Done | Pass | Pass | Pass (ASR match, drift) | Pass (ASR match, drift) |
 | `qwen3_tts` custom voice | Done | Pass | --- | Pass (ASR match, drift) | Pass (ASR match, drift) |
 | `qwen3_tts` voice design | Done | Pass | --- | Pass (ASR match, drift) | Pass (ASR match, drift) |
+| `rvc` | Done | --- | --- | Pass | --- |
 | `seed_vc` | Done | Pass | --- | Pass (drift) | Pass (drift) |
 | `silero_vad` | Skip (tiny model) | --- | --- | --- | --- |
 | `sortformer_diar` | Done | Pass | --- | Pass | Pass |

docs/maintainers/model_specs.md

@@ -8,6 +8,24 @@ The only accepted spec shapes are the current source-layout specs already used
 by production models, and the typed schema shown here for new metadata/catalog
 work.
 
+## Migration Trees
+
+`model_specs/*.json` is the authoritative runtime tree. Specs with
+`schema_version: 1` in this directory are active typed specs and participate in
+`model_contract()` validation, embedded GGUF metadata, CLI/server inspection,
+and package discovery.
+
+`model_specs_v1/*.json` is a migration reference tree only. It records the
+expected typed shape for families that have not moved to the v1 runtime contract
+yet. Do not treat it as an active runtime source, and do not assume changes there
+affect loaders, CLI options, server validation, or package installs until the
+family is migrated into `model_specs/` with `schema_version: 1`.
+
+During migration, keep public option keys aligned with the code path that will
+actually consume them. If a live loader still accepts an older public key, either
+migrate the runtime to the normalized key in the same change or keep the legacy
+key out of the active spec until cutover. Avoid creating UI-only aliases.
+
 ## Typed Schema
 
 Top-level fields:
@@ -155,6 +173,9 @@ Required dependencies are unconditional. Optional dependencies must declare
 typed `required_when` rows. Each row is a condition over a public option key.
 Common request keys such as `return_timestamps` stay unprefixed; model-specific
 keys stay namespaced. The dependency is needed when any row matches.
+`dependencies[].option` must name a declared option in the dependency `scope`,
+and every `required_when[].option_key` must refer to a declared public option in
+the referenced scope. Multiple `required_when` rows use OR semantics.
 
 ```json
 {