Merge branch 'fix/cyrillic-unicode-decomposition' into ci/cuda12

Author: Anc813

README.md

@@ -50,7 +50,7 @@ audio.cpp would not be moving this quickly without generous contributors bringin
 
 Task tags: `TTS` text to speech, `Clone` voice cloning, `VC` voice conversion, `ASR` speech recognition, `Align` forced alignment, `VAD` voice activity detection, `Diar` speaker diarization, `Codec` audio codec, `Sep` source separation, `Music` music/song generation, `SFX` sound effects, `Edit` audio/music editing, `Design` voice design, `Dialogue` multi-speaker dialogue TTS, `Ctrl` TTS/clone voice control such as emotion, style, instruction, caption, or non-verbal tag control.
 
-Runtime tags: safetensors is the default model loading path. `GGUF 16/Q8` means both 16-bit and `q8_0` GGUF paths are tested; `GGUF Q8` means only `q8_0` is tested; `GGUF F32` means the original-F32 GGUF path is tested. See [docs/gguf.md](docs/gguf.md) for precision/status details. `Bundled` means the tiny runtime asset ships under `assets/framework/models` and needs no separate model download. `Stream` means the family exposes a streaming server/session path.
+Runtime tags: safetensors is the default model loading path. `GGUF 16/Q8/Q4` means those GGUF precision or quantization paths are tested; `GGUF Q8` means only `q8_0` is tested; `GGUF F32` means the original-F32 GGUF path is tested. See [docs/gguf.md](docs/gguf.md) for precision/status details. `Bundled` means the tiny runtime asset ships under `assets/framework/models` and needs no separate model download. `Stream` means the family exposes a streaming server/session path.
 
 | Family | Task | Lang | Variants | Runtime |
 |---|---|---|---|---|
@@ -80,7 +80,7 @@ Runtime tags: safetensors is the default model loading path. `GGUF 16/Q8` means
 | **vevo2** | TTS, Music, VC, Edit | en, zh | Vevo2 with Qwen2.5-0.5B AR model | GGUF 16 |
 | **vibevoice** | TTS, Dialogue | en, zh | VibeVoice-1.5B, VibeVoice-7B | GGUF 16/Q8 |
 | **vibevoice_asr** | ASR | auto | VibeVoice ASR | GGUF 16/Q8 |
-| **voxtral_realtime** | ASR | auto | Voxtral-Mini-4B-Realtime-2602 | GGUF 16/Q8, Stream |
+| **voxtral_realtime** | ASR | auto | Voxtral-Mini-4B-Realtime-2602 | GGUF 16/Q8/Q4, Stream |
 | **voxcpm2** | TTS, Clone, Design, Ctrl | ar, da, de, el, en, es, fi, fr, he, hi, id, it, ja, km, ko, lo, ms, my, nl, no, pl, pt, ru, sv, sw, th, tl, tr, vi, zh | VoxCPM2-2B, 48 kHz | GGUF 16/Q8, Stream |
 | **index_tts2** | TTS, Clone, Ctrl | zh, en | IndexTTS-2 | GGUF 16/Q8 |
 | **irodori_tts** | TTS, Clone, Design, Ctrl | ja | Irodori-TTS-500M-v3, Irodori-TTS-600M-v3-VoiceDesign | GGUF 16/Q8 |

docs/asr.md

@@ -255,12 +255,13 @@ audiocpp_cli --task asr --family vibevoice_asr --model models/VibeVoice-ASR --ba
 
 ## Voxtral Realtime
 
-Voxtral Realtime is a Mistral realtime ASR model with offline and streaming sessions. The model manager installs the Q8_0 standalone GGUF package by default; native Hugging Face directories and other standalone GGUF variants can also be used when provided directly.
+Voxtral Realtime is a Mistral realtime ASR model with offline and streaming sessions. The model manager installs the Q8_0 standalone GGUF package by default; native Hugging Face directories and other standalone GGUF variants can also be used when provided directly. A Q4_K GGUF package is also available for lower memory use and faster CUDA runs; in a quick path check its transcripts matched Q8_0 except for one capitalization-only difference.
 
 | Field | Value |
 |---|---|
 | Family | `voxtral_realtime` |
 | Model path | `models/Voxtral-Mini-4B-Realtime-2602-GGUF/voxtral-mini-4b-realtime-2602-q8_0.gguf` when installed through the model manager |
+| GGUF variants | `bf16`, `q8_0`, `q4_k` |
 | Task | `asr` |
 | Modes | `offline`, `streaming` |
 | Output | Transcription text |

docs/gguf.md

@@ -94,6 +94,12 @@ Status labels:
 | `voxcpm2` | Done | Pass | Pass | Pass (ASR match, drift) | Pass (ASR match, drift) |
 | `voxtral_realtime` | Done | Pass | --- | Pass | Pass |
 
+Additional lower-bit checks:
+
+| Family | Format | Tested |
+|---|---|---|
+| `voxtral_realtime` | `q4_k` | Pass (quick CUDA check; transcripts match Q8 except one capitalization-only difference) |
+
 Q8 packaging notes:
 
 - `chatterbox` Q8 is intentionally mixed type. Graph-sensitive scalar, norm,
@@ -109,6 +115,9 @@ Q8 packaging notes:
   projection or linear weights, while leaving the speaker encoder, lookup, and
   codebook-sensitive tensors unquantized. Quantizing those speaker-side tensors
   can produce long-form quality problems such as large silence.
+- `voxtral_realtime` also has a tested `q4_k` package. In a quick CUDA path
+  check it was smaller and faster than Q8_0, while transcript output matched
+  Q8_0 except for one capitalization-only difference.
 
 ## Build The Converter
 

docs/reports/gguf_q8_performance.md

@@ -22,6 +22,21 @@ Q8 gives the clearest end-to-end wins on the larger AR-style models:
 
 Smaller or already memory-light models still load and run with Q8, but the speed gain can be modest. Treat Q8 as a measured route choice, not a guaranteed win for every model.
 
+## Voxtral Q4_K Quick Check
+
+Voxtral also has a `q4_k` GGUF package. In this quick CUDA path-test check,
+Q4_K was smaller and faster than Q8_0 while producing effectively the same
+transcripts. Four requests matched exactly; one differed only by capitalization
+(`Mother Nature` vs `mother nature`).
+
+| Route | Q8 RTF | Q4_K RTF | Q4_K vs Q8 |
+|---|---:|---:|---:|
+| Offline short | 0.0862 | 0.0629 | 1.37x |
+| Offline medium | 0.0643 | 0.0476 | 1.35x |
+| Offline longer | 0.0576 | 0.0439 | 1.31x |
+| Offline sampled | 0.0630 | 0.0500 | 1.26x |
+| Streaming path | 0.1036 | 0.0904 | 1.15x |
+
 ## TTS Offline Long-Lived Session
 
 | Model | 16-bit speed vs real time | Q8 speed vs real time | Q8 vs 16-bit | 16-bit peak VRAM | Q8 peak VRAM |

scripts/build_windows.ps1

@@ -17,7 +17,7 @@ param(
 )
 
 Set-StrictMode -Version Latest
-$ErrorActionPreference = "Stop"
+$ErrorActionPreference = "Continue"  # let native command stderr (e.g. cmake warnings) flow without aborting the script
 
 function Invoke-Checked {
     param(
@@ -154,7 +154,7 @@ function Find-VsCMake {
         return $cmake
     }
     $cmd = Get-Command "cmake.exe" -ErrorAction SilentlyContinue
-    return if ($cmd) { $cmd.Source } else { "" }
+    if ($cmd) { return $cmd.Source } else { return "" }
 }
 
 function Find-VsNinja {
@@ -164,7 +164,7 @@ function Find-VsNinja {
         return $ninja
     }
     $cmd = Get-Command "ninja.exe" -ErrorAction SilentlyContinue
-    return if ($cmd) { $cmd.Source } else { "" }
+    if ($cmd) { return $cmd.Source } else { return "" }
 }
 
 function Find-WindowsKitTool {

src/framework/sampling/torch_random.cpp

@@ -312,7 +312,7 @@ TorchCudaSamplingPolicy resolve_torch_cuda_sampling_policy(
         log_default_policy(log_category, "backend is not CUDA");
         return policy;
     }
-#ifdef GGML_USE_CUDA
+    // CUDA runtime probe via dynamic library (works with both static and GGML_BACKEND_DL builds)
     const engine::io::DynamicLibraryHandle driver = engine::io::open_dynamic_library(
         {"libcuda.so.1", "libcuda.so", "libcuda.dylib", "nvcuda.dll"});
     if (driver == nullptr) {
@@ -366,14 +366,6 @@ TorchCudaSamplingPolicy resolve_torch_cuda_sampling_policy(
     policy.cuda_fast_path = true;
     policy.cuda_device_index = device_index;
     return policy;
-#else
-    (void) device_index;
-    if (failure_mode == TorchCudaSamplingPolicyFailureMode::FallbackToDefault) {
-        log_default_policy(log_category, "build does not include CUDA support");
-        return policy;
-    }
-    throw std::runtime_error(std::string(model_name) + " generation requires a build with CUDA support");
-#endif
 }
 
 uint64_t torch_cuda_tensor_iterator_offset_blocks(

src/models/qwen3_tts/tokenizer_speech_decoder.cpp

@@ -1047,9 +1047,7 @@ class Qwen3SpeechTokenizerDecoderGraph {
         int64_t code_frames,
         ggml_backend_t backend,
         int threads) const {
-        const bool frame_match = perf_mode_ == Qwen3TTSPerfMode::FlashAttention
-            ? code_frames_ == code_frames
-            : code_frames_ >= code_frames;
+        const bool frame_match = code_frames_ == code_frames;
         return weights_.get() == &weights && frame_match && backend_ == backend &&
             compute_threads_ == std::max(1, threads);
     }
@@ -1156,7 +1154,6 @@ runtime::AudioBuffer Qwen3SpeechTokenizerDecoderRuntime::decode(const Qwen3Speec
     }
     std::vector<float> samples;
     samples.reserve(static_cast<size_t>(codec_codes.frames * kDecodeSamplesPerCode));
-    const int64_t graph_capacity_frames = std::min<int64_t>(codec_codes.frames, kChunkCodes + kLeftContextCodes);
     double graph_build_ms = 0.0;
     double input_upload_ms = 0.0;
     double graph_compute_ms = 0.0;
@@ -1183,7 +1180,7 @@ runtime::AudioBuffer Qwen3SpeechTokenizerDecoderRuntime::decode(const Qwen3Speec
             graph_.reset();
             graph_ = std::make_unique<Qwen3SpeechTokenizerDecoderGraph>(
                 weights_,
-                std::max(chunk_frames, graph_capacity_frames),
+                chunk_frames,
                 *execution_context_,
                 *constants_,
                 graph_arena_bytes_,

src/models/supertonic/tokenizer_text.cpp

@@ -139,13 +139,47 @@ bool append_japanese_kana_decomposition(uint32_t codepoint, std::vector<uint32_t
     return false;
 }
 
+bool append_cyrillic_decomposition(uint32_t codepoint, std::vector<uint32_t> & out) {
+    constexpr uint32_t kBreve = 0x0306;
+    constexpr uint32_t kDiaeresis = 0x0308;
+    static constexpr std::pair<uint32_t, uint32_t> kBreveLetters[] = {
+        {0x0439, 0x0438},  // й -> и
+        {0x0419, 0x0418},  // Й -> И
+    };
+    static constexpr std::pair<uint32_t, uint32_t> kDiaeresisLetters[] = {
+        {0x0451, 0x0435},  // ё -> е
+        {0x0401, 0x0415},  // Ё -> Е
+        {0x0457, 0x0456},  // ї -> і
+        {0x0407, 0x0406},  // Ї -> І
+    };
+
+    for (const auto & [composed, base] : kBreveLetters) {
+        if (codepoint == composed) {
+            out.push_back(base);
+            out.push_back(kBreve);
+            return true;
+        }
+    }
+    for (const auto & [composed, base] : kDiaeresisLetters) {
+        if (codepoint == composed) {
+            out.push_back(base);
+            out.push_back(kDiaeresis);
+            return true;
+        }
+    }
+    return false;
+}
+
 std::vector<uint32_t> decompose_known_text_codepoints(const std::vector<uint32_t> & codepoints) {
     std::vector<uint32_t> out;
     out.reserve(codepoints.size() * 2);
     for (const uint32_t codepoint : codepoints) {
         if (append_japanese_kana_decomposition(codepoint, out)) {
             continue;
         }
+        if (append_cyrillic_decomposition(codepoint, out)) {
+            continue;
+        }
         append_hangul_decomposition(codepoint, out);
     }
     return out;

webui/README.md

@@ -101,8 +101,10 @@ Starts the Gradio web interface (`webui.py`); open **http://127.0.0.1:7860** in
   (one model in VRAM at a time; switching models restarts it).
 - The UI lets you upload a reference voice, download uninstalled models, enter an HF token / proxy, and so on.
 - Backend is auto-detected (as above: GPU if CUDA is present, otherwise CPU); `AUDIOCPP_BACKEND=gpu|cpu` forces it.
-  In CPU mode, the ggml thread count is set automatically to cores-1 (override with `AUDIOCPP_THREADS=N`), and the
-  VRAM warning is no longer shown.
+  In CPU mode, the ggml thread count is set automatically from the **physical** core count — SMT/Hyper-Threading
+  siblings are not counted, and one core is left free above 4 — so a long run leaves the rest of the machine
+  usable. Filling every logical CPU is faster (~1.4x in a short CPU TTS run on an 8-core/16-thread 5800H); set
+  `AUDIOCPP_THREADS=N` if you want that. The VRAM warning is not shown in CPU mode.
 
 > The web interface (7860) is for humans; to use it as an **API for other programs**, start `audiocpp_server`
 > directly, or once the WebUI is up, hit the port 8080 it manages directly.

webui/README.zh.md

@@ -92,7 +92,9 @@ python3 -m venv venv && ./venv/bin/pip install -r webui/requirements.txt
   起/切换底层的 `audiocpp_server`（一次一个模型在显存里，换模型即重启）。
 - 界面里可上传参考音色、下载未安装的模型、填 HF token / 代理等。
 - 后端自动检测（同上：有 CUDA 用 GPU，否则 CPU）；`AUDIOCPP_BACKEND=gpu|cpu` 可强制。
-  CPU 模式下 ggml 线程数自动设为核数-1（可用 `AUDIOCPP_THREADS=N` 覆盖），且不再显示显存警告。
+  CPU 模式下 ggml 线程数按**物理核数**自动设置（不计超线程的逻辑核，物理核多于 4 时再留一个核给系统），
+  这样长任务跑起来机器仍然可用。把逻辑核占满会更快（8 核 16 线程的 5800H 上，一次短文本 CPU TTS 快约 1.4 倍），
+  想要这个速度就设 `AUDIOCPP_THREADS=N`。CPU 模式下不显示显存警告。
 
 > 网页界面（7860）是给人用的；要给**其它程序**当 API，请直接启动 `audiocpp_server`，
 > 或让 WebUI 起来后直接打它管理的 8080 端口。