Skip to content

Commit cd98afb

Browse files
authored
sanotts: sanoTTS voice family community model (seven voices, 294k-2.27M params, en/vi/id, GGUF FP32) (#449)
* model_specs: add the sanoTTS community family Two GGUF packages from ampixa/sanoTTS on Hugging Face: heart-nano (294,279 parameters, int8, 357 KB) as the default, and heart (2,272,145, f32, 9.1 MB). Both are converted losslessly from the blobs the project already ships. Rebuilding those blobs from the GGUF reproduces them byte for byte, and the golden gate on the rebuilt weights gives the same correlation against the float PyTorch references as the originals -- 0.989703 and 1.000000 against a 0.98 threshold -- including when the GGUF is fetched from Hugging Face rather than built locally. Session options mirror inflect_v2's, since sanoTTS needs the same external eSpeak-ng phonemizer and must not embed it. * sanotts: model spec, GGUF assets and the eSpeak-ng front end Groundwork for a ggml-native sanoTTS community model. model_specs/sanotts.json heart-nano (294,279 params, 24 kHz) from ampixa/sanoTTS on Hugging Face. Session options mirror inflect_v2's, since sanoTTS needs the same external eSpeak-ng phonemizer. assets.{h,cpp} Reads config.json and the GGUF tensors. config.json carries (tensor, offset) regions emitted by the packaging tool, so nothing here translates region names and the two cannot drift. Every shape constant is compared against the build's own, because a lineage mismatch would read weights at the wrong offsets and synthesize noise rather than fail. frontend.{h,cpp} Text -> the 62-symbol phoneme ids the model was trained on: eSpeak-ng IPA, then misaki's E2M rewrite, then the character-level vocabulary. Ported from the project's own JavaScript and Python front ends so all three agree symbol for symbol. eSpeak-ng is opened at runtime and never linked -- it is GPL-3.0 and must not be embedded here, the same treatment inflect_v2 gives it. The packaging is verified upstream: rebuilding both weight blobs from the GGUF reproduces the originals byte for byte, and the golden gate on the rebuilt weights matches the float PyTorch reference at 0.989703 against a 0.98 threshold -- including when the GGUF is fetched from Hugging Face. The inference graph is next, built on the framework's module library rather than a vendored runtime, so sanoTTS gets the shared backends like every other model here. * sanotts: ggml-native runtime, session and docs Three cached graphs (duration, token stage, frame stage + ConvNeXt decoder) with the reference implementations' exact semantics: ATen-compatible MT19937 noise, torch.linspace/expand_features float behavior, LayerNorm eps 1e-6, erf GELU, torch.istft trim, and the 0.9973-pole DC blocker. The frontend gains phonemizer-compatible punctuation preservation and the correct eSpeak-ng tie mode so token streams are byte-identical to the Python front end. Verified against the project's numpy reference (same text, seed, and eSpeak-ng build): correlation 0.999999985, identical sample count; the reference is itself gated 0.987-1.000 against float PyTorch. 38.7 s of audio renders in 0.22 s wall on CPU (peak RSS 76 MB). Claude-Session: https://claude.ai/code/session_01P1iL37FdfJkGxdGrpjH1we * sanotts: bisect chunks that phonemize past the duration token limit The codepoint chunker cannot see phoneme counts, so a dense 280-codepoint chunk can exceed the duration model's 207-token training limit. encode() now throws a typed SanoTtsTooLongError and the session splits the chunk at the whitespace nearest its middle and recurses, so the shared long-form case (6 kB of text, 6.2 minutes of audio) renders instead of failing. Claude-Session: https://claude.ai/code/session_01P1iL37FdfJkGxdGrpjH1we * sanotts: add the heart 2.27M voice as a second package Same graph, wider and deeper; the runtime now derives the expected tensor count from the config instead of hardcoding heart-nano's 103, and the weight arena covers the 9.1 MB FP32 payload. Verified like heart-nano: correlation 0.999999985 against the numpy reference at identical sample count, installed end to end from the published Hugging Face package. Claude-Session: https://claude.ai/code/session_01P1iL37FdfJkGxdGrpjH1we * sanotts: piperlite lineage -- amy, hfc, kristin, vi and id voices Second graph in the family: duration and acoustic students into a 192-channel latent, then a 3-stage ConvTranspose1d decoder with dilated residual banks (kristin adds a learned post filter). Deterministic, 22.05 kHz. The shared front-end structure moves into graph_common.h; the session dispatches on the config's graph field. The piperlite front end reproduces Piper's convention exactly: untied eSpeak-ng phonemes through the phonemizer punctuation pipeline, NFD decomposition to codepoints, the per-voice phoneme_id_map with [BOS, PAD, (id, PAD)..., EOS] framing, the schwa fallback for ids outside a component's trained vocab, and regional-variant-first voice selection (phonemizer rejects bare language codes on espeak-ng >= 1.49, so 'en' must resolve to en-us in both stacks). All five voices verified against the project's numpy reference with the same eSpeak-ng build: correlation >= 0.99999996 at identical sample counts, installed end to end from the published Hugging Face packages. Vietnamese and Indonesian exercise their own espeak voices and language validation. Claude-Session: https://claude.ai/code/session_01P1iL37FdfJkGxdGrpjH1we * sanotts: deduplicate the two runtimes through graph_common The duration and token stages run under identical tensor names in both lineages, so their four graph builders collapse into one shared build_front_graph over a small stage spec; the frame-level acoustic stage both decoders open with becomes acoustic_frame_stage. BackendState now carries the backend/weights/CUDA-duration-mirror boilerplate once, a cached_graph template replaces the six cache accessors, and the host-side feature builders and duration rounding are shared (rounding unified to double, which is what the numpy references both runtimes are gated against actually compute). All seven voices re-verified after the refactor: correlations unchanged (>= 0.99999996) at identical sample counts; unit test and the shared long-form case pass. Net -363 lines. Claude-Session: https://claude.ai/code/session_01P1iL37FdfJkGxdGrpjH1we
1 parent 70f64fe commit cd98afb

18 files changed

Lines changed: 4388 additions & 0 deletions

File tree

CMakeLists.txt

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -630,6 +630,19 @@ audiocpp_add_model(personaplex
630630
engine::models::personaplex::make_personaplex_loader
631631
)
632632
633+
audiocpp_add_model(sanotts
634+
SOURCES
635+
src/community_models/sanotts/assets.cpp
636+
src/community_models/sanotts/frontend.cpp
637+
src/community_models/sanotts/piper_runtime.cpp
638+
src/community_models/sanotts/runtime.cpp
639+
src/community_models/sanotts/session.cpp
640+
INCLUDES
641+
engine/community_models/sanotts/session.h
642+
LOADERS
643+
engine::models::sanotts::make_sanotts_loader
644+
)
645+
633646
audiocpp_add_model(inflect_v2
634647
SOURCES
635648
src/community_models/inflect_v2/assets.cpp
@@ -2614,6 +2627,13 @@ if (ENGINE_BUILD_TESTS)
26142627
COMMAND echo_tts_host_units
26152628
)
26162629

2630+
add_engine_unittest(sanotts_frontend_test tests/unittests/test_sanotts_frontend.cpp)
2631+
target_include_directories(sanotts_frontend_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/tests/unittests)
2632+
add_test(
2633+
NAME sanotts_frontend_test
2634+
COMMAND sanotts_frontend_test
2635+
)
2636+
26172637
add_engine_unittest(inflect_v2_frontend_test tests/unittests/test_inflect_v2_frontend.cpp)
26182638
target_include_directories(inflect_v2_frontend_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/tests/unittests)
26192639
add_test(

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,7 @@ Community model ports live under `community_models` to make the ownership bounda
162162
| **moss_voicegen** | Voice Design | en, zh | GGUF | Joost [@jrohde](https://github.com/jrohde) | [MOSS-VoiceGenerator](docs/community_models/moss_voicegen.md) speech in a voice designed from a written instruction |
163163
| **outetts** | TTS, Clone | en, ar, zh, nl, fr, de, it, ja, ko, lt, ru, es, pt, be, bn, ka, hu, lv, fa, pl, sw, ta, uk | GGUF | Mirek [@mirek190](https://github.com/mirek190) | Llama-OuteTTS-1.0-1B TTS and voice cloning support |
164164
| **parakeet_tdt** | ASR | auto, bg, cs, da, de, el, en, es, et, fi, fr, hr, hu, it, lt, lv, mt, nl, pl, pt, ro, ru, sk, sl, sv, uk | GGUF F32/16/Q8, Stream | [@dleiferives](https://github.com/dleiferives) | [Parakeet-TDT 0.6B v3](docs/community_models/parakeet_tdt.md) offline, long-form, and buffered-streaming ASR support |
165+
| **sanotts** | TTS | en, vi, id | GGUF FP32 | Ashish [@voidash](https://github.com/voidash) | [sanoTTS voice family](docs/community_models/sanotts.md) seven voices from 294k to 2.27M parameters, native offline synthesis |
165166
| **sense_asr** | ASR | auto, zh, en, yue, ja, ko, pt, ru, es, it, fr, de, nl, pl, tr, ar, hi, vi, th, id, ms, fa, nospeech | GGUF Q8, Stream | Jason Chen [@jasonchen31](https://github.com/jasonchen31), [@LauraGPT](https://github.com/LauraGPT) / FunASR | [SenseVoice-Small](docs/community_models/sense_asr.md) offline/streaming SAN-M + CTC transcription with event/emotion/language tags and ITN |
166167
| **soprano_tts** | TTS | en | GGUF Q8, Stream | [@drzsdrtfg](https://github.com/drzsdrtfg) | [Soprano-1.1-80M](https://huggingface.co/WalkingCat/Soprano-1.1-80M-GGUF) ultra-lightweight TTS with Qwen3 LM + Vocos decoder |
167168
| **vietneu_tts** | TTS, Clone | vi, en | GGUF | Phuoc [@phuocnguyen90](https://github.com/phuocnguyen90) | [VieNeu-TTS-v3-Turbo](docs/community_models/vietneu_tts.md) TTS and voice cloning support |

docs/community_models/models.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ Practical expectations:
3131
| **outetts** | TTS, voice cloning | en, ar, zh, nl, fr, de, it, ja, ko, lt, ru, es, pt, be, bn, ka, hu, lv, fa, pl, sw, ta, uk | Mirek [@mirek190](https://github.com/mirek190) | [Llama-OuteTTS-1.0-1B](outetts.md) TTS and voice cloning support |
3232
| **voxcpm1** | TTS, voice cloning | zh, en, ja, ko | Community | [VoxCPM1](voxcpm1.md) tokenizer-free 0.5B TTS with 16 kHz output, streaming, and continuation-mode voice cloning |
3333
| **parakeet_tdt** | ASR | auto, bg, cs, da, de, el, en, es, et, fi, fr, hr, hu, it, lt, lv, mt, nl, pl, pt, ro, ru, sk, sl, sv, uk | [@dleiferives](https://github.com/dleiferives) | [Parakeet-TDT 0.6B v3](parakeet_tdt.md) offline, long-form, and buffered-streaming ASR support |
34+
| **sanotts** | TTS | en, vi, id | Community | [sanoTTS voice family](sanotts.md) seven voices from 294k to 2.27M parameters, FP32 offline synthesis |
3435
| **sense_asr** | ASR | auto, zh, en, yue, ja, ko, pt, ru, es, it, fr, de, nl, pl, tr, ar, hi, vi, th, id, ms, fa, nospeech | Jason Chen [@jasonchen31](https://github.com/jasonchen31), [@LauraGPT](https://github.com/LauraGPT) / FunASR | [SenseVoice-Small](sense_asr.md) offline/streaming SAN-M + CTC transcription with event/emotion/language tags and ITN |
3536
| **vietneu_tts** | TTS, voice cloning | vi, en | Phuoc [@phuocnguyen90](https://github.com/phuocnguyen90) | [VieNeu-TTS-v3-Turbo](vietneu_tts.md) TTS and voice cloning support |
3637
| **moss_voicegen** | Voice design | en, zh | Joost [@jrohde](https://github.com/jrohde) | [MOSS-VoiceGenerator](moss_voicegen.md) voice design from a written instruction, on the MOSS delay architecture |

docs/community_models/sanotts.md

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# sanoTTS voice family
2+
3+
`sanotts` provides native GGML inference for the
4+
[sanoTTS](https://github.com/Ampixa/sanoTTS) voice family — very small
5+
text-to-speech models, the smallest of which also runs on microcontrollers.
6+
All packages download from Hugging Face
7+
([ampixa/sanoTTS](https://huggingface.co/ampixa/sanoTTS) `gguf/`) as
8+
standalone FP32 GGUFs with embedded model specs. Offline FP32 inference only.
9+
10+
Two graphs share one family:
11+
12+
- **nano** — duration student → contextual acoustic student → mel-100 →
13+
noise-fed ConvNeXt-1D decoder → [log-magnitude | phase] head → inverse
14+
STFT. 24 kHz. A seed picks one of many valid renderings.
15+
- **piperlite** — duration student → contextual acoustic student → 192-channel
16+
latent → 3-stage ConvTranspose1d decoder with dilated residual banks →
17+
tanh waveform. 22.05 kHz. Fully deterministic (no seed).
18+
19+
| Package | Voice | Graph | Params | Language | Notes |
20+
|---|---|---|---:|---|---|
21+
| `sanotts_heart_orig` | heart | nano | 2,272,145 | en | best quality of the nano pair |
22+
| `sanotts_heart_nano_orig` | heart-nano | nano | 294,279 | en | microcontroller-class |
23+
| `sanotts_amy_orig` | amy | piperlite | 1,454,284 | en | Piper-distilled |
24+
| `sanotts_hfc_orig` | hfc | piperlite | 1,834,380 | en | largest piperlite voice |
25+
| `sanotts_kristin_orig` | kristin | piperlite | 1,396,151 | en | carries a learned post filter |
26+
| `sanotts_vi_orig` | vi | piperlite | 1,565,484 | vi | Vietnamese |
27+
| `sanotts_id_orig` | id | piperlite | 1,562,124 | id | Indonesian |
28+
29+
## Install
30+
31+
Install eSpeak-ng and its voice data first. On Debian or Ubuntu:
32+
33+
```bash
34+
sudo apt install espeak-ng libespeak-ng1
35+
```
36+
37+
On macOS:
38+
39+
```bash
40+
brew install espeak-ng
41+
```
42+
43+
Then install any package, e.g.:
44+
45+
```bash
46+
python3 tools/model_manager_v2.py install sanotts_heart_orig --models-root models
47+
python3 tools/model_manager_v2.py install sanotts_amy_orig --models-root models
48+
```
49+
50+
## Run
51+
52+
```bash
53+
audiocpp_cli --task tts --family sanotts \
54+
--model models/sanoTTS-heart-GGUF --backend cpu \
55+
--text "Hello from sano T T S, a very small neural text to speech model." \
56+
--out sanotts.wav
57+
```
58+
59+
Swap `--model` for any installed package directory
60+
(`models/sanoTTS-amy-GGUF`, `models/sanoTTS-vi-GGUF`, ...). The Vietnamese
61+
and Indonesian voices accept `--language vi` / `--language id`; a session
62+
rejects text tagged with a language the voice was not trained on.
63+
64+
eSpeak-ng is loaded dynamically at runtime, never linked. If it is not on the
65+
default library path:
66+
67+
```bash
68+
audiocpp_cli --task tts --family sanotts \
69+
--model models/sanoTTS-heart-GGUF --backend cpu \
70+
--session-option sanotts.espeak_library_path=/path/to/libespeak-ng.so \
71+
--session-option sanotts.espeak_data_path=/path/to/espeak-ng-data \
72+
--text "A configured eSpeak installation." --out sanotts.wav
73+
```
74+
75+
## Options
76+
77+
- `speaking_rate` (request, 0.5..2.0, default 1.0) — duration multiplier on
78+
the voice's tuned length scale; larger is slower.
79+
- `seed` (request, default 0) — nano voices only: the decoder is noise-fed,
80+
so a given seed picks one of many valid renderings. `0` derives the seed
81+
from each text chunk as `sha256(text)[:8]`, which is what the reference
82+
implementations do; an explicit seed advances by one per long-form chunk.
83+
Piperlite voices are deterministic and ignore the seed.
84+
- `text_chunk_size` (request, default 280) — maximum codepoints per long-form
85+
chunk; chunks split on sentence punctuation first, and a chunk that
86+
phonemizes past the voice's token limit is bisected at whitespace.
87+
88+
## Determinism and parity
89+
90+
The runtimes reproduce the reference implementations' exact semantics:
91+
92+
- Front ends: the phonemizer punctuation-preservation pipeline through the
93+
same eSpeak-ng library. The nano voices add the misaki E2M rewrite with
94+
tie characters; the piperlite voices use Piper's NFD-decompose-to-
95+
codepoints convention, per-voice `phoneme_id_map`, `[BOS, PAD, (id, PAD)…,
96+
EOS]` framing, and the schwa fallback for ids outside a component's
97+
trained vocabulary.
98+
- nano: ATen-compatible MT19937 noise (24-bit uniform, Box–Muller in blocks
99+
of 16), torch.istft window normalisation and centre trim, and the
100+
reference's DC blocker `H(z) = (1 - z^-1)/(1 - 0.9973 z^-1)`.
101+
- Shared: torch.linspace / expand_features float behaviour, LayerNorm eps
102+
1e-6 (nano), ties-to-even duration rounding.
103+
104+
Measured against the project's numpy references (same text, same
105+
eSpeak-ng build), every voice: **correlation ≥ 0.99999996 with identical
106+
sample counts**; max sample delta ~1.7e-05 is the WAV's own int16
107+
quantisation. The numpy references are themselves gated ≥ 0.987 against the
108+
float PyTorch models.
109+
110+
## Performance
111+
112+
CPU-only, 12-thread x86 (default 4 backend threads), FP32, the shared 6 kB
113+
long-form text:
114+
115+
| Voice | Audio | Wall | vs real time | Peak RSS |
116+
|---|---:|---:|---:|---:|
117+
| heart-nano | 373 s | 1.3 s | ~283× | 220 MB |
118+
| amy | 394 s | 18.5 s | ~21× | 497 MB |
119+
120+
The nano decoder runs at frame rate with a host iSTFT; the piperlite decoder
121+
runs convolutions at audio rate, which is why it is heavier. Graphs are
122+
cached per token count (duration and token stages) and per frame count
123+
(decoder); `--log` prints cache hits and per-stage timings.
124+
125+
## Licensing
126+
127+
The sanoTTS runtimes and weights are MIT-licensed. eSpeak-ng is GPL-3.0 and
128+
is therefore opened with `dlopen` at runtime and never linked, matching how
129+
`inflect_v2` treats it.

docs/tts.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@
3131
| GLM-TTS | `glm_tts` | `tts`, `clon` | [GLM-TTS](#glm-tts) |
3232
| Inflect Micro v2 | `inflect_v2` | `tts` | [Inflect v2](#inflect-v2) |
3333
| OuteTTS | `outetts` | `tts`, `clon` | [OuteTTS](#outetts) |
34+
| sanoTTS voice family | `sanotts` | `tts` | [sanoTTS](#sanotts) |
3435
| Supertonic | `supertonic` | `tts` | [Supertonic](#supertonic) |
3536
| VieNeu-TTS | `vietneu_tts` | `tts`, `clon` | [VieNeu-TTS](community_models/vietneu_tts.md) |
3637
| VibeVoice | `vibevoice` | `tts` | [VibeVoice](#vibevoice) |
@@ -770,6 +771,27 @@ See the [Inflect v2 community model guide](community_models/inflect_v2.md) for
770771
eSpeak-ng paths, long-form behavior, source/conversion instructions, and
771772
limitations.
772773

774+
## sanoTTS
775+
776+
sanoTTS is a family of very small offline TTS voices (English, Vietnamese,
777+
Indonesian; 294k to 2.27M parameters) with native GGML runtimes; the
778+
smallest voice also runs on microcontrollers. The GGUF packages are
779+
standalone and download from Hugging Face. sanoTTS requires an external
780+
eSpeak-ng installation:
781+
782+
```bash
783+
python3 tools/model_manager_v2.py install sanotts_heart_nano_orig --models-root models
784+
785+
audiocpp_cli --task tts --family sanotts \
786+
--model models/sanoTTS-heart-nano-GGUF --backend cpu \
787+
--text "Hello from sano T T S, a very small neural text to speech model." \
788+
--request-option speaking_rate=1.0 \
789+
--out sanotts.wav
790+
```
791+
792+
See the [sanoTTS community model guide](community_models/sanotts.md) for
793+
eSpeak-ng paths, seed semantics, parity evidence, and performance numbers.
794+
773795
## Supertonic
774796

775797
Supertonic 3 is a preset-voice multilingual TTS model. It does not use external speaker references in the current integration.
Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
#pragma once
2+
3+
#include "engine/framework/assets/resource_bundle.h"
4+
#include "engine/framework/assets/tensor_source.h"
5+
6+
#include <array>
7+
#include <cstdint>
8+
#include <filesystem>
9+
#include <memory>
10+
#include <string>
11+
#include <unordered_map>
12+
#include <vector>
13+
14+
namespace engine::models::sanotts {
15+
16+
struct SanoTtsConfig {
17+
int64_t vocab_size = 62;
18+
int64_t sample_rate = 24000;
19+
int64_t hop_length = 256;
20+
int64_t n_fft = 1024;
21+
int64_t mels = 100;
22+
int64_t dim = 0;
23+
int64_t blocks = 0;
24+
int64_t pw_hidden = 0;
25+
int64_t noise_channels = 4;
26+
int64_t dw_kernel = 7;
27+
int64_t embed_kernel = 7;
28+
29+
int64_t duration_hidden = 0;
30+
int64_t duration_depth = 0;
31+
int64_t duration_kernel = 5;
32+
int64_t duration_max_tokens = 207;
33+
int64_t duration_max_frames = 80;
34+
35+
int64_t acoustic_hidden = 0;
36+
int64_t acoustic_token_depth = 0;
37+
int64_t acoustic_depth = 0;
38+
int64_t acoustic_kernel = 5;
39+
40+
std::string voice;
41+
};
42+
43+
enum class SanoTtsGraph {
44+
Nano, // mel-100 -> ConvNeXt-1D -> iSTFT, noise-fed (heart, heart-nano)
45+
Piperlite, // 192-ch latent -> 3-stage ConvTranspose1d, deterministic (amy, ...)
46+
};
47+
48+
struct SanoTtsPiperConfig {
49+
std::string voice;
50+
std::string language; // short code the session validates against: en, vi, id
51+
std::string espeak_voice;
52+
int64_t sample_rate = 22050;
53+
double duration_length_scale = 1.0;
54+
55+
int64_t duration_vocab = 0;
56+
int64_t duration_hidden = 0;
57+
int64_t duration_depth = 0;
58+
int64_t duration_kernel = 5;
59+
int64_t duration_max_tokens = 0;
60+
int64_t duration_max_frames = 0;
61+
62+
int64_t acoustic_vocab = 0;
63+
int64_t acoustic_hidden = 0;
64+
int64_t acoustic_depth = 0;
65+
int64_t acoustic_token_depth = 0;
66+
int64_t acoustic_kernel = 5;
67+
int64_t acoustic_out_channels = 0;
68+
69+
std::array<int64_t, 4> channels = {0, 0, 0, 0};
70+
std::array<std::vector<int64_t>, 3> stage_branches;
71+
int64_t post_filter_channels = 0;
72+
int64_t post_filter_layers = 0;
73+
int64_t post_filter_kernel = 9;
74+
double post_filter_scale = 0.0;
75+
76+
/** Piper phoneme_id_map: one UTF-8 codepoint -> id. */
77+
std::unordered_map<std::string, int32_t> phoneme_id_map;
78+
};
79+
80+
struct SanoTtsAssets {
81+
assets::ResourceBundle resources;
82+
SanoTtsGraph graph = SanoTtsGraph::Nano;
83+
SanoTtsConfig config; // valid when graph == Nano
84+
SanoTtsPiperConfig piper; // valid when graph == Piperlite
85+
std::shared_ptr<const assets::TensorSource> weights;
86+
};
87+
88+
std::shared_ptr<const SanoTtsAssets> load_sanotts_assets(
89+
const std::filesystem::path & model_path);
90+
91+
} // namespace engine::models::sanotts
Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
#pragma once
2+
3+
#include <cstdint>
4+
#include <stdexcept>
5+
#include <filesystem>
6+
#include <memory>
7+
#include <string>
8+
#include <unordered_map>
9+
#include <vector>
10+
11+
namespace engine::models::sanotts {
12+
13+
/** Thrown by encode() when a chunk phonemizes past the duration model's
14+
* token limit; the session responds by bisecting the chunk. */
15+
struct SanoTtsTooLongError : std::runtime_error {
16+
using std::runtime_error::runtime_error;
17+
};
18+
19+
struct SanoTtsEncoded {
20+
std::vector<int32_t> token_ids;
21+
std::string dropped; // symbols outside the vocabulary, for tracing
22+
};
23+
24+
/**
25+
* Text -> the 62-symbol phoneme ids the sanoTTS front end was trained on.
26+
*
27+
* eSpeak-ng produces the IPA; misaki's E2M table then rewrites it into the
28+
* character-level inventory this model uses. Both steps are reproduced from
29+
* the project's own JavaScript and Python front ends so the three agree
30+
* symbol for symbol.
31+
*
32+
* eSpeak-ng is opened at runtime and never linked, matching how inflect_v2
33+
* treats it: it is GPL-3.0 and must not be embedded in this project.
34+
*/
35+
class SanoTtsFrontend {
36+
public:
37+
SanoTtsFrontend(
38+
std::filesystem::path espeak_library_path,
39+
std::filesystem::path espeak_data_path,
40+
int64_t max_tokens);
41+
~SanoTtsFrontend();
42+
43+
[[nodiscard]] SanoTtsEncoded encode(const std::string & text) const;
44+
45+
/** Long-form splitting on sentence punctuation, then a codepoint budget. */
46+
[[nodiscard]] static std::vector<std::string> split_text(
47+
const std::string & text,
48+
int64_t max_codepoints);
49+
50+
/** Pause inserted between chunks, longer after a sentence end. */
51+
[[nodiscard]] static double boundary_pause_seconds(const std::string & chunk);
52+
53+
private:
54+
struct Impl;
55+
std::unique_ptr<Impl> impl_;
56+
int64_t max_tokens_;
57+
};
58+
59+
/**
60+
* Text -> Piper phoneme ids for the piperlite voices.
61+
*
62+
* Reproduces piper's phonemize_espeak / phonemes_to_ids convention through
63+
* the same eSpeak-ng library: phonemizer-style punctuation preservation,
64+
* NFD decomposition to single codepoints, then the voice's phoneme_id_map
65+
* with [BOS, PAD, (id, PAD)..., EOS] framing. Deterministic per voice.
66+
*/
67+
class SanoTtsPiperFrontend {
68+
public:
69+
SanoTtsPiperFrontend(
70+
std::filesystem::path espeak_library_path,
71+
std::filesystem::path espeak_data_path,
72+
std::string espeak_voice,
73+
std::unordered_map<std::string, int32_t> phoneme_id_map,
74+
int64_t max_tokens);
75+
~SanoTtsPiperFrontend();
76+
77+
[[nodiscard]] SanoTtsEncoded encode(const std::string & text) const;
78+
79+
private:
80+
struct Impl;
81+
std::unique_ptr<Impl> impl_;
82+
std::unordered_map<std::string, int32_t> id_map_;
83+
int64_t max_tokens_;
84+
};
85+
86+
} // namespace engine::models::sanotts

0 commit comments

Comments
 (0)