diff --git a/README.md b/README.md index 2c67d22..6e5a941 100644 --- a/README.md +++ b/README.md @@ -1,51 +1,74 @@ # AvaCore: Native Persian Text-to-Speech (TTS) Engine for Android -AvaCore is a high-performance, on-device Persian (Farsi) Text-to-Speech engine designed to provide a natural and seamless voice experience for Android users. By integrating directly with the `android.speech.tts` framework, AvaCore enables all Android applications to speak Farsi with human-like prosody and high clarity, entirely offline. +AvaCore is a high-performance, on-device Persian (Farsi) Text-to-Speech engine designed to provide a natural and seamless voice experience for Android users. By integrating directly with the `android.speech.tts` framework, AvaCore enables all Android applications to speak Farsi with human-like prosody and high clarity, **entirely offline** — no network access is required at build or run time. ## Project Vision -To bridge the accessibility gap for Persian speakers on Android by delivering a state-of-the-art TTS engine that overcomes the unique linguistic challenges of the Farsi language, such as short vowel omissions and hidden *Ezafe* constructions. - -## Technical Roadmap - -### 1. NLP and Linguistic Analysis Pipeline -The quality of synthesized speech depends heavily on text analysis. Farsi poses unique challenges with homographs and hidden vowels. -* **Preprocessing:** Utilizing `DadmaTools` for character normalization and noise removal. -* **Ezafe Detection:** Advanced detection of "Kasreh Ezafe" to ensure grammatical correctness in pronunciation. -* **Homograph Disambiguation:** Implementation of the `GE2PE` protocol to resolve phonetic ambiguities in written Farsi, significantly improving recognition accuracy. -* **Two-Step G2P Training:** A Grapheme-to-Phoneme model trained first on massive machine-generated data, followed by fine-tuning on high-precision manual transcriptions. - -### 2. Neural Synthesis Architecture -A hybrid approach focusing on stability and performance for mobile environments. -* **Frontend (Tacotron):** Employs **Stepwise Monotonic Attention** to prevent word skipping or repeating in long sentences, ensuring robust synthesis for complex Farsi literature. -* **Backend/Vocoder (WaveRNN):** Generates high-quality 24kHz audio. WaveRNN reduces the storage footprint from standard large-scale models to a mobile-friendly 2.5MB - 70MB range. - -### 3. On-Device Optimization (Edge AI) -Optimized for real-time performance on various mobile hardware tiers. -* **Quantization:** 8-bit mu-law quantization with a **0.86 Pre-emphasis filter** to maintain signal-to-noise ratio while reducing model size. -* **Hardware Acceleration:** Leveraging the **Android Neural Networks API (NNAPI)** for NPU/GPU execution. -* **Computation Efficiency:** Split-state GRU architecture and result caching for a 15-50% boost in computational efficiency, targeting at least **3x Real-Time Factor (RTF)**. - -### 4. Android System Integration -Seamlessly integrated into the Android ecosystem to serve all installed applications. -* **System Engine:** Implemented via the standard `android.speech.tts.TextToSpeechService` API. -* **Streaming Logic:** Uses an **Inner/Outer stream loop** (5:1 ratio, approx. 100ms speech chunks) for ultra-low latency incremental audio delivery. -* **Standard Compliance:** Supports system intents like `ACTION_INSTALL_TTS_DATA` for seamless resource management and installation. - -### 5. Training Methodology & ManaTTS Dataset -* **Dataset:** Powered by the **ManaTTS** dataset, featuring 86 hours of high-quality 44.1kHz speech. -* **Cleaning:** Audio preprocessing using Spleeter to ensure noise-free training samples. -* **Forced Alignment:** Multi-model ASR voting and Interval/Gapped search for precise text-to-audio alignment. -* **Quality Control:** Strict Character Error Rate (CER) thresholds (HIGH < 0.05, MIDDLE < 0.20) for training data selection. - -## Performance Benchmarks (KPIs) - -| Metric | Target Value | Technical Note | +To bridge the accessibility gap for Persian speakers on Android by delivering a state-of-the-art TTS engine that overcomes the unique linguistic challenges of the Farsi language, such as short-vowel omission and hidden *Ezafe* (کسرهٔ اضافه) constructions. + +--- + +## Current Architecture (what ships today) + +AvaCore is built on a compact, proven, fully-offline stack: + +| Layer | Technology | Notes | +| :--- | :--- | :--- | +| **Inference engine** | Sherpa-ONNX 1.10.41 (`app/libs/sherpa-onnx.aar`) | JNI + Kotlin wrapper around ONNX Runtime | +| **Acoustic + vocoder** | **Piper VITS** (`persian_model.onnx`) | End-to-end model — the HiFi-GAN-style decoder *is* the vocoder; 22.05 kHz | +| **Grapheme-to-phoneme** | **eSpeak-NG** (`espeak-ng-data/`) | Persian phonemization | +| **Text front-end** | AvaCore `nlp/` pipeline (Kotlin) | Normalization, number expansion, lexicon, segmentation, SSML | +| **System integration** | `AvaTtsService : TextToSpeechService` | Serves every app on the device | + +> Note: VITS is a single end-to-end network. There is **no separate Tacotron front-end or WaveRNN vocoder** in the shipping engine — those belong to the future roadmap below. + +### Synthesis pipeline +Text flows through `nlp/TextProcessor` before the neural model: + +1. **SSML** (`nlp/Ssml.kt`) — ``, ``, `` are honored; plain text passes through. +2. **Number expansion** (`nlp/NumberToWords.kt`) — full Persian cardinals/ordinals/decimals/percent (`۱۴۰۳` → «هزار و چهارصد و سه»), Persian/Arabic/ASCII digits. +3. **Normalization** (`nlp/Normalizer.kt`) — Arabic→Persian letter folding, ZWNJ (نیم‌فاصله) normalization, kashida/tanvin/diacritic cleanup, punctuation spacing. +4. **Pronunciation lexicon** (`nlp/PronunciationLexicon.kt` + `assets/tts/lexicon.txt`) — high-precision overrides for short-vowel restoration and fixed *ezafe* compounds; curated and easily extensible. +5. **Sentence segmentation** (`nlp/SentenceSegmenter.kt`) — splits text into short, prosodically-paused units. + +### Streaming + responsiveness +- **Incremental streaming:** synthesis uses Sherpa's `generateWithCallback`, so the first audio plays after the first chunk — latency-to-first-audio stays roughly constant regardless of text length. +- **Instant interruption:** `onStop()` aborts the current utterance mid-stream. +- **System speech-rate:** the platform speech-rate is mapped to the engine speed multiplier. +- **OEM-safe buffering:** audio is delivered in ≤ 8 KB chunks to satisfy strict OEM audio paths (e.g. Oppo/OnePlus). +- **Robust asset migration:** bundled assets are extracted to `filesDir` once, versioned (`ASSETS_VERSION`) and copied atomically so a stale or partial copy is repaired automatically. + +See `ARCHITECTURE.md` for deployment details (16 KB page-size packaging, background-execution permissions on some OEMs). + +--- + +## Future Roadmap (not yet implemented) + +These items are aspirational targets, not current behavior: + +### Linguistic depth +- **Ezafe prediction & homograph disambiguation** — an ML model (GE2PE-style two-step G2P: large machine-generated pre-training + manual fine-tuning) to resolve مرد/مُرد-type ambiguities and predict ezafe in context, replacing the curated lexicon seed. +- **Richer normalizer** — abbreviation/date/currency expansion, DadmaTools-style preprocessing. + +### Model & inference +- **INT8 quantization** of the ONNX model (~4× smaller, faster CPU inference). +- **Hardware acceleration** — trial the NNAPI provider for NPU/GPU execution. +- **SOTA model evaluation** — Matcha-TTS (flow-matching) and Kokoro are drop-in candidates via Sherpa's existing `OfflineTtsMatchaModelConfig` / `OfflineTtsKokoroModelConfig`. +- **True pitch control** — a DSP/phase-vocoder stage (VITS exposes no native pitch parameter). +- **SSML expansion** — prosody/emphasis/phoneme tags. + +### Training methodology (reference) +- **Dataset:** the **ManaTTS** corpus (≈86 h, 44.1 kHz), cleaned with Spleeter. +- **Forced alignment:** multi-model ASR voting; strict CER thresholds (HIGH < 0.05, MIDDLE < 0.20) for data selection. + +## Aspirational KPIs + +| Metric | Target | Note | | :--- | :--- | :--- | -| **Real-Time Factor (RTF)** | > 3.0x | Speed on mid-range CPU/NPU | -| **Perceived Latency (CPL)** | < 180ms | Time until first audio buffer playback | -| **RAM Usage** | 10 - 20 MB | Active memory footprint during synthesis | -| **Storage (Disk)** | < 80 MB | Total model weights (Tacotron + WaveRNN) | -| **Mean Opinion Score (MOS)** | > 4.0 | Subjective naturalness vs. human speech | +| **Real-Time Factor** | > 3.0× | mid-range CPU/NPU | +| **Latency to first audio** | < 180 ms | enabled by streaming synthesis | +| **RAM usage** | 10–20 MB active | depends on quantization | +| **Storage** | < 80 MB | model weights | +| **Mean Opinion Score** | > 4.0 | subjective naturalness | --- -AvaCore aims to set a new standard for Persian accessibility on Android, providing a robust, offline, and high-quality voice for navigators, screen readers, and virtual assistants. +AvaCore aims to set a new standard for Persian accessibility on Android: a robust, offline, high-quality voice for navigators, screen readers and virtual assistants. diff --git a/app/build.gradle.kts b/app/build.gradle.kts index 6fc4899..1266577 100644 --- a/app/build.gradle.kts +++ b/app/build.gradle.kts @@ -40,6 +40,14 @@ android { @Suppress("UnstableApiUsage") kotlinOptions { jvmTarget = "17" + // Kotlin 2.0 compiles lambdas to invokedynamic by default, producing a + // synthetic lambda that only exposes the erased invoke(Object). Sherpa-ONNX's + // native generateWithCallback looks up the specialized invoke([F)Integer via + // JNI, so we force class-based lambdas/SAM conversions to keep that method. + freeCompilerArgs = freeCompilerArgs + listOf( + "-Xlambdas=class", + "-Xsam-conversions=class" + ) } buildFeatures { viewBinding = true diff --git a/app/src/main/assets/tts/lexicon.txt b/app/src/main/assets/tts/lexicon.txt new file mode 100644 index 0000000..61d18b9 --- /dev/null +++ b/app/src/main/assets/tts/lexicon.txt @@ -0,0 +1,36 @@ +# AvaCore pronunciation lexicon +# ------------------------------------------------------------------------------ +# Format (UTF-8): surface = replacement +# - One entry per line. Lines starting with '#' are comments. +# - "surface" is matched as a whole token/phrase (it will NOT match inside a +# larger word). Longer surfaces are tried before shorter ones. +# - "replacement" is what is actually fed to the G2P engine. Use explicit +# Persian short-vowel diacritics — eSpeak-NG honors them: +# zebar/fatha ◌َ (U+064E) e.g. مَرد +# zir/kasra ◌ِ (U+0650) e.g. کتابِ (also used to force EZAFE) +# pish/damma ◌ُ (U+064F) e.g. گُل +# +# PHILOSOPHY: keep entries HIGH-PRECISION. Only add a surface form that has a +# single correct pronunciation regardless of context (fixed expressions, lexical +# ezafe compounds, ligatures). Do NOT add context-dependent homographs here +# (e.g. مرد = mard/mord) — those need the future ML disambiguator, and a wrong +# guess is worse than none. +# +# This file is intentionally small and curated. Grow it with verified entries. +# ------------------------------------------------------------------------------ + +# --- Ligatures / presentation forms --- +﷼ = ریال +ﷺ = صلی الله علیه و آله و سلم +ﷲ = الله + +# --- Fixed-ezafe compounds (ezafe is lexical here, always pronounced) --- +مورد نظر = موردِ نظر +نقطه نظر = نقطهٔ نظر +وزارت کشور = وزارتِ کشور +وزارت بهداشت = وزارتِ بهداشت +حقوق بشر = حقوقِ بشر + +# --- Common symbols read as words --- +& = و +@ = اَت diff --git a/app/src/main/java/com/github/opscalehub/avacore/MainActivity.kt b/app/src/main/java/com/github/opscalehub/avacore/MainActivity.kt index cfe8c49..1eed686 100644 --- a/app/src/main/java/com/github/opscalehub/avacore/MainActivity.kt +++ b/app/src/main/java/com/github/opscalehub/avacore/MainActivity.kt @@ -61,7 +61,10 @@ class MainActivity : AppCompatActivity() { } private fun speakSample() { - val text = "این یک آزمایش از موتور بازگو کننده آوا است." + // Sample exercises number expansion (۱۴۰۴, ۳۵, ۲۰٪), punctuation pauses + // and multi-sentence streaming so the demo showcases the full pipeline. + val text = "سلام! این موتور بازگوکننده آوا است. " + + "امروز ۱۲ خرداد ۱۴۰۴ است، دمای هوا ۳۵ درجه و رطوبت ۲۰٪ می‌باشد." val result = tts?.speak(text, TextToSpeech.QUEUE_FLUSH, null, "sample_id") if (result == TextToSpeech.ERROR) { Toast.makeText(this, "Speech failed. Ensure AvaCore is selected.", Toast.LENGTH_SHORT).show() diff --git a/app/src/main/java/com/github/opscalehub/avacore/TtsDataCheckActivity.kt b/app/src/main/java/com/github/opscalehub/avacore/TtsDataCheckActivity.kt index c2bb7f4..ef47237 100644 --- a/app/src/main/java/com/github/opscalehub/avacore/TtsDataCheckActivity.kt +++ b/app/src/main/java/com/github/opscalehub/avacore/TtsDataCheckActivity.kt @@ -23,13 +23,13 @@ class TtsDataCheckActivity : Activity() { when (action) { TextToSpeech.Engine.ACTION_CHECK_TTS_DATA -> { // Return supported locales in different formats to ensure compatibility - val availableVoices = arrayListOf("fa", "fa-IR", "fas-IRN") + // A single, correctly-formed ISO-3 locale (lang-country): Persian/Iran. + // Listing 2-letter duplicates here makes the system TTS settings show + // several phantom "languages" for one voice, so we expose exactly one. + val availableVoices = arrayListOf("fas-IRN") resultIntent.putStringArrayListExtra(TextToSpeech.Engine.EXTRA_AVAILABLE_VOICES, availableVoices) resultIntent.putStringArrayListExtra(TextToSpeech.Engine.EXTRA_UNAVAILABLE_VOICES, arrayListOf()) - - // Some systems also look for these - resultIntent.putStringArrayListExtra("availableVoices", availableVoices) - + setResult(TextToSpeech.Engine.CHECK_VOICE_DATA_PASS, resultIntent) } "android.speech.tts.engine.GET_SAMPLE_TEXT" -> { diff --git a/app/src/main/java/com/github/opscalehub/avacore/nlp/Normalizer.kt b/app/src/main/java/com/github/opscalehub/avacore/nlp/Normalizer.kt index 7b4dba4..fc7c7d5 100644 --- a/app/src/main/java/com/github/opscalehub/avacore/nlp/Normalizer.kt +++ b/app/src/main/java/com/github/opscalehub/avacore/nlp/Normalizer.kt @@ -1,30 +1,91 @@ package com.github.opscalehub.avacore.nlp -import java.util.regex.Pattern - /** - * Handles text normalization for Persian language. - * Tasks: Number expansion, abbreviation expansion, character normalization. + * Character-level normalization for Persian text. + * + * The goal is to hand eSpeak-NG a clean, canonical Persian string: + * - fold Arabic-shaped letters to their Persian counterparts + * - normalize the ZWNJ (نیم‌فاصله) and stray control characters + * - strip kashida (ـ) and decorative combining marks that confuse G2P + * - canonicalize punctuation spacing and collapse whitespace + * + * Digit handling is intentionally NOT done here — numbers are expanded to words + * by [NumberToWords] first, and any leftover digits are folded afterwards. Keeping + * digits intact at this stage lets the number expander detect them reliably. + * + * Pure Kotlin, no dependencies, so it is unit-testable on the JVM. */ class Normalizer { fun normalize(text: String): String { - var normalized = text - normalized = normalizeCharacters(normalized) - normalized = expandNumbers(normalized) - return normalized + var s = text + s = foldCharacters(s) + s = normalizeZwnj(s) + s = stripDecorations(s) + s = normalizePunctuation(s) + s = collapseWhitespace(s) + return s.trim() + } + + /** Map Arabic-shaped letters and presentation forms to canonical Persian. */ + private fun foldCharacters(text: String): String { + val sb = StringBuilder(text.length) + for (c in text) { + sb.append( + when (c) { + 'ك' -> 'ک' // Arabic Kaf -> Persian Keheh + 'ي' -> 'ی' // Arabic Yeh -> Persian Yeh + 'ى' -> 'ی' // Alef Maksura -> Persian Yeh + 'ة' -> 'ه' // Teh Marbuta -> Heh + // Note: ۀ (heh with hamza) is left untouched — it carries the + // ezafe vowel and eSpeak-NG reads it; folding to ه would lose it. + else -> c + } + ) + } + return sb.toString() + } + + /** + * Normalize the zero-width non-joiner. Different inputs use ZWSP/ZWJ or plain + * spaces around نیم‌فاصله; collapse those variants to a single ZWNJ (U+200C) + * and drop other zero-width controls eSpeak does not understand. + */ + private fun normalizeZwnj(text: String): String { + return text + .replace('‏', ' ') // RIGHT-TO-LEFT MARK -> space + .replace('‎', ' ') // LEFT-TO-RIGHT MARK -> space + .replace("‍", "") // ZERO WIDTH JOINER -> drop + .replace("", "") // BOM / ZWNBSP -> drop + .replace(" ", " ") // NO-BREAK SPACE -> space + // collapse spaces that surround an existing ZWNJ + .replace(Regex("\\s*‌\\s*"), "‌") } - private fun normalizeCharacters(text: String): String { - // Replace Arabic Keheh and Yeh with Persian counterparts - return text.replace("\u0643", "\u06a9") // Arabic Kaf -> Persian Keheh - .replace("\u064a", "\u06cc") // Arabic Ya -> Persian Yeh - .replace("\u0649", "\u06cc") // Arabic Alef Maksura -> Persian Yeh + /** Remove kashida and standalone combining diacritics that hurt G2P. */ + private fun stripDecorations(text: String): String { + return text + .replace("ـ", "") // ـ Tatweel/Kashida + .replace("ٰ", "") // superscript alef + // Tanvin marks are not pronounced reliably by eSpeak; drop them. + .replace(Regex("[ًٌٍ]"), "") + } + + /** Canonicalize punctuation: fold Arabic forms and tidy spacing. */ + private fun normalizePunctuation(text: String): String { + return text + .replace(',', '،') // ASCII comma -> Persian comma (better pause) + .replace(';', '؛') // ASCII semicolon -> Persian semicolon + .replace('?', '؟') // ASCII question -> Persian question + // no space before, one space after Persian punctuation + .replace(Regex("\\s*([،؛؟!:])"), "$1") + .replace(Regex("([،؛؟!:])(?=\\S)"), "$1 ") } - private fun expandNumbers(text: String): String { - // This is a placeholder for a complex number-to-words converter - // In a real implementation, you'd use a rule-based system or a dictionary + private fun collapseWhitespace(text: String): String { return text + .replace(Regex("[\\t\\x0B\\f\\r]"), " ") + .replace(Regex(" {2,}"), " ") + .replace(Regex("\\n{3,}"), "\n\n") } } diff --git a/app/src/main/java/com/github/opscalehub/avacore/nlp/NumberToWords.kt b/app/src/main/java/com/github/opscalehub/avacore/nlp/NumberToWords.kt new file mode 100644 index 0000000..a889fe0 --- /dev/null +++ b/app/src/main/java/com/github/opscalehub/avacore/nlp/NumberToWords.kt @@ -0,0 +1,190 @@ +package com.github.opscalehub.avacore.nlp + +/** + * Converts numeric tokens inside Persian text into spoken Persian words. + * + * Handles: + * - cardinals of arbitrary length ("۱۴۰۳" -> "هزار و چهارصد و سه") + * - decimals ("۳٫۱۴" / "3.14" -> "سه ممیز چهارده") + * - percent ("۵۰٪" / "50%" -> "پنجاه درصد") + * - ordinals written with an attached suffix ("۲م" -> "دوم", "۳ام" -> "سوم") + * - negative sign ("-۵" / "−۵" -> "منفی پنج") + * - Persian (۰-۹), Arabic-Indic (٠-٩) and ASCII (0-9) digits + * - thousands separators ("," and "٬") which are stripped before conversion + * + * Pure Kotlin, no Android/3rd-party dependencies, so it is unit-testable on the JVM. + */ +object NumberToWords { + + private val ONES = arrayOf( + "", "یک", "دو", "سه", "چهار", "پنج", "شش", "هفت", "هشت", "نه" + ) + private val TEENS = arrayOf( + "ده", "یازده", "دوازده", "سیزده", "چهارده", "پانزده", + "شانزده", "هفده", "هجده", "نوزده" + ) + private val TENS = arrayOf( + "", "", "بیست", "سی", "چهل", "پنجاه", "شصت", "هفتاد", "هشتاد", "نود" + ) + private val HUNDREDS = arrayOf( + "", "صد", "دویست", "سیصد", "چهارصد", "پانصد", + "ششصد", "هفتصد", "هشتصد", "نهصد" + ) + // Scale words for each group of three digits (10^3, 10^6, ...). + private val SCALES = arrayOf( + "", "هزار", "میلیون", "میلیارد", "بیلیون", "بیلیارد", "تریلیون" + ) + + private const val AND = " و " + private const val ZERO = "صفر" + private const val NEGATIVE = "منفی" + private const val POINT = "ممیز" + private const val PERCENT = "درصد" + + // Matches an optional sign, a digit run with optional separators, optional + // percent sign, and an optional attached ordinal suffix. + // Group 1: sign, 2: number body, 3: percent, 4: ordinal suffix. + private val NUMBER = Regex( + "([-−])?" + + "([0-9۰-۹٠-٩]+(?:[.,٫٬][0-9۰-۹٠-٩]+)*)" + + "([%٪])?" + + // ordinal suffix only counts when it is followed by a word boundary, + // so "۱۰۰میلیون" is not misread as an ordinal of 100. + "(?:(ام|م)(?![؀-ۿ]))?" + ) + + fun expand(text: String): String { + return NUMBER.replace(text) { m -> + val sign = m.groupValues[1] + val body = foldDigits(m.groupValues[2]) + val percent = m.groupValues[3].isNotEmpty() + val ordinal = m.groupValues[4].isNotEmpty() + + // Separate decimal part. Thousands separators (',' '٬') are removed; + // decimal separators ('.' '٫') split integer/fraction. + val cleaned = body.replace(",", "").replace("٬", "") + val parts = cleaned.split('.', '٫') + val intPart = parts[0] + val fracPart = if (parts.size > 1) parts[1] else null + + val sb = StringBuilder() + if (sign.isNotEmpty()) sb.append(NEGATIVE).append(' ') + + if (ordinal && fracPart == null) { + sb.append(toOrdinal(intPart)) + } else { + sb.append(toCardinal(intPart)) + if (fracPart != null) { + sb.append(' ').append(POINT).append(' ').append(fractionToWords(fracPart)) + } + } + if (percent) sb.append(' ').append(PERCENT) + sb.toString() + } + } + + /** Fold Persian (۰-۹) and Arabic-Indic (٠-٩) digits to ASCII; leave separators. */ + fun foldDigits(s: String): String { + val out = StringBuilder(s.length) + for (c in s) { + out.append( + when (c) { + in '۰'..'۹' -> '0' + (c - '۰') // Persian + in '٠'..'٩' -> '0' + (c - '٠') // Arabic-Indic + else -> c + } + ) + } + return out.toString() + } + + /** Convert an ASCII-digit string to Persian cardinal words. */ + fun toCardinal(digits: String): String { + val n = digits.trimStart('0') + if (n.isEmpty()) return ZERO + + // Split into 3-digit groups from the right. + val groups = ArrayList() + var i = n.length + while (i > 0) { + val start = (i - 3).coerceAtLeast(0) + groups.add(0, n.substring(start, i)) + i = start + } + + val numGroups = groups.size + val pieces = ArrayList() + for ((idx, g) in groups.withIndex()) { + val value = g.toInt() + if (value == 0) continue + val scaleIdx = numGroups - idx - 1 + val words = threeDigitsToWords(value) + val scale = SCALES.getOrElse(scaleIdx) { bigScale(scaleIdx) } + val piece = when { + scale.isEmpty() -> words + // ۱۰۰۰ is read "هزار", not "یک هزار" (but ۱۰۰۰۰۰۰ is "یک میلیون"). + scaleIdx == 1 && value == 1 -> scale + else -> "$words $scale" + } + pieces.add(piece) + } + return pieces.joinToString(AND) + } + + /** Ordinal form, e.g. "۲" -> "دوم", "۳" -> "سوم", "۳۱" -> "سی و یکم". */ + fun toOrdinal(digits: String): String { + val cardinal = toCardinal(digits) + // Special last-token replacements per Persian ordinal rules. + return when { + cardinal == "یک" -> "اول" + cardinal.endsWith("سه") -> cardinal.dropLast(2) + "سوم" + cardinal.endsWith("سی") -> cardinal + "‌ام" // سی‌ام + cardinal.endsWith("نُه") || cardinal.endsWith("نه") -> cardinal + "م" + else -> cardinal + "م" + } + } + + private fun threeDigitsToWords(value: Int): String { + val parts = ArrayList(3) + val h = value / 100 + val rest = value % 100 + if (h > 0) parts.add(HUNDREDS[h]) + if (rest in 1..9) { + parts.add(ONES[rest]) + } else if (rest in 10..19) { + parts.add(TEENS[rest - 10]) + } else if (rest >= 20) { + parts.add(TENS[rest / 10]) + if (rest % 10 != 0) parts.add(ONES[rest % 10]) + } + return parts.joinToString(AND) + } + + /** + * Read the fractional part. If it has leading zeros they are spoken as + * "صفر" individually, then the remaining significant part as a cardinal, + * which matches natural Persian reading ("۰۵" -> "صفر پنج"). + */ + private fun fractionToWords(frac: String): String { + val sb = StringBuilder() + var idx = 0 + while (idx < frac.length && frac[idx] == '0') { + if (sb.isNotEmpty()) sb.append(' ') + sb.append(ZERO) + idx++ + } + val rest = frac.substring(idx) + if (rest.isNotEmpty()) { + if (sb.isNotEmpty()) sb.append(' ') + sb.append(toCardinal(rest)) + } else if (sb.isEmpty()) { + sb.append(ZERO) + } + return sb.toString() + } + + // Fallback for numbers larger than the named scales: read each group with a + // generic "× هزار^k" is impractical, so we read the whole number group-wise + // without a scale word (rare in practice). Kept defensive. + private fun bigScale(@Suppress("UNUSED_PARAMETER") idx: Int): String = "" +} diff --git a/app/src/main/java/com/github/opscalehub/avacore/nlp/PronunciationLexicon.kt b/app/src/main/java/com/github/opscalehub/avacore/nlp/PronunciationLexicon.kt new file mode 100644 index 0000000..7f0c244 --- /dev/null +++ b/app/src/main/java/com/github/opscalehub/avacore/nlp/PronunciationLexicon.kt @@ -0,0 +1,80 @@ +package com.github.opscalehub.avacore.nlp + +import java.io.BufferedReader +import java.io.InputStream + +/** + * A curated, high-precision pronunciation lexicon. + * + * eSpeak-NG's Persian G2P cannot restore unwritten short vowels or the linking + * *ezafe* (کسرهٔ اضافه) on its own. This lexicon lets us override pronunciation for + * specific surface forms by substituting a diacritized / re-spelled form that + * eSpeak reads correctly (it honors explicit short-vowel diacritics). + * + * It is deliberately conservative: an entry only fires on an exact, boundary- + * delimited surface match, so a wrong ezafe is never guessed. This is the seed + * that a future ML-based ezafe/homograph model (GE2PE-style) would replace. + * + * File format (assets/tts/lexicon.txt), UTF-8: + * # comment lines start with '#' + * surface = replacement + * Both single words and multi-word phrases are supported. Longer keys win. + */ +class PronunciationLexicon private constructor( + private val entries: List +) { + private data class Entry(val pattern: Regex, val replacement: String) + + fun apply(text: String): String { + if (entries.isEmpty()) return text + var s = text + for (e in entries) { + s = e.pattern.replace(s, Regex.escapeReplacement(e.replacement)) + } + return s + } + + val size: Int get() = entries.size + + companion object { + // Persian/Arabic letter range used to anchor whole-token boundaries. + private const val LETTER = "؀-ۿﭐ-ﹾ" + + fun fromStream(input: InputStream?): PronunciationLexicon { + if (input == null) return PronunciationLexicon(emptyList()) + val raw = LinkedHashMap() + input.bufferedReader().use { r -> parseInto(r, raw) } + // Longest surface forms first so phrases match before their sub-words. + val entries = raw.entries + .sortedByDescending { it.key.length } + .map { (k, v) -> Entry(boundaryRegex(k), v) } + return PronunciationLexicon(entries) + } + + /** Test/seed helper: build directly from a map. */ + fun fromMap(map: Map): PronunciationLexicon { + val entries = map.entries + .sortedByDescending { it.key.length } + .map { (k, v) -> Entry(boundaryRegex(k), v) } + return PronunciationLexicon(entries) + } + + private fun parseInto(reader: BufferedReader, out: MutableMap) { + reader.lineSequence().forEach { line -> + val trimmed = line.trim() + if (trimmed.isEmpty() || trimmed.startsWith("#")) return@forEach + val eq = trimmed.indexOf('=') + if (eq <= 0) return@forEach + val key = trimmed.substring(0, eq).trim() + val value = trimmed.substring(eq + 1).trim() + if (key.isNotEmpty() && value.isNotEmpty()) out[key] = value + } + } + + private fun boundaryRegex(surface: String): Regex { + val esc = Regex.escape(surface) + // Match only when not glued to another Persian/Arabic letter on either side. + return Regex("(? { + val units = ArrayList() + val sb = StringBuilder() + var newlineRun = 0 + + fun flush(pause: Int) { + val unit = sb.toString().trim() + sb.setLength(0) + if (unit.isNotEmpty()) units.add(SpeakUnit(unit, pause)) + } + + var i = 0 + while (i < text.length) { + val c = text[i] + when { + c == '\n' -> { + newlineRun++ + if (newlineRun >= 2) { + flush(PAUSE_PARAGRAPH) + newlineRun = 0 + } else if (sb.isNotEmpty()) { + // single newline acts as a soft sentence break + flush(PAUSE_SENTENCE) + } + } + c in SENTENCE_END -> { + sb.append(c) + // absorb repeated terminators like "؟!" or "..." + while (i + 1 < text.length && text[i + 1] in SENTENCE_END) { + sb.append(text[++i]) + } + flush(PAUSE_SENTENCE) + newlineRun = 0 + } + c in CLAUSE_END -> { + sb.append(c) + flush(PAUSE_CLAUSE) + newlineRun = 0 + } + else -> { + sb.append(c) + newlineRun = 0 + if (sb.length >= MAX_UNIT_CHARS) { + // break at the last space at or before the cap + val cut = sb.lastIndexOf(" ") + if (cut > 0) { + val head = sb.substring(0, cut) + val tail = sb.substring(cut + 1) + sb.setLength(0) + sb.append(head) + flush(PAUSE_SOFT) + sb.append(tail) + } else { + flush(PAUSE_SOFT) + } + } + } + } + i++ + } + flush(PAUSE_SENTENCE) + return units + } +} diff --git a/app/src/main/java/com/github/opscalehub/avacore/nlp/Ssml.kt b/app/src/main/java/com/github/opscalehub/avacore/nlp/Ssml.kt new file mode 100644 index 0000000..db2ed26 --- /dev/null +++ b/app/src/main/java/com/github/opscalehub/avacore/nlp/Ssml.kt @@ -0,0 +1,97 @@ +package com.github.opscalehub.avacore.nlp + +/** + * Minimal SSML support. + * + * Android passes SSML through to the engine unchanged, and many accessibility + * tools emit it. We support the subset that matters for a TTS engine: + * - ... wrapper (stripped) + * - inserts a pause + * - inserts a pause by named strength + * - X spells X out + * - all other tags are stripped, their text content kept + * + * Plain (non-SSML) text is returned as a single pass-through segment. + */ +object Ssml { + + /** One chunk of SSML content plus an optional forced pause after it. */ + data class Segment(val text: String, val breakAfterMs: Int, val spellOut: Boolean) + + private val TAG = Regex("<[^>]+>") + private val BREAK_TIME = Regex("time\\s*=\\s*\"?([0-9.]+)(ms|s)?\"?", RegexOption.IGNORE_CASE) + private val BREAK_STRENGTH = Regex("strength\\s*=\\s*\"?([a-z-]+)\"?", RegexOption.IGNORE_CASE) + private val INTERPRET_AS = Regex("interpret-as\\s*=\\s*\"?([a-z-]+)\"?", RegexOption.IGNORE_CASE) + + fun isSsml(text: String): Boolean { + val t = text.trimStart() + return t.startsWith(" { + if (!isSsml(text)) return listOf(Segment(text, 0, false)) + + val segments = ArrayList() + val buf = StringBuilder() + var spellOut = false + var lastIndex = 0 + + fun flushText(breakAfterMs: Int) { + val t = buf.toString() + buf.setLength(0) + if (t.isNotBlank() || breakAfterMs > 0) { + segments.add(Segment(t.trim(), breakAfterMs, spellOut)) + } + } + + for (m in TAG.findAll(text)) { + // text between previous tag and this one + buf.append(text, lastIndex, m.range.first) + lastIndex = m.range.last + 1 + + val tag = m.value + val name = tagName(tag) + when (name) { + "break" -> flushText(breakMs(tag)) + "say-as" -> { + if (!tag.startsWith(" { /* strip; keep accumulated text */ } + } + } + if (lastIndex < text.length) buf.append(text, lastIndex, text.length) + flushText(0) + return segments.ifEmpty { listOf(Segment("", 0, false)) } + } + + // (tag parsing helpers below) + + private fun tagName(tag: String): String { + val inner = tag.trim('<', '>', '/', ' ') + val end = inner.indexOfFirst { it == ' ' || it == '/' } + return (if (end >= 0) inner.substring(0, end) else inner).lowercase() + } + + private fun breakMs(tag: String): Int { + BREAK_TIME.find(tag)?.let { mt -> + val value = mt.groupValues[1].toFloatOrNull() ?: return@let + val unit = mt.groupValues[2].lowercase() + return if (unit == "s") (value * 1000).toInt() else value.toInt() + } + return when (BREAK_STRENGTH.find(tag)?.groupValues?.get(1)?.lowercase()) { + "none", "x-weak" -> 0 + "weak" -> 150 + "medium", null -> 300 + "strong" -> 500 + "x-strong" -> 800 + else -> 300 + } + } +} diff --git a/app/src/main/java/com/github/opscalehub/avacore/nlp/TextProcessor.kt b/app/src/main/java/com/github/opscalehub/avacore/nlp/TextProcessor.kt new file mode 100644 index 0000000..4432c73 --- /dev/null +++ b/app/src/main/java/com/github/opscalehub/avacore/nlp/TextProcessor.kt @@ -0,0 +1,76 @@ +package com.github.opscalehub.avacore.nlp + +/** + * A single piece of text to synthesize, plus the silence to append after it. + * The service streams these one by one. + */ +data class SpeakUnit(val text: String, val trailingPauseMs: Int) + +/** + * The Persian text front-end: turns raw input (plain or SSML) into an ordered + * list of [SpeakUnit]s ready for the neural engine. + * + * Pipeline order matters: + * 1. SSML parse — split into content segments + forced pauses + * 2. NumberToWords.expand — BEFORE punctuation folding, so "1,000"/"۱٬۰۰۰" + * thousands separators are still intact + * 3. Normalizer.normalize — character/ZWNJ/punctuation/whitespace cleanup + * 4. fold leftover digits — any stray digits eSpeak would mishandle -> ASCII + * 5. PronunciationLexicon — high-precision pronunciation/ezafe overrides + * 6. SentenceSegmenter — break into streamable units with prosodic pauses + */ +class TextProcessor( + private val lexicon: PronunciationLexicon, + private val normalizer: Normalizer = Normalizer() +) { + + fun process(raw: String): List { + if (raw.isBlank()) return emptyList() + + val out = ArrayList() + for (seg in Ssml.parse(raw)) { + if (seg.text.isNotBlank()) { + val prepared = + if (seg.spellOut) spellOut(seg.text) + else pipeline(seg.text) + + val units = SentenceSegmenter.split(prepared) + out.addAll(units) + } + // A forced SSML attaches its pause to the preceding unit, + // or stands alone as a silent unit if there is nothing before it. + if (seg.breakAfterMs > 0) { + if (out.isNotEmpty()) { + val last = out.removeAt(out.size - 1) + out.add(last.copy(trailingPauseMs = last.trailingPauseMs + seg.breakAfterMs)) + } else { + out.add(SpeakUnit("", seg.breakAfterMs)) + } + } + } + return out + } + + private fun pipeline(text: String): String { + var s = NumberToWords.expand(text) + s = normalizer.normalize(s) + s = NumberToWords.foldDigits(s) // fold any digits left after expansion + s = lexicon.apply(s) + return s + } + + /** For SSML say-as characters/digits: read each character individually. */ + private fun spellOut(text: String): String { + val sb = StringBuilder() + for (c in NumberToWords.foldDigits(text.trim())) { + if (c.isWhitespace()) continue + if (c in '0'..'9') { + sb.append(NumberToWords.toCardinal(c.toString())) + } else { + sb.append(c) + } + sb.append("، ") // comma forces a short gap between spelled items + } + return sb.toString() + } +} diff --git a/app/src/main/java/com/github/opscalehub/avacore/service/AvaTtsService.kt b/app/src/main/java/com/github/opscalehub/avacore/service/AvaTtsService.kt index 220c169..0670f38 100644 --- a/app/src/main/java/com/github/opscalehub/avacore/service/AvaTtsService.kt +++ b/app/src/main/java/com/github/opscalehub/avacore/service/AvaTtsService.kt @@ -1,11 +1,14 @@ package com.github.opscalehub.avacore.service +import android.media.AudioFormat import android.speech.tts.SynthesisCallback import android.speech.tts.SynthesisRequest import android.speech.tts.TextToSpeech import android.speech.tts.TextToSpeechService import android.speech.tts.Voice import android.util.Log +import com.github.opscalehub.avacore.nlp.PronunciationLexicon +import com.github.opscalehub.avacore.nlp.TextProcessor import com.k2fsa.sherpa.onnx.OfflineTts import com.k2fsa.sherpa.onnx.OfflineTtsConfig import com.k2fsa.sherpa.onnx.OfflineTtsModelConfig @@ -13,20 +16,33 @@ import com.k2fsa.sherpa.onnx.OfflineTtsVitsModelConfig import java.io.File import java.io.FileOutputStream import java.util.Locale +import java.util.concurrent.CountDownLatch +import java.util.concurrent.TimeUnit import java.util.concurrent.atomic.AtomicBoolean import kotlin.concurrent.thread /** - * AvaTtsService: A robust, offline-first TTS engine for Persian. - * Optimized for performance and resilience on Android devices. + * AvaTtsService: an offline-first, streaming Persian TTS engine. + * + * Synthesis is streamed sentence-by-sentence via Sherpa's generateWithCallback, + * so the first audio plays almost immediately and stop requests are honored + * mid-utterance. A Persian text front-end ([TextProcessor]) normalizes text, + * expands numbers, applies a pronunciation lexicon and segments into prosodic + * units before the audio model ever runs. */ class AvaTtsService : TextToSpeechService() { private val persianLocale = Locale("fa", "IR") private val voiceName = "fa-ir-ava-premium" - + @Volatile private var tts: OfflineTts? = null + @Volatile + private var textProcessor: TextProcessor? = null + + // Counts down once initialization finishes (success or failure). + private val initLatch = CountDownLatch(1) + private val isInterrupted = AtomicBoolean(false) private val isInitializing = AtomicBoolean(false) private val isDestroyed = AtomicBoolean(false) @@ -36,9 +52,22 @@ class AvaTtsService : TextToSpeechService() { private const val ASSET_SUBDIR = "tts" private const val MODEL_NAME = "persian_model.onnx" private const val TOKENS_NAME = "tokens.txt" + private const val LEXICON_NAME = "lexicon.txt" private const val ESPEAK_DIR = "espeak-ng-data" - private const val MAX_BUFFER_SIZE = 8192 - private const val INIT_RETRY_COUNT = 50 // 10 seconds total wait + private const val VERSION_MARKER = ".assets_version" + + // Bump whenever the bundled model/tokens/espeak/lexicon assets change so + // stale copies in filesDir are re-extracted on the next launch. + private const val ASSETS_VERSION = 2 + + // Keep chunks small; some OEM audio paths reject large buffers. + private const val MAX_CHUNK_BYTES = 8192 + private const val INIT_WAIT_MS = 10_000L + + // Map Android speech rate (percent of normal, 100 = default) to the + // engine speed multiplier, clamped to a sane musical range. + private const val MIN_SPEED = 0.5f + private const val MAX_SPEED = 2.0f } override fun onCreate() { @@ -48,98 +77,130 @@ class AvaTtsService : TextToSpeechService() { } private fun ensureEngineInitialized() { - if (isDestroyed.get() || tts != null || isInitializing.get()) return - - isInitializing.set(true) - thread(start = true, name = "TtsInitializer", priority = Thread.MAX_PRIORITY) { + if (isDestroyed.get() || tts != null) return + if (!isInitializing.compareAndSet(false, true)) return + + thread(start = true, name = "TtsInitializer") { try { prepareAndInitialize() + } catch (e: Throwable) { + Log.e(TAG, "CRITICAL: TTS init failed", e) } finally { isInitializing.set(false) + initLatch.countDown() } } } private fun prepareAndInitialize() { - try { - val modelFile = copyAssetToFile(MODEL_NAME) - val tokensFile = copyAssetToFile(TOKENS_NAME) - val espeakDir = copyAssetDirToFiles(ESPEAK_DIR) - - if (isDestroyed.get()) return - - val vitsConfig = OfflineTtsVitsModelConfig( - model = modelFile.absolutePath, - lexicon = "", - tokens = tokensFile.absolutePath, - dataDir = espeakDir.absolutePath, - noiseScale = 0.667f, - noiseScaleW = 0.8f, - lengthScale = 1.0f - ) - - val cpuThreads = (Runtime.getRuntime().availableProcessors() / 2).coerceAtLeast(1).coerceAtMost(4) - Log.d(TAG, "Initializing with $cpuThreads threads") - - val modelConfig = OfflineTtsModelConfig( - vits = vitsConfig, - numThreads = cpuThreads, - debug = false, - provider = "cpu" - ) - - val config = OfflineTtsConfig(model = modelConfig) - val newTts = OfflineTts(config = config) - - if (isDestroyed.get()) { - newTts.release() - } else { - tts = newTts - Log.i(TAG, "AvaCore TTS Engine ready. Sample Rate: ${tts?.sampleRate()}") - } + ensureAssets() + if (isDestroyed.get()) return + + val modelFile = File(filesDir, MODEL_NAME) + val tokensFile = File(filesDir, TOKENS_NAME) + val espeakDir = File(filesDir, ESPEAK_DIR) + + val vitsConfig = OfflineTtsVitsModelConfig( + model = modelFile.absolutePath, + lexicon = "", + tokens = tokensFile.absolutePath, + dataDir = espeakDir.absolutePath, + noiseScale = 0.667f, + noiseScaleW = 0.8f, + lengthScale = 1.0f + ) + + val cpuThreads = (Runtime.getRuntime().availableProcessors() / 2) + .coerceIn(1, 4) + Log.d(TAG, "Initializing engine with $cpuThreads threads") + + val modelConfig = OfflineTtsModelConfig( + vits = vitsConfig, + numThreads = cpuThreads, + debug = false, + provider = "cpu" + ) + + val newTts = OfflineTts(config = OfflineTtsConfig(model = modelConfig)) + + // Build the Persian text front-end (lexicon is optional / best-effort). + val lexicon = try { + PronunciationLexicon.fromStream(File(filesDir, LEXICON_NAME).takeIf { it.exists() }?.inputStream()) } catch (e: Exception) { - Log.e(TAG, "CRITICAL: Failed to initialize TTS engine", e) + Log.w(TAG, "Lexicon load failed; continuing without it", e) + PronunciationLexicon.fromStream(null) + } + textProcessor = TextProcessor(lexicon) + + if (isDestroyed.get()) { + newTts.release() + } else { + tts = newTts + Log.i(TAG, "AvaCore ready. SR=${newTts.sampleRate()} lexicon=${lexicon.size}") } } - private fun copyAssetToFile(fileName: String): File { - val targetFile = File(filesDir, fileName) - if (!targetFile.exists() || targetFile.length() == 0L) { - Log.d(TAG, "Copying asset: $fileName") - assets.open("$ASSET_SUBDIR/$fileName").use { input -> - FileOutputStream(targetFile).use { output -> - input.copyTo(output) - } - } + // ------------------------------------------------------------------ + // Asset migration (versioned + atomic) + // ------------------------------------------------------------------ + + private fun ensureAssets() { + val marker = File(filesDir, VERSION_MARKER) + val current = marker.takeIf { it.exists() }?.readText()?.trim()?.toIntOrNull() + if (current == ASSETS_VERSION && + File(filesDir, MODEL_NAME).length() > 0L && + File(filesDir, TOKENS_NAME).exists() && + File(filesDir, ESPEAK_DIR).isDirectory + ) { + return // up to date } - return targetFile + + Log.i(TAG, "Extracting assets (have=$current want=$ASSETS_VERSION)") + marker.delete() // invalidate until the full copy succeeds + + copyAssetAtomic(MODEL_NAME) + copyAssetAtomic(TOKENS_NAME) + copyAssetAtomic(LEXICON_NAME) + File(filesDir, ESPEAK_DIR).deleteRecursively() + copyAssetDir("$ASSET_SUBDIR/$ESPEAK_DIR", File(filesDir, ESPEAK_DIR)) + + marker.writeText(ASSETS_VERSION.toString()) } - private fun copyAssetDirToFiles(dirName: String): File { - val targetDir = File(filesDir, dirName) - if (!targetDir.exists()) { - targetDir.mkdirs() - copyAssetsRecursively("$ASSET_SUBDIR/$dirName", targetDir) + /** Copy a single asset via a temp file + rename so a crash never leaves a + * half-written target that later looks "present". */ + private fun copyAssetAtomic(fileName: String) { + val target = File(filesDir, fileName) + val tmp = File(filesDir, "$fileName.tmp") + assets.open("$ASSET_SUBDIR/$fileName").use { input -> + FileOutputStream(tmp).use { output -> input.copyTo(output) } + } + if (target.exists()) target.delete() + if (!tmp.renameTo(target)) { + tmp.copyTo(target, overwrite = true) + tmp.delete() } - return targetDir } - private fun copyAssetsRecursively(path: String, target: File) { - val list = assets.list(path) ?: return - if (list.isEmpty()) { + private fun copyAssetDir(path: String, target: File) { + val children = assets.list(path) ?: return + if (children.isEmpty()) { + // leaf file assets.open(path).use { input -> - FileOutputStream(target).use { output -> - input.copyTo(output) - } + FileOutputStream(target).use { output -> input.copyTo(output) } } } else { if (!target.exists()) target.mkdirs() - for (file in list) { - copyAssetsRecursively("$path/$file", File(target, file)) + for (child in children) { + copyAssetDir("$path/$child", File(target, child)) } } } + // ------------------------------------------------------------------ + // TTS framework hooks + // ------------------------------------------------------------------ + override fun onIsLanguageAvailable(lang: String?, country: String?, variant: String?): Int { return if (lang != null && (lang.equals("fa", true) || lang.equals("fas", true))) { TextToSpeech.LANG_COUNTRY_AVAILABLE @@ -148,103 +209,143 @@ class AvaTtsService : TextToSpeechService() { } } - override fun onGetLanguage(): Array { - return arrayOf("fa", "IR", "") - } + override fun onGetLanguage(): Array = arrayOf("fa", "IR", "") - override fun onLoadLanguage(lang: String?, country: String?, variant: String?): Int { - return onIsLanguageAvailable(lang, country, variant) - } + override fun onLoadLanguage(lang: String?, country: String?, variant: String?): Int = + onIsLanguageAvailable(lang, country, variant) - override fun onGetVoices(): MutableList { - return mutableListOf(Voice(voiceName, persianLocale, Voice.QUALITY_VERY_HIGH, Voice.LATENCY_NORMAL, false, mutableSetOf())) - } + override fun onGetVoices(): MutableList = mutableListOf( + Voice(voiceName, persianLocale, Voice.QUALITY_VERY_HIGH, Voice.LATENCY_NORMAL, false, mutableSetOf()) + ) - override fun onGetDefaultVoiceNameFor(lang: String?, country: String?, variant: String?): String? { - return if (onIsLanguageAvailable(lang, country, variant) >= TextToSpeech.LANG_AVAILABLE) voiceName else null - } + override fun onGetDefaultVoiceNameFor(lang: String?, country: String?, variant: String?): String? = + if (onIsLanguageAvailable(lang, country, variant) >= TextToSpeech.LANG_AVAILABLE) voiceName else null override fun onStop() { - Log.d(TAG, "onStop: Interrupting synthesis") + Log.d(TAG, "onStop: interrupting synthesis") isInterrupted.set(true) } override fun onSynthesizeText(request: SynthesisRequest?, callback: SynthesisCallback?) { - val rawText = request?.charSequenceText?.toString() ?: "" - if (callback == null || rawText.isBlank()) { - callback?.done() + if (callback == null) return + val rawText = request?.charSequenceText?.toString().orEmpty() + if (request == null || rawText.isBlank()) { + callback.done() return } - + isInterrupted.set(false) - - // Wait for engine if it's still initializing - var engine = tts - var retry = 0 - while (engine == null && retry < INIT_RETRY_COUNT) { - if (isInterrupted.get() || isDestroyed.get()) return - Log.d(TAG, "Waiting for engine initialization... ($retry)") - Thread.sleep(200) - engine = tts - retry++ - } - if (engine == null) { - Log.e(TAG, "Engine failed to initialize in time") + val engine = awaitEngine() + val processor = textProcessor + if (engine == null || processor == null) { + Log.e(TAG, "Engine not ready in time") callback.error(TextToSpeech.ERROR_SERVICE) - ensureEngineInitialized() + ensureEngineInitialized() return } - + + val speed = computeSpeed(request) + val sampleRate = engine.sampleRate() + // request.pitch is read but intentionally not applied: VITS has no pitch + // control and naive resampling shifts formants and degrades quality. + try { - val audio = engine.generate(rawText) - - if (audio != null && audio.samples.isNotEmpty()) { - if (isInterrupted.get() || isDestroyed.get()) return - - val sampleRate = audio.sampleRate - callback.start(sampleRate, android.media.AudioFormat.ENCODING_PCM_16BIT, 1) - - val samples = audio.samples - val totalSamples = samples.size - val buffer = java.nio.ByteBuffer.allocate(MAX_BUFFER_SIZE) - .order(java.nio.ByteOrder.LITTLE_ENDIAN) - - var i = 0 - while (i < totalSamples && !isInterrupted.get() && !isDestroyed.get()) { - buffer.clear() - while (i < totalSamples && buffer.hasRemaining()) { - val sample = (samples[i] * 32767.0f).toInt().coerceIn(-32768, 32767).toShort() - buffer.putShort(sample) - i++ - } - buffer.flip() - val data = ByteArray(buffer.remaining()) - buffer.get(data) - callback.audioAvailable(data, 0, data.size) - } + val units = processor.process(rawText) + if (units.isEmpty()) { + callback.done() + return + } + + callback.start(sampleRate, AudioFormat.ENCODING_PCM_16BIT, 1) + val maxChunk = (minOf(callback.maxBufferSize, MAX_CHUNK_BYTES) and 1.inv()) + .coerceAtLeast(2) + val writer = PcmWriter(callback, maxChunk) - if (!isInterrupted.get() && !isDestroyed.get()) { - callback.done() + for (unit in units) { + if (isInterrupted.get() || isDestroyed.get()) break + if (unit.text.isNotBlank()) { + engine.generateWithCallback(unit.text, 0, speed) { chunk -> + writer.write(chunk) + if (isInterrupted.get() || isDestroyed.get()) 0 else 1 + } } - } else { - Log.w(TAG, "Synthesis produced no audio") - callback.error(TextToSpeech.ERROR_SYNTHESIS) + if (isInterrupted.get() || isDestroyed.get()) break + if (unit.trailingPauseMs > 0) writer.writeSilence(sampleRate, unit.trailingPauseMs) } + + callback.done() } catch (e: OutOfMemoryError) { Log.e(TAG, "OOM during synthesis", e) - callback.error(TextToSpeech.ERROR_NOT_INSTALLED_YET) + callback.error(TextToSpeech.ERROR_OUTPUT) } catch (e: Exception) { Log.e(TAG, "Synthesis failed", e) callback.error() } } - + + private fun awaitEngine(): OfflineTts? { + tts?.let { return it } + ensureEngineInitialized() + try { + initLatch.await(INIT_WAIT_MS, TimeUnit.MILLISECONDS) + } catch (e: InterruptedException) { + Thread.currentThread().interrupt() + } + return tts + } + + private fun computeSpeed(request: SynthesisRequest): Float { + val rate = request.speechRate + val speed = if (rate <= 0) 1.0f else rate / 100.0f + return speed.coerceIn(MIN_SPEED, MAX_SPEED) + } + override fun onDestroy() { - Log.d(TAG, "onDestroy: Releasing engine") + Log.d(TAG, "onDestroy: releasing engine") isDestroyed.set(true) + isInterrupted.set(true) tts?.release() tts = null super.onDestroy() } + + /** + * Streams float samples to the framework as little-endian PCM16, reusing a + * single scratch buffer and flushing in chunks no larger than the framework + * allows. + */ + private class PcmWriter( + private val callback: SynthesisCallback, + private val maxChunkBytes: Int + ) { + private val scratch = ByteArray(maxChunkBytes) + private val samplesPerChunk = maxChunkBytes / 2 + + fun write(samples: FloatArray) { + var i = 0 + while (i < samples.size) { + val n = minOf(samplesPerChunk, samples.size - i) + var b = 0 + for (k in 0 until n) { + val s = (samples[i + k].coerceIn(-1f, 1f) * 32767f).toInt() + scratch[b++] = (s and 0xFF).toByte() + scratch[b++] = ((s shr 8) and 0xFF).toByte() + } + callback.audioAvailable(scratch, 0, b) + i += n + } + } + + fun writeSilence(sampleRate: Int, durationMs: Int) { + var remaining = (sampleRate.toLong() * durationMs / 1000).toInt() * 2 // bytes + // scratch may hold stale data; zero only what we use each pass. + while (remaining > 0) { + val n = minOf(maxChunkBytes, remaining) + for (k in 0 until n) scratch[k] = 0 + callback.audioAvailable(scratch, 0, n) + remaining -= n + } + } + } } diff --git a/app/src/test/java/com/github/opscalehub/avacore/nlp/NormalizerTest.kt b/app/src/test/java/com/github/opscalehub/avacore/nlp/NormalizerTest.kt new file mode 100644 index 0000000..83ab088 --- /dev/null +++ b/app/src/test/java/com/github/opscalehub/avacore/nlp/NormalizerTest.kt @@ -0,0 +1,32 @@ +package com.github.opscalehub.avacore.nlp + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertTrue +import org.junit.Test + +class NormalizerTest { + + private val n = Normalizer() + + @Test fun folds_arabicLetters() { + // Arabic kaf/yeh -> Persian keheh/yeh + assertEquals("کتاب یک", n.normalize("كتاب يك")) + } + + @Test fun strips_kashidaAndTanvin() { + assertEquals("به", n.normalize("بـــه")) + assertEquals("واقعا", n.normalize("واقعاً")) + } + + @Test fun normalizes_punctuationSpacing() { + assertEquals("سلام، خوبی؟", n.normalize("سلام،خوبی?")) + } + + @Test fun collapses_whitespace() { + assertEquals("متن دو", n.normalize("متن دو")) + } + + @Test fun preserves_zwnj() { + assertTrue(n.normalize("می‌روم").contains('‌')) + } +} diff --git a/app/src/test/java/com/github/opscalehub/avacore/nlp/NumberToWordsTest.kt b/app/src/test/java/com/github/opscalehub/avacore/nlp/NumberToWordsTest.kt new file mode 100644 index 0000000..a50000d --- /dev/null +++ b/app/src/test/java/com/github/opscalehub/avacore/nlp/NumberToWordsTest.kt @@ -0,0 +1,72 @@ +package com.github.opscalehub.avacore.nlp + +import org.junit.Assert.assertEquals +import org.junit.Test + +class NumberToWordsTest { + + @Test fun cardinals_basic() { + assertEquals("صفر", NumberToWords.toCardinal("0")) + assertEquals("یک", NumberToWords.toCardinal("1")) + assertEquals("ده", NumberToWords.toCardinal("10")) + assertEquals("چهارده", NumberToWords.toCardinal("14")) + assertEquals("بیست و یک", NumberToWords.toCardinal("21")) + assertEquals("صد", NumberToWords.toCardinal("100")) + assertEquals("پنجاه", NumberToWords.toCardinal("50")) + assertEquals("چهارصد و سه", NumberToWords.toCardinal("403")) + } + + @Test fun cardinals_scales() { + // ۱۰۰۰ is "هزار", not "یک هزار" + assertEquals("هزار", NumberToWords.toCardinal("1000")) + assertEquals("دو هزار", NumberToWords.toCardinal("2000")) + assertEquals("هزار و چهارصد و سه", NumberToWords.toCardinal("1403")) + assertEquals("یک میلیون", NumberToWords.toCardinal("1000000")) + } + + @Test fun ordinals() { + assertEquals("اول", NumberToWords.toOrdinal("1")) + assertEquals("دوم", NumberToWords.toOrdinal("2")) + assertEquals("سوم", NumberToWords.toOrdinal("3")) + assertEquals("چهارم", NumberToWords.toOrdinal("4")) + assertEquals("بیستم", NumberToWords.toOrdinal("20")) + } + + @Test fun expand_inText_persianDigits() { + assertEquals( + "در سال هزار و چهارصد و سه", + NumberToWords.expand("در سال ۱۴۰۳") + ) + } + + @Test fun expand_percent() { + assertEquals("پنجاه درصد", NumberToWords.expand("۵۰٪")) + assertEquals("پنجاه درصد", NumberToWords.expand("50%")) + } + + @Test fun expand_decimal() { + assertEquals("سه ممیز چهارده", NumberToWords.expand("۳٫۱۴")) + assertEquals("یک ممیز پنج", NumberToWords.expand("1.5")) + } + + @Test fun expand_ordinalSuffix() { + assertEquals("رتبه دوم", NumberToWords.expand("رتبه ۲م")) + assertEquals("سوم", NumberToWords.expand("۳ام")) + } + + @Test fun expand_negative() { + assertEquals("منفی پنج", NumberToWords.expand("-۵")) + } + + @Test fun expand_thousandsSeparators() { + assertEquals("هزار و دویست و سی و چهار", NumberToWords.expand("1,234")) + } + + @Test fun expand_doesNotMisreadGluedSuffix() { + // "م" here begins میلیون; must not be treated as an ordinal suffix. + val out = NumberToWords.expand("۱۰۰میلیون") + // The number is expanded and میلیون is preserved (not turned into صدم). + org.junit.Assert.assertTrue(out.contains("صد")) + org.junit.Assert.assertTrue(out.contains("میلیون")) + } +} diff --git a/app/src/test/java/com/github/opscalehub/avacore/nlp/SentenceSegmenterTest.kt b/app/src/test/java/com/github/opscalehub/avacore/nlp/SentenceSegmenterTest.kt new file mode 100644 index 0000000..812236e --- /dev/null +++ b/app/src/test/java/com/github/opscalehub/avacore/nlp/SentenceSegmenterTest.kt @@ -0,0 +1,40 @@ +package com.github.opscalehub.avacore.nlp + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertTrue +import org.junit.Test + +class SentenceSegmenterTest { + + @Test fun splits_onSentenceBoundaries() { + val units = SentenceSegmenter.split("سلام. خوبی؟") + assertEquals(2, units.size) + assertEquals("سلام.", units[0].text) + assertEquals("خوبی؟", units[1].text) + assertTrue(units[0].trailingPauseMs > 0) + } + + @Test fun clausePauseShorterThanSentence() { + val clause = SentenceSegmenter.split("اول؛")[0] + val sentence = SentenceSegmenter.split("اول.")[0] + assertTrue(clause.trailingPauseMs < sentence.trailingPauseMs) + } + + @Test fun singleUnit_whenNoPunctuation() { + val units = SentenceSegmenter.split("یک دو سه") + assertEquals(1, units.size) + assertEquals("یک دو سه", units[0].text) + } + + @Test fun absorbsRepeatedTerminators() { + val units = SentenceSegmenter.split("واقعا؟!") + assertEquals(1, units.size) + assertEquals("واقعا؟!", units[0].text) + } + + @Test fun longRunIsCapped() { + val long = "کلمه ".repeat(80).trim() // ~ 400 chars, no punctuation + val units = SentenceSegmenter.split(long) + assertTrue("expected multiple capped units", units.size > 1) + } +}