Feat/streaming persian frontend

README.md

@@ -1,51 +1,74 @@
 # AvaCore: Native Persian Text-to-Speech (TTS) Engine for Android
 
-AvaCore is a high-performance, on-device Persian (Farsi) Text-to-Speech engine designed to provide a natural and seamless voice experience for Android users. By integrating directly with the `android.speech.tts` framework, AvaCore enables all Android applications to speak Farsi with human-like prosody and high clarity, entirely offline.
+AvaCore is a high-performance, on-device Persian (Farsi) Text-to-Speech engine designed to provide a natural and seamless voice experience for Android users. By integrating directly with the `android.speech.tts` framework, AvaCore enables all Android applications to speak Farsi with human-like prosody and high clarity, **entirely offline** — no network access is required at build or run time.
 
 ## Project Vision
-To bridge the accessibility gap for Persian speakers on Android by delivering a state-of-the-art TTS engine that overcomes the unique linguistic challenges of the Farsi language, such as short vowel omissions and hidden *Ezafe* constructions.
-
-## Technical Roadmap
-
-### 1. NLP and Linguistic Analysis Pipeline
-The quality of synthesized speech depends heavily on text analysis. Farsi poses unique challenges with homographs and hidden vowels.
-*   **Preprocessing:** Utilizing `DadmaTools` for character normalization and noise removal.
-*   **Ezafe Detection:** Advanced detection of "Kasreh Ezafe" to ensure grammatical correctness in pronunciation.
-*   **Homograph Disambiguation:** Implementation of the `GE2PE` protocol to resolve phonetic ambiguities in written Farsi, significantly improving recognition accuracy.
-*   **Two-Step G2P Training:** A Grapheme-to-Phoneme model trained first on massive machine-generated data, followed by fine-tuning on high-precision manual transcriptions.
-
-### 2. Neural Synthesis Architecture
-A hybrid approach focusing on stability and performance for mobile environments.
-*   **Frontend (Tacotron):** Employs **Stepwise Monotonic Attention** to prevent word skipping or repeating in long sentences, ensuring robust synthesis for complex Farsi literature.
-*   **Backend/Vocoder (WaveRNN):** Generates high-quality 24kHz audio. WaveRNN reduces the storage footprint from standard large-scale models to a mobile-friendly 2.5MB - 70MB range.
-
-### 3. On-Device Optimization (Edge AI)
-Optimized for real-time performance on various mobile hardware tiers.
-*   **Quantization:** 8-bit mu-law quantization with a **0.86 Pre-emphasis filter** to maintain signal-to-noise ratio while reducing model size.
-*   **Hardware Acceleration:** Leveraging the **Android Neural Networks API (NNAPI)** for NPU/GPU execution.
-*   **Computation Efficiency:** Split-state GRU architecture and result caching for a 15-50% boost in computational efficiency, targeting at least **3x Real-Time Factor (RTF)**.
-
-### 4. Android System Integration
-Seamlessly integrated into the Android ecosystem to serve all installed applications.
-*   **System Engine:** Implemented via the standard `android.speech.tts.TextToSpeechService` API.
-*   **Streaming Logic:** Uses an **Inner/Outer stream loop** (5:1 ratio, approx. 100ms speech chunks) for ultra-low latency incremental audio delivery.
-*   **Standard Compliance:** Supports system intents like `ACTION_INSTALL_TTS_DATA` for seamless resource management and installation.
-
-### 5. Training Methodology & ManaTTS Dataset
-*   **Dataset:** Powered by the **ManaTTS** dataset, featuring 86 hours of high-quality 44.1kHz speech.
-*   **Cleaning:** Audio preprocessing using Spleeter to ensure noise-free training samples.
-*   **Forced Alignment:** Multi-model ASR voting and Interval/Gapped search for precise text-to-audio alignment.
-*   **Quality Control:** Strict Character Error Rate (CER) thresholds (HIGH < 0.05, MIDDLE < 0.20) for training data selection.
-
-## Performance Benchmarks (KPIs)
-
-| Metric | Target Value | Technical Note |
+To bridge the accessibility gap for Persian speakers on Android by delivering a state-of-the-art TTS engine that overcomes the unique linguistic challenges of the Farsi language, such as short-vowel omission and hidden *Ezafe* (کسرهٔ اضافه) constructions.
+
+---
+
+## Current Architecture (what ships today)
+
+AvaCore is built on a compact, proven, fully-offline stack:
+
+| Layer | Technology | Notes |
+| :--- | :--- | :--- |
+| **Inference engine** | Sherpa-ONNX 1.10.41 (`app/libs/sherpa-onnx.aar`) | JNI + Kotlin wrapper around ONNX Runtime |
+| **Acoustic + vocoder** | **Piper VITS** (`persian_model.onnx`) | End-to-end model — the HiFi-GAN-style decoder *is* the vocoder; 22.05 kHz |
+| **Grapheme-to-phoneme** | **eSpeak-NG** (`espeak-ng-data/`) | Persian phonemization |
+| **Text front-end** | AvaCore `nlp/` pipeline (Kotlin) | Normalization, number expansion, lexicon, segmentation, SSML |
+| **System integration** | `AvaTtsService : TextToSpeechService` | Serves every app on the device |
+
+> Note: VITS is a single end-to-end network. There is **no separate Tacotron front-end or WaveRNN vocoder** in the shipping engine — those belong to the future roadmap below.
+
+### Synthesis pipeline
+Text flows through `nlp/TextProcessor` before the neural model:
+
+1. **SSML** (`nlp/Ssml.kt`) — `<speak>`, `<break>`, `<say-as>` are honored; plain text passes through.
+2. **Number expansion** (`nlp/NumberToWords.kt`) — full Persian cardinals/ordinals/decimals/percent (`۱۴۰۳` → «هزار و چهارصد و سه»), Persian/Arabic/ASCII digits.
+3. **Normalization** (`nlp/Normalizer.kt`) — Arabic→Persian letter folding, ZWNJ (نیم‌فاصله) normalization, kashida/tanvin/diacritic cleanup, punctuation spacing.
+4. **Pronunciation lexicon** (`nlp/PronunciationLexicon.kt` + `assets/tts/lexicon.txt`) — high-precision overrides for short-vowel restoration and fixed *ezafe* compounds; curated and easily extensible.
+5. **Sentence segmentation** (`nlp/SentenceSegmenter.kt`) — splits text into short, prosodically-paused units.
+
+### Streaming + responsiveness
+- **Incremental streaming:** synthesis uses Sherpa's `generateWithCallback`, so the first audio plays after the first chunk — latency-to-first-audio stays roughly constant regardless of text length.
+- **Instant interruption:** `onStop()` aborts the current utterance mid-stream.
+- **System speech-rate:** the platform speech-rate is mapped to the engine speed multiplier.
+- **OEM-safe buffering:** audio is delivered in ≤ 8 KB chunks to satisfy strict OEM audio paths (e.g. Oppo/OnePlus).
+- **Robust asset migration:** bundled assets are extracted to `filesDir` once, versioned (`ASSETS_VERSION`) and copied atomically so a stale or partial copy is repaired automatically.
+
+See `ARCHITECTURE.md` for deployment details (16 KB page-size packaging, background-execution permissions on some OEMs).
+
+---
+
+## Future Roadmap (not yet implemented)
+
+These items are aspirational targets, not current behavior:
+
+### Linguistic depth
+- **Ezafe prediction & homograph disambiguation** — an ML model (GE2PE-style two-step G2P: large machine-generated pre-training + manual fine-tuning) to resolve مرد/مُرد-type ambiguities and predict ezafe in context, replacing the curated lexicon seed.
+- **Richer normalizer** — abbreviation/date/currency expansion, DadmaTools-style preprocessing.
+
+### Model & inference
+- **INT8 quantization** of the ONNX model (~4× smaller, faster CPU inference).
+- **Hardware acceleration** — trial the NNAPI provider for NPU/GPU execution.
+- **SOTA model evaluation** — Matcha-TTS (flow-matching) and Kokoro are drop-in candidates via Sherpa's existing `OfflineTtsMatchaModelConfig` / `OfflineTtsKokoroModelConfig`.
+- **True pitch control** — a DSP/phase-vocoder stage (VITS exposes no native pitch parameter).
+- **SSML expansion** — prosody/emphasis/phoneme tags.
+
+### Training methodology (reference)
+- **Dataset:** the **ManaTTS** corpus (≈86 h, 44.1 kHz), cleaned with Spleeter.
+- **Forced alignment:** multi-model ASR voting; strict CER thresholds (HIGH < 0.05, MIDDLE < 0.20) for data selection.
+
+## Aspirational KPIs
+
+| Metric | Target | Note |
 | :--- | :--- | :--- |
-| **Real-Time Factor (RTF)** | > 3.0x | Speed on mid-range CPU/NPU |
-| **Perceived Latency (CPL)** | < 180ms | Time until first audio buffer playback |
-| **RAM Usage** | 10 - 20 MB | Active memory footprint during synthesis |
-| **Storage (Disk)** | < 80 MB | Total model weights (Tacotron + WaveRNN) |
-| **Mean Opinion Score (MOS)** | > 4.0 | Subjective naturalness vs. human speech |
+| **Real-Time Factor** | > 3.0× | mid-range CPU/NPU |
+| **Latency to first audio** | < 180 ms | enabled by streaming synthesis |
+| **RAM usage** | 10–20 MB active | depends on quantization |
+| **Storage** | < 80 MB | model weights |
+| **Mean Opinion Score** | > 4.0 | subjective naturalness |
 
 ---
-AvaCore aims to set a new standard for Persian accessibility on Android, providing a robust, offline, and high-quality voice for navigators, screen readers, and virtual assistants.
+AvaCore aims to set a new standard for Persian accessibility on Android: a robust, offline, high-quality voice for navigators, screen readers and virtual assistants.

app/build.gradle.kts

@@ -40,6 +40,14 @@ android {
     @Suppress("UnstableApiUsage")
     kotlinOptions {
         jvmTarget = "17"
+        // Kotlin 2.0 compiles lambdas to invokedynamic by default, producing a
+        // synthetic lambda that only exposes the erased invoke(Object). Sherpa-ONNX's
+        // native generateWithCallback looks up the specialized invoke([F)Integer via
+        // JNI, so we force class-based lambdas/SAM conversions to keep that method.
+        freeCompilerArgs = freeCompilerArgs + listOf(
+            "-Xlambdas=class",
+            "-Xsam-conversions=class"
+        )
     }
     buildFeatures {
         viewBinding = true

app/src/main/assets/tts/lexicon.txt

@@ -0,0 +1,36 @@
+# AvaCore pronunciation lexicon
+# ------------------------------------------------------------------------------
+# Format (UTF-8):   surface = replacement
+#   - One entry per line. Lines starting with '#' are comments.
+#   - "surface" is matched as a whole token/phrase (it will NOT match inside a
+#     larger word). Longer surfaces are tried before shorter ones.
+#   - "replacement" is what is actually fed to the G2P engine. Use explicit
+#     Persian short-vowel diacritics — eSpeak-NG honors them:
+#         zebar/fatha  ◌َ (U+064E)   e.g. مَرد
+#         zir/kasra    ◌ِ (U+0650)   e.g. کتابِ  (also used to force EZAFE)
+#         pish/damma   ◌ُ (U+064F)   e.g. گُل
+#
+# PHILOSOPHY: keep entries HIGH-PRECISION. Only add a surface form that has a
+# single correct pronunciation regardless of context (fixed expressions, lexical
+# ezafe compounds, ligatures). Do NOT add context-dependent homographs here
+# (e.g. مرد = mard/mord) — those need the future ML disambiguator, and a wrong
+# guess is worse than none.
+#
+# This file is intentionally small and curated. Grow it with verified entries.
+# ------------------------------------------------------------------------------
+
+# --- Ligatures / presentation forms ---
+﷼ = ریال
+ﷺ = صلی الله علیه و آله و سلم
+ﷲ = الله
+
+# --- Fixed-ezafe compounds (ezafe is lexical here, always pronounced) ---
+مورد نظر = موردِ نظر
+نقطه نظر = نقطهٔ نظر
+وزارت کشور = وزارتِ کشور
+وزارت بهداشت = وزارتِ بهداشت
+حقوق بشر = حقوقِ بشر
+
+# --- Common symbols read as words ---
+& = و
+@ = اَت

app/src/main/java/com/github/opscalehub/avacore/MainActivity.kt

@@ -61,7 +61,10 @@ class MainActivity : AppCompatActivity() {
     }
 
     private fun speakSample() {
-        val text = "این یک آزمایش از موتور بازگو کننده آوا است."
+        // Sample exercises number expansion (۱۴۰۴, ۳۵, ۲۰٪), punctuation pauses
+        // and multi-sentence streaming so the demo showcases the full pipeline.
+        val text = "سلام! این موتور بازگوکننده آوا است. " +
+            "امروز ۱۲ خرداد ۱۴۰۴ است، دمای هوا ۳۵ درجه و رطوبت ۲۰٪ می‌باشد."
         val result = tts?.speak(text, TextToSpeech.QUEUE_FLUSH, null, "sample_id")
         if (result == TextToSpeech.ERROR) {
             Toast.makeText(this, "Speech failed. Ensure AvaCore is selected.", Toast.LENGTH_SHORT).show()

app/src/main/java/com/github/opscalehub/avacore/TtsDataCheckActivity.kt

@@ -23,13 +23,13 @@ class TtsDataCheckActivity : Activity() {
         when (action) {
             TextToSpeech.Engine.ACTION_CHECK_TTS_DATA -> {
                 // Return supported locales in different formats to ensure compatibility
-                val availableVoices = arrayListOf("fa", "fa-IR", "fas-IRN")
+                // A single, correctly-formed ISO-3 locale (lang-country): Persian/Iran.
+                // Listing 2-letter duplicates here makes the system TTS settings show
+                // several phantom "languages" for one voice, so we expose exactly one.
+                val availableVoices = arrayListOf("fas-IRN")
                 resultIntent.putStringArrayListExtra(TextToSpeech.Engine.EXTRA_AVAILABLE_VOICES, availableVoices)
                 resultIntent.putStringArrayListExtra(TextToSpeech.Engine.EXTRA_UNAVAILABLE_VOICES, arrayListOf())
-
-                // Some systems also look for these
-                resultIntent.putStringArrayListExtra("availableVoices", availableVoices)
-
+
                 setResult(TextToSpeech.Engine.CHECK_VOICE_DATA_PASS, resultIntent)
             }
             "android.speech.tts.engine.GET_SAMPLE_TEXT" -> {

app/src/main/java/com/github/opscalehub/avacore/nlp/Normalizer.kt

@@ -1,30 +1,91 @@
 package com.github.opscalehub.avacore.nlp
 
-import java.util.regex.Pattern
-
 /**
- * Handles text normalization for Persian language.
- * Tasks: Number expansion, abbreviation expansion, character normalization.
+ * Character-level normalization for Persian text.
+ *
+ * The goal is to hand eSpeak-NG a clean, canonical Persian string:
+ *  - fold Arabic-shaped letters to their Persian counterparts
+ *  - normalize the ZWNJ (نیم‌فاصله) and stray control characters
+ *  - strip kashida (ـ) and decorative combining marks that confuse G2P
+ *  - canonicalize punctuation spacing and collapse whitespace
+ *
+ * Digit handling is intentionally NOT done here — numbers are expanded to words
+ * by [NumberToWords] first, and any leftover digits are folded afterwards. Keeping
+ * digits intact at this stage lets the number expander detect them reliably.
+ *
+ * Pure Kotlin, no dependencies, so it is unit-testable on the JVM.
  */
 class Normalizer {
 
     fun normalize(text: String): String {
-        var normalized = text
-        normalized = normalizeCharacters(normalized)
-        normalized = expandNumbers(normalized)
-        return normalized
+        var s = text
+        s = foldCharacters(s)
+        s = normalizeZwnj(s)
+        s = stripDecorations(s)
+        s = normalizePunctuation(s)
+        s = collapseWhitespace(s)
+        return s.trim()
+    }
+
+    /** Map Arabic-shaped letters and presentation forms to canonical Persian. */
+    private fun foldCharacters(text: String): String {
+        val sb = StringBuilder(text.length)
+        for (c in text) {
+            sb.append(
+                when (c) {
+                    'ك' -> 'ک' // Arabic Kaf -> Persian Keheh
+                    'ي' -> 'ی' // Arabic Yeh -> Persian Yeh
+                    'ى' -> 'ی' // Alef Maksura -> Persian Yeh
+                    'ة' -> 'ه' // Teh Marbuta -> Heh
+                    // Note: ۀ (heh with hamza) is left untouched — it carries the
+                    // ezafe vowel and eSpeak-NG reads it; folding to ه would lose it.
+                    else -> c
+                }
+            )
+        }
+        return sb.toString()
+    }
+
+    /**
+     * Normalize the zero-width non-joiner. Different inputs use ZWSP/ZWJ or plain
+     * spaces around نیم‌فاصله; collapse those variants to a single ZWNJ (U+200C)
+     * and drop other zero-width controls eSpeak does not understand.
+     */
+    private fun normalizeZwnj(text: String): String {
+        return text
+            .replace('‏', ' ')   // RIGHT-TO-LEFT MARK -> space
+            .replace('‎', ' ')   // LEFT-TO-RIGHT MARK -> space
+            .replace("‍", "")    // ZERO WIDTH JOINER -> drop
+            .replace("", "")    // BOM / ZWNBSP -> drop
+            .replace(" ", " ")   // NO-BREAK SPACE -> space
+            // collapse spaces that surround an existing ZWNJ
+            .replace(Regex("\\s*‌\\s*"), "‌")
     }
 
-    private fun normalizeCharacters(text: String): String {
-        // Replace Arabic Keheh and Yeh with Persian counterparts
-        return text.replace("\u0643", "\u06a9") // Arabic Kaf -> Persian Keheh
-            .replace("\u064a", "\u06cc") // Arabic Ya -> Persian Yeh
-            .replace("\u0649", "\u06cc") // Arabic Alef Maksura -> Persian Yeh
+    /** Remove kashida and standalone combining diacritics that hurt G2P. */
+    private fun stripDecorations(text: String): String {
+        return text
+            .replace("ـ", "")            // ـ Tatweel/Kashida
+            .replace("ٰ", "")            // superscript alef
+            // Tanvin marks are not pronounced reliably by eSpeak; drop them.
+            .replace(Regex("[ًٌٍ]"), "")
+    }
+
+    /** Canonicalize punctuation: fold Arabic forms and tidy spacing. */
+    private fun normalizePunctuation(text: String): String {
+        return text
+            .replace(',', '،')        // ASCII comma -> Persian comma (better pause)
+            .replace(';', '؛')        // ASCII semicolon -> Persian semicolon
+            .replace('?', '؟')        // ASCII question -> Persian question
+            // no space before, one space after Persian punctuation
+            .replace(Regex("\\s*([،؛؟!:])"), "$1")
+            .replace(Regex("([،؛؟!:])(?=\\S)"), "$1 ")
     }
 
-    private fun expandNumbers(text: String): String {
-        // This is a placeholder for a complex number-to-words converter
-        // In a real implementation, you'd use a rule-based system or a dictionary
+    private fun collapseWhitespace(text: String): String {
         return text
+            .replace(Regex("[\\t\\x0B\\f\\r]"), " ")
+            .replace(Regex(" {2,}"), " ")
+            .replace(Regex("\\n{3,}"), "\n\n")
     }
 }