Feat/streaming persian frontend#1
Merged
Merged
Conversation
Rework AvaTtsService for incremental, interruptible synthesis and add a real Persian NLP pipeline in front of the neural engine. Engine (AvaTtsService): - Stream audio via Sherpa generateWithCallback so first audio plays after the first chunk; honor onStop mid-utterance; reuse a single PCM scratch buffer; deliver <=8KB chunks for strict OEM audio paths. - Map system speechRate to the engine speed multiplier (pitch read, not yet applied: VITS has no pitch parameter). - Replace the busy-wait init with a CountDownLatch. - Versioned, atomic asset migration (ASSETS_VERSION + .assets_version marker) that repairs stale or partially-copied assets. Persian front-end (nlp/, previously dead code): - TextProcessor orchestrates Ssml -> NumberToWords -> Normalizer -> foldDigits -> PronunciationLexicon -> SentenceSegmenter. - NumberToWords: full Persian cardinals/ordinals/decimals/percent over Persian/Arabic/ASCII digits. - Normalizer: Arabic->Persian folding, ZWNJ, kashida/tanvin cleanup, punctuation spacing. - PronunciationLexicon + assets/tts/lexicon.txt: curated, boundary-safe pronunciation/ezafe overrides. - SentenceSegmenter: prosodic units with pauses; basic SSML (break/say-as). Build/compat fix: - Force class-based lambdas (-Xlambdas=class, -Xsam-conversions=class). Kotlin 2.0 invokedynamic lambdas lack the specialized invoke([F)Integer that Sherpa's native generateWithCallback looks up via JNI, which aborted the process on every device. TTS settings: - TtsDataCheckActivity advertises a single ISO-3 locale (fas-IRN) instead of duplicates that showed phantom languages in system TTS settings. Docs/tests: - README corrected to the real stack (Sherpa-ONNX + Piper VITS + eSpeak-NG); unbuilt items moved to a Future Roadmap section. - Unit tests for NumberToWords, Normalizer, SentenceSegmenter. Verified on device (OnePlus Nord): clean ~3.5s playback, no crash. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Verified on device: ~11s, 3 sentences with numbers/percent stream without crash at 22050 Hz. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
AliRezaTaleghani
added a commit
that referenced
this pull request
Jun 11, 2026
Feat/streaming persian frontend
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause: Kotlin 2.0 compiles lambdas to invokedynamic, producing a synthetic lambda that only exposes the erased invoke(Object). Sherpa's native streaming code (generateWithCallbackImpl) looks up the specialized invoke([F)Integer via JNI — not finding it, it aborts the entire process with SIGABRT. This would crash on every Android device, so catching it on your phone was exactly the right test.
Fix: forced class-based lambdas in app/build.gradle.kts:
freeCompilerArgs = freeCompilerArgs + listOf("-Xlambdas=class", "-Xsam-conversions=class")
Verified fixed on your device — clean ~3.5 s playback, no abort.
It came from TtsDataCheckActivity advertising three voice strings (fa, fa-IR, fas-IRN) for one voice — the system TTS settings turns each into a "language" row, and fa/fa-IR are malformed for that API (it wants ISO-3). I changed it to advertise exactly one correct locale: fas-IRN (Persian/Iran).
Good instinct — here's where things stand for broad compatibility:
┌────────────────────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────┐⚠️ user must allow background activity (documented; inherent to those OEMs) │
│ Concern │ Status │
├────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Kotlin-2.0 lambda crash │ ✅ Fixed globally (affected all devices) │
├────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Strict OEM audio buffers (Oppo/OnePlus/Xiaomi) │ ✅ ≤8 KB chunking + respects callback.maxBufferSize │
├────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Android 15 / 16 KB page sizes (new ARM) │ ✅ useLegacyPackaging=true │
├────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ arm64-v8a + armeabi-v7a │ ✅ both ABIs bundled (covers ~all phones) │
├────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ First-run asset integrity │ ✅ versioned + atomic copy (repairs partial extraction) │
├────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ System speech-rate honored │ ✅ maps to engine speed │
├────────────────────────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ Background-kill on Chinese OEMs │
└────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┘
One thing worth noting: I couldn't set the default engine via adb on your OnePlus (it blocks WRITE_SECURE_SETTINGS even over adb) — but that's only a test-automation limitation. Real users select the engine through the Settings UI, which works on every variant; it was already selected on yours.
Net: the crash that would have killed the app on all devices is fixed and verified by real audio playback, and the phantom-languages list is cleaned up. Want me to also wire request.pitch (currently ignored), or move on to committing these fixes on a branch?