Skip to content

feat(android): add privacy-aware TTS foundation#1494

Open
agent-p1p wants to merge 3 commits into
masterfrom
pip/whitenoise-android-1479
Open

feat(android): add privacy-aware TTS foundation#1494
agent-p1p wants to merge 3 commits into
masterfrom
pip/whitenoise-android-1479

Conversation

@agent-p1p

@agent-p1p agent-p1p commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Fixes #1479

Summary

  • discover installed Android TTS engines through public APIs and prefer the OS default only when it is in the local-engine allowlist
  • classify GrapheneOS Speech Services, RHVoice, and eSpeak as local; treat every other engine, including Google TTS, as unknown and persist warning acknowledgement per engine package
  • prefer offline voices within the resolved engine and handle empty, failed, cancelled, missing, or stale engine resolutions without crashing
  • arbitrate audio focus centrally so TTS and voice-note playback cannot overlap
  • add Text-to-Speech settings, engine selection, privacy warning UI, package visibility, and localized copy

Verification

  • ./gradlew :app:compileDevZapstoreDebugKotlin :app:compileDevPlayDebugKotlin --no-daemon --stacktrace
  • ./gradlew :app:testDevZapstoreDebugUnitTest :app:testDevPlayDebugUnitTest --no-daemon --stacktrace
  • ./gradlew :app:verifyRoborazziDevZapstoreDebug :app:verifyRoborazziDevPlayDebug --no-daemon --stacktrace
  • ./gradlew :app:ktlintCheck --no-daemon --stacktrace
  • ./gradlew :app:lintDevZapstoreDebug :app:lintDevPlayDebug --no-daemon --stacktrace
  • repeated the complete gate after rebasing onto current origin/master; all commands passed
  • semantic rebase sweep: clean parent, no conflict markers, no duplicate declarations, localized key counts consistent

Sensitive paths

Privacy-sensitive plaintext-routing and trust-decision paths:

  • app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineResolver.kt
  • app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsTrustWarning.kt
  • app/src/main/java/dev/ipf/whitenoise/android/state/TtsWarningPreferences.kt

No crypto, MLS, CGKA, key-handling, authentication, or protocol-state code changed.

Follow-up scope

The actual Speak action and final message-action UI remain in #1480 and #1481 as planned.


Open in Stage

Summary by CodeRabbit

  • New Features

    • Added Text-to-Speech settings with engine discovery and selection.
    • Added support for choosing a preferred speech engine and remembering that choice.
    • Added safety warnings for unverified or network-capable engines before reading content aloud.
    • Improved audio handoff between voice playback and speech synthesis.
    • Added localized TTS text across supported languages.
  • Bug Fixes

    • Improved audio-focus handling and cleanup during playback transitions.
    • Added safeguards for unavailable, failed, or cancelled speech-engine initialization.

Resolve and classify installed TTS engines, persist per-engine trust acknowledgements, and arbitrate audio focus with voice-note playback.

Fixes #1479
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@agent-p1p, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 25 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1105f2fb-38b2-4d4c-9f24-0335a9a6c443

📥 Commits

Reviewing files that changed from the base of the PR and between ca948a9 and d1bb943.

📒 Files selected for processing (3)
  • app/src/main/java/dev/ipf/whitenoise/android/audio/AudioFocusOwner.kt
  • app/src/main/res/values-de/strings.xml
  • app/src/test/java/dev/ipf/whitenoise/android/audio/AudioFocusOwnerHandoffTest.kt

Walkthrough

Adds TTS engine discovery and selection, trust warnings with per-engine acknowledgement, shared audio-focus handoff between voice and TTS playback, persisted preferences, settings UI, localization, and extensive Robolectric coverage.

Changes

TTS foundation

Layer / File(s) Summary
Shared audio-focus arbitration
app/src/main/java/dev/ipf/whitenoise/android/audio/*, app/src/test/java/dev/ipf/whitenoise/android/audio/*
AudioFocusOwner coordinates voice and TTS focus ownership, handoff callbacks, stale listeners, release behavior, and concurrency handling; VoicePlaybackController uses the shared owner.
Engine discovery and resolution
app/src/main/AndroidManifest.xml, app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineCatalog.kt, .../TtsEngineResolver.kt, app/src/test/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineResolverTest.kt
The manifest exposes TTS service discovery, while resolver models and tests cover engine classification, initialization, preferred selection, cancellation, timeouts, trust verification, and offline voices.
Selection state and lifecycle management
app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineSelection.kt, TtsResolutionDispatch.kt, app/src/main/java/dev/ipf/whitenoise/android/state/*, app/src/test/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineSelectionTest.kt, app/src/test/java/dev/ipf/whitenoise/android/state/*
App state resolves and adopts engines, releases obsolete handles, refreshes availability at startup, and persists engine overrides and warning acknowledgements.
Trust warning and settings surface
app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsTrustWarning.kt, app/src/main/java/dev/ipf/whitenoise/android/ui/settings/*, app/src/main/java/dev/ipf/whitenoise/android/ui/navigation/AppDestinations.kt, app/src/main/res/values*/strings.xml, app/src/test/java/dev/ipf/whitenoise/android/audio/tts/TtsTrustWarningTest.kt
The settings navigation and Compose screen display engine status, selection options, trust labels, and first-use warnings, with localized resources and warning-condition tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related issues

  • #1478: The changes implement the broader TTS engine resolution, trust warning, and shared audio-focus objectives described by this issue.
  • #1484: The shared audio-focus changes relate to extending TTS playback coordination toward background or foreground-service playback.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.87% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main change: a privacy-aware TTS foundation.
Linked Issues check ✅ Passed The PR covers the TTS resolver, trust warnings, shared audio focus, settings UI, strings, and tests aligned with #1479's foundation scope.
Out of Scope Changes check ✅ Passed All changes stay within the TTS foundation, localization, and test coverage; no unrelated modifications stand out.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch pip/whitenoise-android-1479

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@stage-review

stage-review Bot commented Jul 16, 2026

Copy link
Copy Markdown

Ready to review this PR? Stage has broken it down into 5 individual chapters for you:

Title
1 Centralize audio focus arbitration
2 Implement privacy-aware TTS engine resolution
3 Persist TTS preferences and trust warnings
4 Wire TTS state into AppState
5 Add TTS settings and localized strings
Open in Stage

Chapters generated by Stage for commit d1bb943 on Jul 16, 2026 7:09pm UTC.

@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

Preview APK for PR #1494

Home-screen label: 1494 White Noise with the blueprint launcher icon — installs side-by-side with the production and staging apps.

Install directly — one tap, no GitHub sign-in (public, content-addressed .apk). ARM64 only.

Every push to this PR rebuilds and updates this comment.

@agent-p1p

Copy link
Copy Markdown
Contributor Author

Adversarial review — changes requested

Reviewed exact head e4b552fe939713a45899e6e23b2018b2599a91c9 against origin/master.

Blocking

  1. The handle can be labeled Local even when Android silently connected it to an Unknown fallback engine.

    TtsEngineResolver.kt:211-212,230-240 assumes requestedEnginePackage is the engine behind a successful TextToSpeech instance. The public three-argument constructor used at :267 sets useFallback=true; AOSP's initTts() tries the requested package, then the OS default, then the highest-ranked engine. The public API explicitly does not expose which one ultimately connected (getCurrentEngine() is hidden). An installed local engine can fail to bind while Google/another unknown engine succeeds; this code still returns the requested package with EngineTrust.Local. That suppresses the trust warning before later code sends decrypted text to the fallback engine.

    The membership check in engines does not prove the package actually bound. Please make this fail closed: do not attach Local trust to a handle unless the active engine identity is established by a design that cannot fall back. Add a regression test that models requested-local bind failure followed by unknown-engine success; the current factory ignores its engine argument and cannot expose this case.

  2. Selecting an engine is a cancellable screen side effect, so Back can persist one package while leaving the old package active.

    TextToSpeechScreen.kt:61-71 writes the preference first and relies on a composition-owned LaunchedEffect to switch the handle. Leaving the screen before that effect starts/completes cancels refreshTtsAvailability(). Since AppState.kt:3229-3270 preserves the previous handle when no replacement is published, the stored selection and resolvedTtsEnginePackage() then name the new engine while ttsResolution.handle still targets the old one. The inconsistency lasts until this screen is reopened or the process restarts.

    Move selection/switching into AppState-owned work and publish the selected package + handle/trust atomically (or expose an explicit switching state). Add a test for select-then-immediate-navigation-away.

  3. TTS availability is never initialized for users who enter through onboarding in the current process.

    The only non-Settings refresh is inside the existing-account branch of bootstrap (AppState.kt:2137-2139). createIdentity() (:2202-2210), importIdentity() (:2261-2271), and the Amber ready transition do not refresh it. Those users keep ttsResolution == null, so the planned hasUsableEngine gate hides Speak until they manually visit Text-to-Speech Settings or restart.

    Availability is device-level, not account-level. Initialize/refresh it independently of whether bootstrap found an account, and cover the onboarding-to-Ready paths.

Suggestions

  • Bound initialization and keep engine Binder work off the UI thread. resolve() has no timeout, while refreshTtsAvailability() runs from Dispatchers.Main.immediate; setAudioAttributes, voice, voices, and setVoice are synchronous calls into a separate engine app. A broken engine can wedge both refresh mutexes or freeze startup. Also keep a distinct unresolved/loading state so Settings does not render “No usable engine” before discovery finishes.
  • Check offline-voice activation. TtsEngineResolver.kt:244-249 discards setVoice()'s result and does not exclude KEY_FEATURE_NOT_INSTALLED. A failed candidate leaves the previous network-required voice active while resolution reports success. Try viable offline candidates in order and retain the current voice only as an explicit fallback.
  • Enforce the audio arbiter's threading model. AudioFocusOwner mutates process-wide request/owner/callback state without synchronization or main-thread assertions. Current voice calls are Main-confined, but the TTS controller will introduce progress/lifecycle callbacks. Either enforce Main confinement or serialize owner transitions; stale release/callback races must not abandon or stop a successor session.

Verification

  • :app:testDevZapstoreDebugUnitTest: 2,206 passed
  • :app:testDevPlayDebugUnitTest: 2,180 passed
  • :app:ktlintCheck: passed
  • Focused TTS/audio-focus tests: passed
  • git diff --check: passed
  • GitHub CI: all checks green

No out-of-scope follow-up issues filed; these findings are in #1479's engine-resolution/lifecycle scope.

Address adversarial review findings for #1479.
@agent-p1p

Copy link
Copy Markdown
Contributor Author

Addressed all three blocking adversarial findings and the three follow-up suggestions in signed commit ca948a98.

  • TTS trust now fails closed when Android cannot verify the connected engine; verified fallback-package mismatches are rejected and released.
  • Engine selection runs in AppState-owned work and persists only after coherent handle adoption. Dispatcher-return cancellation releases completed handles.
  • Device-level discovery starts independently of account bootstrap/onboarding.
  • Resolver init is bounded, offline voice activation tries viable installed candidates, and temporary instances are shut down on timeout/cancellation/late callbacks.
  • Audio-focus transitions are serialized; external callbacks run outside the state lock and block overlapping acquisitions until completion.

Verification: both compile variants, 4,434 unit tests, both Roborazzi variants, ktlint, Android lint, diff/locale/security sweeps, and an independent pre-commit review all pass.

@agent-p1p

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@agent-p1p
agent-p1p marked this pull request as ready for review July 16, 2026 18:34

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/src/main/java/dev/ipf/whitenoise/android/audio/AudioFocusOwner.kt`:
- Around line 54-61: Update acquire() around previousOwnerSurrender so a thrown
surrender callback does not leave the newly acquired owner and focus active
while propagating failure. Catch the callback failure and roll back the newly
acquired focus/ownership before returning or rethrowing, or enforce that
completeCallback() is non-throwing; preserve normal completion when surrender
succeeds.

In
`@app/src/main/java/dev/ipf/whitenoise/android/ui/settings/TextToSpeechScreen.kt`:
- Around line 122-127: Update the requiresTtsTrustWarning call in the engine
selection flow to pass the candidate engine’s static trust via engine.trust
instead of appState.runtimeTrustForTtsSelectionWarning(...). Preserve the
existing package name and warning preference arguments.

In `@app/src/main/res/values-de/strings.xml`:
- Around line 1076-1083: Standardize the German TTS terminology on
“Sprachausgabe” across the strings in this block: replace the “Engine” wording
in tts_settings_subtitle, tts_settings_explainer_usable, and
tts_settings_engine_local, and replace “Sprachmodul” in
tts_settings_engine_title. Preserve each string’s existing meaning while
ensuring the selected component is described consistently.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 738c6202-0322-4b3e-8739-eb194d65ac86

📥 Commits

Reviewing files that changed from the base of the PR and between 5c6654d and ca948a9.

📒 Files selected for processing (32)
  • app/src/main/AndroidManifest.xml
  • app/src/main/java/dev/ipf/whitenoise/android/audio/AudioFocusOwner.kt
  • app/src/main/java/dev/ipf/whitenoise/android/audio/VoicePlaybackController.kt
  • app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineCatalog.kt
  • app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineResolver.kt
  • app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineSelection.kt
  • app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsResolutionDispatch.kt
  • app/src/main/java/dev/ipf/whitenoise/android/audio/tts/TtsTrustWarning.kt
  • app/src/main/java/dev/ipf/whitenoise/android/state/AppState.kt
  • app/src/main/java/dev/ipf/whitenoise/android/state/TtsEnginePreferences.kt
  • app/src/main/java/dev/ipf/whitenoise/android/state/TtsWarningPreferences.kt
  • app/src/main/java/dev/ipf/whitenoise/android/ui/navigation/AppDestinations.kt
  • app/src/main/java/dev/ipf/whitenoise/android/ui/settings/SettingsScreen.kt
  • app/src/main/java/dev/ipf/whitenoise/android/ui/settings/TextToSpeechScreen.kt
  • app/src/main/res/values-b+zh+Hant/strings.xml
  • app/src/main/res/values-de/strings.xml
  • app/src/main/res/values-es/strings.xml
  • app/src/main/res/values-fr/strings.xml
  • app/src/main/res/values-it/strings.xml
  • app/src/main/res/values-pt/strings.xml
  • app/src/main/res/values-ru/strings.xml
  • app/src/main/res/values-tr/strings.xml
  • app/src/main/res/values-zh/strings.xml
  • app/src/main/res/values/strings.xml
  • app/src/test/java/dev/ipf/whitenoise/android/audio/AudioFocusOwnerHandoffTest.kt
  • app/src/test/java/dev/ipf/whitenoise/android/audio/VoicePlaybackControllerFocusTest.kt
  • app/src/test/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineResolverTest.kt
  • app/src/test/java/dev/ipf/whitenoise/android/audio/tts/TtsEngineSelectionTest.kt
  • app/src/test/java/dev/ipf/whitenoise/android/audio/tts/TtsTrustWarningTest.kt
  • app/src/test/java/dev/ipf/whitenoise/android/state/TtsEnginePreferencesTest.kt
  • app/src/test/java/dev/ipf/whitenoise/android/state/TtsStartupDiscoveryCoverageTest.kt
  • app/src/test/java/dev/ipf/whitenoise/android/state/TtsWarningPreferencesTest.kt

Comment thread app/src/main/res/values-de/strings.xml Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TTS: engine resolution, trust warning, and audio-focus ownership

1 participant