Merge docs-on-device-llm: record the on-device LLM/Gemma decision

Claude-Session: https://claude.ai/code/session_01VSz5xqHNZDMi4ewUJwStnR

Author: foodlbs

README.md

@@ -65,6 +65,9 @@ text (e.g. `SKIP INTRO`) is kept only as a case-insensitive fallback.
 **SkipperKit therefore uses no `MediaProjection`, no screenshots, no OCR, no image
 recognition, no ML vision, no root, and no ADB automation.**
 
+> No on-device LLM either — see [docs/on-device-llm.md](docs/on-device-llm.md) for
+> why Gemma and similar models are the wrong fit for a background button-tapper.
+
 ---
 
 ## Supported apps

docs/on-device-llm.md

@@ -0,0 +1,77 @@
+# On-device LLM / ML — design decision
+
+**Status:** Decided — SkipperKit stays heuristic-only. Reaffirmed 2026-07 after a
+Gemma reassessment.
+
+SkipperKit matches streaming-app controls (Skip Intro / Skip Recap / Next Episode)
+by reading the **accessibility node tree** — text, content-description, view-id,
+bounds, and clickability that the OS already publishes. It uses **no** screen
+capture, OCR, image recognition, or ML model. This doc records why an on-device
+LLM (specifically the Gemma family) is not used, so the question isn't relitigated
+without new evidence.
+
+## Where an LLM could even fit
+
+Never the hot path. `onAccessibilityEvent` fires continuously during playback;
+running inference there would drain battery and risk a mis-tap. The only plausible
+place is the **discovery tier** — an off-hot-path, throttled, propose-only step that
+classifies an unrecognized button (skip-intro / skip-recap / next-episode end-card /
+control-bar / dismiss / none) and offers it for the user to approve. That tier never
+taps on its own.
+
+## Why not Gemma (or any general on-device LLM)
+
+Reassessed 2026-07 against current Gemma releases. Three independent reasons, plus a
+structural blocker:
+
+### 1. Size and latency are grossly disproportionate
+
+| Model | Disk (int4, LiteRT) | Peak RAM | Note |
+|---|---|---|---|
+| Gemma 3n E2B | 3.0–3.7 GB | 2.7–3.4 GB | "effective 2B" hides ~5B real params; never GA on LiteRT |
+| Gemma 3n E4B | 4.3–4.9 GB | ~4–6 GB | needs 12–16 GB device RAM to be stable |
+| Gemma 3 1B | ~530 MB | ~1 GB (4 GB floor) | smallest "real" LLM |
+| Gemma / FunctionGemma 270M | ~125–250 MB | ~256 MB | smallest; non-deterministic |
+| Gemma 4 E2B (2026-04) | 2.0–2.6 GB | ~1.7 GB | Apache-2.0/ungated, still oversized |
+
+For comparison, the task-appropriate alternative (below) is **~28 MB / ~53 ms**, or a
+word-embedding classifier **<1 MB / ~0.03 ms**. Even the smallest Gemma is ~25× larger
+and orders of magnitude slower, in a long-running service that must coexist with a
+foreground streaming app on real devices.
+
+Runtime is LiteRT-LM (MediaPipe `tasks-genai` is deprecated). Models are too large to
+bundle in an APK, so they'd be a multi-GB **runtime download** behind HuggingFace
+license gating, with engine init up to ~10 s and 30–120 s NPU JIT on first run.
+
+### 2. It's the wrong tool for a button-tapper
+
+The task is short-text classification into a small fixed label set. An LLM is
+generative and **non-deterministic** — a hallucinated positive means a mis-tap.
+A deterministic classifier is smaller, faster, and safer for something that
+performs `ACTION_CLICK` on the user's behalf.
+
+### 3. The "official" on-device path is structurally blocked
+
+ML Kit GenAI / AICore (Gemini Nano) enforces `BACKGROUND_USE_BLOCKED`: on-device
+inference runs only while the calling app is the **top foreground** app (a Private
+Compute Core constraint). An accessibility service is never the foreground app, so
+this path is structurally incompatible with SkipperKit — independent of the
+allowlist gating that already blocks sideloaded apps.
+
+## If on-device ML is ever justified
+
+It isn't today. If field data ever shows the heuristic misses a meaningful fraction
+of real controls, use a **fine-tuned TFLite text classifier bundled in the APK**
+(e.g. MobileBERT NLClassifier, ~28 MB, deterministic, sub-53 ms) — not an LLM — and
+keep it behind the existing discovery flow (propose → user approves → never
+auto-tap) so a wrong prediction can never cause a tap.
+
+Only revisit a general LLM if all of these hold: (a) field data shows heuristic +
+tiny-TFLite still miss a significant fraction of nodes, (b) a zero-shot multi-locale
+need can't be met by retraining the classifier, and (c) output is
+confidence-thresholded behind the human-approval gate.
+
+## Cheaper win first
+
+Extend the heuristic with locale-aware label patterns as new services arrive through
+the contribution pipeline. No model, no download, fully deterministic.