feat: hybrid stroke-based + Claude Vision OCR pipeline

[mrangelmarino]

Problem

Today every handwriting page is rasterized to PNG and sent to Claude Vision (~$0.006/page, 4–5s, BYOK). For pages that are stroke-only with no image layers, this is leaving accuracy and cost on the table — reMarkable's own "Convert to Text" feature uses MyScript IINK, which:

• Accepts vector ink directly (no rasterization, no loss of stroke order or pen-lift information)
• Reports CER ~1–3% on Latin script (comparable or better than Claude on cursive)
• Costs ~$0.01/page at scale ($10/1K, 2K free/month)
• Returns in ~1–2s instead of 4–5s

The catch: MyScript is text-only. It cannot describe diagrams, transcribe math, or handle mixed pages. So the right shape is a hybrid pipeline, not a wholesale replacement.

Proposed Architecture

1. Parse .rm with rmscene (already done).
2. If only typed text → extract directly (already done).
3. NEW: If page has handwriting strokes AND no image layers AND no typed text fragments → convert rmscene strokes → MyScript JIIX format → call MyScript Cloud API. Return text + confidence.
4. If MyScript returns low confidence (< threshold to be tuned) OR the page has image layers OR has mixed content → fall back to Claude Vision (current path).

Implementation Surface

• src/rm_to_jiix.py (NEW) — convert rmscene stroke geometry to MyScript JIIX. Main work: mapping x/y/pressure/timing tuples to MyScript's strokeGroups schema.
• src/myscript_client.py (NEW) — MyScript Cloud API client with per-request HMAC signing. Accepts JIIX, returns plain text + confidence.
• src/handler.py — routing heuristic in process_page(): stroke-only + no image layers + no typed fragments → MyScript path; else Claude.
• src/secrets.py — second secret entry for MyScript application key + HMAC key.
• terraform/ — new Secrets Manager entry; IAM policy update.
• Tests: tests/fixtures/stroke_only.rm, tests/fixtures/mixed_content.rm with expected transcriptions; routing assertions.

Open Questions

• Confidence threshold for MyScript fallback to Claude — needs real notebook data once Speed optimizations: parallel page processing, reduced API calls, smaller PNGs #8 lands fixtures.
• Whether MyScript usage triggers any reMarkable ToS exposure beyond what we already have via rmapi-js. (Suspected: no, since MyScript is what reMarkable themselves use.)
• Pricing implications for a future MAS sub-tier — does MyScript usage enable a "no-Anthropic-key needed for stroke-only pages" tier?

Estimated Effort

1–2 weeks engineering. Out of scope for v1.1 launch.

Related

• Speed optimizations: parallel page processing, reduced API calls, smaller PNGs #8 — speed optimizations (must land first; blocks fixture creation)
• Strategic context: filed after the v1.1 plan-review pass; see Early access: reMarkable tablet sync prose#369 for the early-access launch scope