tesserocr A/B spike: eliminate per-call tesseract spawn/model-load floor

[WilfordGrimley]

From the 2026-07-24 IO/throughput audit: pytesseract spawns a fresh tesseract process (PNG temp file + fork/exec + traineddata init) per OCR attempt — measured ~98ms fixed floor inside every ~195-205ms call, paid up to 8x/card under escalation. A persistent in-process binding (tesserocr PyTessBaseAPI per compute worker) is estimated at 20-25% of TOTAL per-card wall-clock. Prerequisites before any code change: (1) ARM64/aarch64 availability of libtesseract-dev/leptonica + tesserocr wheel in the django image; (2) an A/B parity spike proving byte-identical TSV/text output vs current run_tesseract* helpers on a real sample; (3) local_fallback/local_phash call sites (PROTECTED CORE importers) pass their existing tests unmodified — substrate swap under an unchanged public API only.

🤖 Generated with Claude Code

[WilfordGrimley]

Owner-prompted addition to this measurement round (2026-07-24): re-validate DEFAULT_FETCH_DPI. Stage C ships 250 as a safety margin above the empirically-best 200 (printing-tags.md addendum item 4, pre-escalation-criterion era). A/B the OCR funnel at 200 vs 250 under the current lexicon-gated escalation (#392) — if parse outcomes are identical, dropping to 200 is free fetch+OCR savings compounding with the tesserocr change. Pairs with this issue's existing crop-size/upscale-cost split measurement.

🤖 Generated with Claude Code

[WilfordGrimley]

A/B spike results (tesserocr vs pytesseract) — throwaway container, no prod/DB contact

Phase 1 — install feasibility: SUCCESS, but corrects this issue's own premise

tesserocr does ship a prebuilt aarch64 wheel now (tesserocr-2.10.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl) — no source build of tesserocr itself was needed. Only its pure-C dependency cysignals built from source (a few seconds, no issue). Full throwaway image build (Debian libtesseract-dev/libleptonica-dev/gcc/g++ + cython + tesserocr + pytesseract, from python:3.13-slim, matching docker/django/Dockerfile's own builder base) took ~94s total on this box, dominated by apt-get fetch/install, not compilation.

Real gotcha (not in the issue's own list of prerequisites): the wheel statically vendors its own tesseract/leptonica build, decoupled from the apt-installed tesseract this image's pytesseract shells out to:

• pytesseract path (apt tesseract-ocr 5.5.0-1+b1): tesseract 5.5.0, leptonica 1.84.1
• tesserocr 2.10.0 wheel: tesseract 5.5.1, leptonica 1.85.0

These are two different compiled OCR engines reading the same eng.traineddata, not the same substrate at two call sites. This is very likely the root cause of the divergences below, not just spawn-vs-persistent architecture.

Second gotcha: tesserocr.PyTessBaseAPI() / get_languages() called with no explicit path= hangs indefinitely (verified — had to docker kill it) because TESSDATA_PREFIX is unset and the wheel's own default search doesn't find Debian's /usr/share/tesseract-ocr/5/tessdata/ layout. Passing path= explicitly fixes it immediately. Any integration must set this explicitly — a silent hang, not a clean error, is what happens if it's missed.

Phase 2 — parity A/B

Deviation, stated up front: cardpicker/golden_set.py has no locally-stored fixture image files — GOLDEN_CARD_IDS are real production Card pks, and every one of its own recorded expectations was originally generated against a live prod DB row + a real Google Drive network fetch (see that module's own comments). Reading real production rows / fetching real Drive images was judged the same class of prod contact this spike's Phase 1 instruction explicitly ruled out, and I didn't have fresh confirmation to extend that to Phase 2, so I didn't do it. Substituted: synthetic collector-line-style crops built the same way test_local_ocr.py's own _text_crop fixture helper already builds fixtures for this exact module (PIL default bitmap font, black bg/white text) — content patterns drawn from real collector-line shapes documented in local_ocr.py's own regex comments, not invented. This means the sample's OCR difficulty and error modes may not represent real card photos — flagging this explicitly rather than presenting it as a full production-image validation.

Ran the exact production call path both ways: local_ocr.preprocess_variants → same two polarity variants → run_tesseract_text_and_words(config="--psm 6") for pytesseract; tesserocr.PyTessBaseAPI(path=..., psm=PSM.SINGLE_BLOCK) (PSM 6, default OEM 3 on both sides, same eng traineddata file) + GetTSVText (parsed with the same block/par/line text-reconstruction local_ocr._text_from_tesseract_data uses) for tesserocr.

• N = 20 (10 collector-line-shaped texts × 2 preprocess-variant polarities)
• Byte-identical raw text: 14/20 (70%)
• 6 divergences (verbatim), all attributable to the engine-version mismatch above, not config:
  • '239/280 R\nZNRe EN b> DAARKEN' → pytesseract: '239/280 R\nZNRe EN b> DAARKEN' / tesserocr: '239/280 R\nZNPRe EN b> DAARKEN'
  • '421/500 R\nDMR • EN' → pytesseract: '421/500 R\nDMREEN' / tesserocr: '421.500 R\nDMREEN'
  • '7\nAHR • EN' (×2 variants) → pytesseract: '7\nAHRGEN' / tesserocr: '?\nAHRGOEN'
  • '570\nMSH • EN' (×2 variants) → pytesseract: '370\nMSH a EN' / tesserocr: '570\nMSH GEN' (note: tesserocr got the leading digit right here where pytesseract read 3 for 5 — not a strictly worse engine, just a different one)
• Mean |confidence delta| on words present in both: 7.17 points
• Latency: pytesseract mean 120.6ms/call vs tesserocr mean 21.0ms/call across the real OCR sample (~5.7x)
• Direct spawn-floor measurement (trivial 1-char image, 20 reps each, isolates process-spawn/init overhead from recognition work): pytesseract mean 97.0ms/call vs tesserocr mean 0.2ms/call → measured floor delta 96.8ms, matching this issue's own ~98ms estimate almost exactly.

Recommendation: NEEDS-BIGGER-SAMPLE (blocking prerequisite identified, not just "more data")

The throughput case is confirmed and strong (~97ms/call floor eliminated, in-process binding ~500x faster than spawn on the trivial case). But raw-text parity is not byte-identical, and the divergence traces to a real, fixable substrate mismatch: tesserocr's wheel vendors tesseract 5.5.1/leptonica 1.85.0, this image's apt package is tesseract 5.5.0/leptonica 1.84.1. Before any GO:

1. Reconcile these versions (pin an apt tesseract-ocr version matching what a chosen tesserocr wheel vendors, or vice versa) so a re-run isolates architecture (spawn vs persistent) from engine-version drift.
2. Re-run parity against real golden-set images (cardpicker/golden_set.py's 30 real cards) — requires an explicit owner-approved live DB read + Google Drive fetch pass, out of scope for this spike.

If GO after that: integration shape

Swap behind local_ocr.py's existing run_tesseract/run_tesseract_text_and_words call signatures, gated by a settings flag (e.g. OCR_ENGINE=tesserocr, default pytesseract) with fallback to pytesseract on any tesserocr init failure — unchanged public API, matching this issue's own substrate-swap discipline. Two concrete requirements the spike surfaced: (a) always construct PyTessBaseAPI with an explicit path=/TESSDATA_PREFIX (never rely on auto-discovery — it hangs, doesn't error); (b) hold one persistent PyTessBaseAPI instance per compute worker process, not one per call — constructing it per call would pay its own (smaller but nonzero) init cost every time and forfeit most of the win.

Out of scope for this pass: the DEFAULT_FETCH_DPI 200-vs-250 re-validation from the owner-prompted addition above — not part of the instructions this spike executed against; flagging so it isn't assumed done.

No pipeline code was modified, no prod container was touched, no PR opened — this comment and a throwaway /tmp Dockerfile/harness (not committed) are the only artifacts.