Gate OCR escalation acceptance on the set-code lexicon

[WilfordGrimley]

Description

Changes the OCR escalation acceptance criterion from "any parse" to
"lexicon-valid parse, else keep escalating, else keep the best invalid
candidate" in both engines that run collector-line OCR escalation
(owner-ordered task, recorded disposition on issue #370).

1. image_evidence.py's collector-line escalation loop (the
compute_card_evidence tier-1→tier-3 attempt loop): a tier's parse now
only terminates escalation if its set code is None (the pre-M15
collector-number-only case, unaffected — same carve-out
calculate_join_key_verdict's own gate uses) or is a real
local_calculate_verdicts.known_set_codes() member. A live structural
finding (docs/reports/2026-07-23-ocr-preprocessing-probe-2.md) traced
94% of a lexicon-invalid-no-match sample to the old "any parse" criterion
accepting tier-1's own OCR noise before tiers 2–3 (built for exactly
this recovery) ever ran. If no tier ever yields a lexicon-valid parse,
the loop keeps the best invalid candidate — defined as the first
collector_number-bearing parse by tier order, which is provably
identical to what pre-gate code already stored for that bucket (old
code never distinguished valid from invalid, so its "first parse found"
is that same first invalid parse) — so stored fields and the #380
vote-layer gate are byte-identical for the no-valid case; only the path
there (genuine escalation) changed.

known_set_codes() is built once per run (one DB query) and threaded
through explicitly — run_image_evidence_cohort.py's handle() builds
it before forking the compute pool and passes it through
_run_cohort/_compute_one_card to every process-pool worker.

2. local_identify_printing_tags.py's live-pilot OCR engine
(run_ocr_for_card): the same criterion at its own "parsed-but-no-match"
outcome — the deferred follow-up local_calculate_verdicts.py's own
module docstring flagged ("known_set_codes() below is written so that
engine's own selection loop could reuse it directly in a focused
follow-up"), now applied and that flag note removed/updated. A
parsed-but-no-match outcome only keeps that label (which casts a real
is_no_match=True vote) when at least one tried variant's set code is
lexicon-valid or None; otherwise it demotes to a new, non-rescannable
unknown-set-code abstention — identical semantics to
calculate_join_key_verdict's own gate. Built once per run_pilot
invocation via a deferred import (a module-level import would be
circular — local_calculate_verdicts.py already imports FROM this
module) and threaded through _compute_card.

Interplay preserved: the #340 digit-free short-circuit (governs
whether escalation starts) is untouched and independent of this
criterion (governs acceptance during escalation) — a
collector_number-bearing parse always contains a digit, so it can
never trigger that short-circuit regardless of lexicon validity.
--no-shortcircuit is unaffected.

Compute delta: zero for the overwhelming majority (first parse
already lexicon-valid — identical behavior and compute). For the ~52k
invalid-parse cards (per issue #370's own DB-verified sizing): these
cards' tier-1 read is always digit-bearing (it parsed a collector
number), so the digit-free short-circuit never applies to them — they
now pay for genuine escalation through the remaining tiers instead of
stopping at tier 1. Live-measured cross-validation (both 2026-07-23 OCR
preprocessing probes, reusing the identical escalation harness): this
population's old cost was ~216ms/card at ~1.15 attempts; a comparable
near-full-escalation population measured ~909ms/card at ~7.06 attempts
— roughly a 4x per-card wall-clock increase (lower than the theoretical
8x ceiling since per-attempt cost isn't perfectly uniform across tiers).
Aggregate one-time delta if this population were ever re-scanned in
full: ~52,000 × ~0.7s ≈ 36,000 CPU-seconds (~10 CPU-hours,
single-threaded). No repair pass is planned or needed (see Open items
below) — this cost is only ever paid incrementally, once per card, at
first-extraction time for future cards that land in this bucket.

Checklist

• I have installed pre-commit and installed the hooks with pre-commit install before creating any commits.
• I have updated any related tests for code I modified or added new tests where appropriate.
• I have manually tested my changes as follows:
  • Full pytest cardpicker suite: 1834 passed, 4 skipped (named skips), 0 failed.
  • Targeted files individually green: test_image_evidence.py (78), test_local_identify_printing_tags.py (217), test_run_image_evidence_cohort.py (44), test_local_calculate_verdicts.py + test_golden_set.py + test_reparse_collector_evidence.py (165).
  • black/isort/ruff/mypy all clean on every changed file; docs_lint.py clean.
  • New tests cover: valid-first-parse short-circuits exactly as before (attempt-count assertions), invalid-then-valid escalates and accepts the valid parse, all-invalid keeps the best-candidate with today's exact stored shape, gate-disabled (known_set_codes=None) reproduces pre-2026-07-23 behavior byte-for-bite, --no-shortcircuit/digit-free-short-circuit interplay unaffected, and both engines' own live-pilot vote-casting/abstention wiring end-to-end.
• I have updated any relevant documentation or created new documentation where appropriate.
  • docs/identification-pipeline.md's Stage C bullet (one in-place paragraph, mirroring the existing Stage D g2 lexicon-gate bullet's style).
  • local_calculate_verdicts.py's module docstring — the "NOT APPLIED to local_identify_printing_tags" flag note updated to "APPLIED", describing the actual mechanism.
  • Every touched function/loop carries its own docstring/inline-comment explanation of the new criterion (matches this repo's existing density convention).

Open items

1. No repair pass is needed or planned for the existing ~52k-card
   invalid-parse population — this fix only changes what future
   compute_card_evidence/run_ocr_for_card calls compute; it never
   retroactively touches already-persisted ImageEvidence rows. The
   existing population's vote-casting behavior is already correct today
   via Gate join-key no-match votes on set-code lexicon, fix duplicate-suffix name matching #380's own Stage-D-time lexicon check (which reads whatever's
   currently stored, live, at verdict time) — a repair/re-extraction
   pass would only matter if the owner separately decides to chase the
   ~0.3% later-tier recovery rate the 2026-07-23 probe measured, which
   is a distinct, owner-gated decision outside this PR's scope.