perf(image-evidence): collapse Stage C OCR attempt ladder from 3 tiers to 2 (issue #677) (re-land of #698) - #702
Merged
Conversation
…s to 2 (issue #677) (#698) * perf(image-evidence): collapse Stage C OCR attempt ladder from 3 tiers to 2 (issue #677) _collector_line_ocr_attempts drops tier 3 (a PSM-11 re-try of tier 1's own variants) - measured, not guessed. Two fresh real-production probes (450 forced-escalation cards combined) walked the old 3-tier ladder to full completion and recorded, per card, the first tier at which a candidate-validated genuine match appeared: tier 2 produced 2 genuine matches across the sample, tier 3 produced zero - only more lexicon-valid -but-uncorroborated noise, consistent with the 2026-07-23 preprocessing probes' own ~99% hopeless-art-noise finding for this population. Cross-checked against all 30 golden_set.GOLDEN_CARD_IDS cards: none ever resolved uniquely at tier 3, so this collapse changes zero golden-set expectations. Worst case drops from 8 tesseract calls to 6 (down from #259-#677's own 4x pre-#259 cost to 3x). collector_line_ocr/collector_line_tsv/artist_ocr extractor versions bumped v2->v3 (all three read through the same escalation loop, per EXTRACTOR_OWNERSHIP's own 'bump every listed key together' convention); legal_line unchanged. Investigated and explicitly did NOT ship the companion 'unify the collector-line/legal-line OCR reads into one full-width pass' idea named in this issue's own brief: a real-image accuracy probe found it regresses 46/150 (30.7%) of currently-correct collector-number reads (mostly proxy/ watermark text getting picked up by the set-code regex in place of the real code, or the wider crop causing tesseract to drop the collector- number line outright), with zero improvements. Reported as a legitimate negative finding rather than shipped anyway. Lifts the EXTRACTOR_OWNERSHIP EXCLUDED_HELPERS entry for the OCR ladder (check_extractor_ownership_totality.py) that PR #690 left in place while this branch was in flight - _collector_line_ocr_attempts and preprocess_fallback_variants now carry real ownership entries. Verification: full backend suite (3598 passed, 8 skipped, 0 failed), ownership-totality + manifest-sync scripts clean with their own unit tests green (40/40), docs_lint --strict clean, pre-commit clean on all tracked files. * fix(image-evidence): bump artist_ocr to v4 to avoid colliding with #685's v3 ARTIST_OCR_EXTRACTOR_VERSION was set to "artist-ocr-v3" on this branch for issue #677's ladder-collapse dedup, but PR #685 (merged to master separately) already claims "artist-ocr-v3" for an unrelated change (wiring modern_artist_credit's crop fallback into live extraction). Two different behaviours under one version string would defeat the staleness filter MANIFEST_EXTRACTOR_CURRENT_VERSIONS exists to enforce. Bumps artist_ocr straight to v4, updates the matching manifest entry in run_image_evidence_cohort.py, and documents the collision in both the version constant's own history comment and catalog-completion-plan.md's still has both at v2, no collision). Also fixes golden_set.py's #677 re-verification comment, which cited MPCAutofill/scripts/experiments/ocr_ladder_tier_attribution.py - a worktree-only analysis script never committed to this repo, per the established convention (see docs/reports/2026-07-23-ocr-preprocessing- probe.md's own LIVE STATE) of not committing one-off probes. Points at catalog-completion-plan.md's already-committed #677 section instead, which documents the same 450-card probe's methodology and results. * fix(docs): cite OCR ladder probe scripts as prose, not backticked paths Both scripts are deliberately uncommitted (worktree-only analysis tooling, same convention as the 2026-07-23 probe reports). Backticked citations to their paths tripped docs_lint.py's path-reference rule since MPCAutofill/scripts/ doesn't exist in the repo. Rephrase as prose so the provenance stays legible without a machine-checkable path reference to a file that was never meant to be committed. (cherry picked from commit c5eee1c)
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_collector_line_ocr_attempts(MPCAutofill/cardpicker/image_evidence.py) dropsits own tier 3: a PSM-11 re-try of tier 1's own preprocessing variants. Two
independent real-production probes (450 forced-escalation cards combined) walked
the old 3-tier ladder to full completion and recorded, per card, the first tier
at which a candidate-validated genuine match appeared: tier 2 produced 2
genuine matches across the combined sample; the removed tier 3 produced zero -
only more lexicon-valid-but-uncorroborated noise, consistent with the earlier
preprocessing probes' own ~99% hopeless-art-noise finding for this population.
All 30
golden_set.GOLDEN_CARD_IDScards were cross-checked and none everresolved uniquely at tier 3, so this collapse changes zero golden-set
expectations. Worst case drops from 8 tesseract calls to 6.
collector_line_ocrandcollector_line_tsvbumpv2 -> v3(both read throughthe same escalation loop and are stale together under the new ladder, per
EXTRACTOR_OWNERSHIP's own "bump every listed key together" convention).artist_ocrskips straight tov4rather thanv3:artist-ocr-v3was alreadyclaimed by #685 (merged separately, unrelated - wires
modern_artist_credit.recognize_artist_creditin as a third artist fallback, noladder involvement) before this change's own bump landed. Stamping this ladder
collapse as
v3too would put two different behaviours under one versionstring, defeating
MANIFEST_EXTRACTOR_CURRENT_VERSIONS's own staleness filter.legal_lineis untouched atv2.This also lifts the
EXCLUDED_HELPERSentry that #690 left in place for thisexact branch while it was in flight -
_collector_line_ocr_attemptsandpreprocess_fallback_variantsnow carry real ownership entries incheck_extractor_ownership_totality.py.A companion idea named in the same issue - unifying the collector-line and
legal-line OCR reads into one full-width pass - was investigated and
deliberately not shipped: a real-image probe found it regressed 46 of 150
currently-correct collector-number reads (mostly proxy/watermark text absorbed
by the set-code regex, or the wider crop dropping the collector-number line
outright) with zero improvements.
docs/features/catalog-completion-plan.mdrecords that as a legitimate negative finding.
Re-land note
This re-lands #698, which merged cleanly (
c5eee1c8, 2026-08-05T03:05:26Z) butinto the wrong base branch: it was stacked on
feat/extractor-ownership-visibility(PR #690's branch) while #690 was still in flight, and its base was never
re-pointed at
masterafter #690 landed separately as a squash-merge(
966b58ac). The merge therefore went into that now-permanently-divergedfeature branch instead of
master, and none of #698's changes ever reachedproduction. This PR is
c5eee1c8's diff cherry-picked onto currentmaster,with the two resulting conflicts (both in files
masterhad since moved under#685) resolved to keep both #685's artist-crop-fallback wiring and this ladder
collapse rather than choosing one over the other.
feat/extractor-ownership-visibilityitself is left untouched - it is stale and merging it would revert five other
landed PRs.
Test plan
pytest cardpicker/tests/test_image_evidence.py(alone): 115 passedpytest cardpicker/tests/test_image_evidence.py cardpicker/tests/test_golden_set.py cardpicker/tests/test_run_image_evidence_cohort.py(together): 211 passedpytest .github/scripts/tests/test_check_extractor_ownership_totality.py .github/scripts/tests/test_check_extractor_manifest_sync.py: 40 passedpython3 .github/scripts/check_extractor_ownership_totality.py: exit 0, "clean (19 declared contributors)"python3 .github/scripts/check_extractor_manifest_sync.py: exit 0, "clean (11 manifest keys, derived from MPCAutofill/cardpicker/image_evidence.py)"python3 .github/scripts/docs_lint.py --strict: exit 0, "docs-lint: clean."pre-commit(ran automatically ongit cherry-pick --continue, all touched files): ruff, isort, black, mypy, prettier all passedCOLLECTOR_LINE_OCR_EXTRACTOR_VERSION = "collector-line-ocr-v3",ARTIST_OCR_EXTRACTOR_VERSION = "artist-ocr-v4",COLLECTOR_LINE_TSV_EXTRACTOR_VERSION = "collector-line-tsv-v3",LEGAL_LINE_EXTRACTOR_VERSION = "legal-line-v2"(unchanged) inimage_evidence.py, andMANIFEST_EXTRACTOR_CURRENT_VERSIONSinrun_image_evidence_cohort.pyagrees key-for-key