feat(golden-set): pin genuine trimmed real-printing and sideways-layout cards (issue #683 follow-up) - #703
Merged
Conversation
…ut cards (issue #683 follow-up) Closes two known golden-set coverage gaps, additively (original 30 cards and their expectations untouched). Trimmed geometry: the trimmed cohort's dominant-custom-render finding (issue #689) was based on the 8 highest-volume trimmed sources (88.4% of the 2,786-card population). Sampled 7 cards from 6 previously- unchecked long-tail sources and visually inspected each real fetched image against the live Scryfall API. Found the population is NOT entirely custom renders: 3 of 7 carried real, Scryfall-verifiable collector-line data. One of those (a "MH3-AI" credit line) looked equally plausible but was confirmed FABRICATED against Scryfall - not pinned. The genuine one, card 35449 ("Fierce Guardianship", C20 #35, artist Randy Vargas - verified including exact flavor-text match), is now pinned. It is a real printing but not a photographic scan - its dimensions match Scryfall's own border_crop asset, not a camera/ scanner capture, so no genuine photographic bleed-trimmed scan was found in this catalogue. Sideways layout: pinned card 130028 ("Bowie Base One"), a real Scryfall layout:planar card (verified via a live Scryfall layout:planar/scheme search), landscape-oriented with all text rotated 90° from what every extractor's fixed-fraction crop boxes assume. Its real run does not show the clean across-the-board abstention a naive prediction would expect: collector_line_ocr and legal_line confidently misparse the card's own sideways credit-line text (a false-positive "not"/"2024" set-code/collector-number pair), while artist_ocr correctly abstains despite reading real (but irrelevant) rules-text noise from the same crop. All 11 GOLDEN_EXPECTATIONS keys extended for both new ids, each with an inline comment stating why the card is pinned and what regression it protects against. Values recorded against real, fetch_and_compute_card_evidence_for_tests() runs over the live production images (host venv, real network fetch, zero persistence), same method as every existing pin. Docs: docs/features/catalog-completion-plan.md's golden-set substrate paragraph and docs/troubleshooting.md's issue #683 entry both updated in place to reflect the 32-card total and the refined finding. Verification: cardpicker/tests/test_golden_set.py - 29/29 passed (host venv against the isolated pp_test_pg test database, no production writes). black/ruff/isort clean on golden_set.py. docs_lint.py clean.
…sideways-coverage # Conflicts: # MPCAutofill/cardpicker/golden_set.py # docs/troubleshooting.md
…ured run (issue #683) Fetched both new golden cards (35449, 130028) from the CDN at DEFAULT_FETCH_DPI=250 and ran compute_card_evidence() directly against current code - no DB, no persistence. Every pinned value for both cards reproduced exactly, including the two the prior analytical argument didn't cover (130028's collector_line_ocr match and both cards' collector_line_tsv=True), confirming neither ever depended on the tier 3 that #677 removed.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes two known golden-set coverage gaps (
cardpicker/golden_set.py), additively — the original 30 pinned cards and their expectations are untouched.What was measured
Trimmed geometry. Issue #689's "the
trimmedcohort is dominated by fan-made custom renders" finding was based on the 8 highest-volume trimmed sources (88.4% of the 2,786-card trimmed population). This PR sampled 7 real cards from 6 previously-unchecked long-tail sources (Monza81, Mursu, MrTeferi, Mithrane, CompC, TorskeRavn, Kevin Kneupper) and visually inspected each fetched image against the live Scryfall API.Result: the population is not entirely custom renders. 3 of 7 sampled cards carried real, Scryfall-verifiable collector-line data. One of those — a "MH3-AI • EN Kevin Kneupper ft. Midjourney" credit line on "Witch-Blessed Meadow" — looked equally plausible but was confirmed fabricated: the real MH3 #239 printing is by Tyler Walpole (painted art), and "MH3-AI" does not exist as a Scryfall set. Not pinned.
The genuine one, card 35449 ("Fierce Guardianship"), verified against Scryfall's own
c20/35record (artist Randy Vargas, exact flavor-text match), is now pinned. It is a real printing but not a photographic scan — its dimensions match Scryfall's ownborder_cropimage asset, not a camera/scanner capture of a physical card. No genuine photographic bleed-trimmed scan was found in this catalogue.Sideways layout. Pinned card 130028 ("Bowie Base One"), a real Scryfall
layout:planarcard (verified via a livelayout:planar or layout:schemeScryfall search, not guessed from the name). Its real extraction run does not show the clean across-the-board abstention a naive prediction would expect:collector_line_ocr/legal_lineconfidently misparse the card's own sideways credit-line text into a false-positive(set_code="not", collector_number="2024")pair, whileartist_ocrcorrectly abstains despite reading real (but irrelevant) rules-text noise from the same crop region.What's covered / not covered
All 11
GOLDEN_EXPECTATIONSkeys extended for both new ids, each with an inline comment explaining why the card is pinned and what regression it protects against — seegolden_set.py's ownGOLDEN_CARD_IDScomment for the full verification writeup.Remains uncovered: a genuine photographic bleed-trimmed scan of a real card (none turned up in this sample — see the
trimmed-cohort comment for what that does and doesn't mean). Sideways coverage is a single card from one cohort (planar);schemelayout is not separately represented.Reconcile with master (2026-08-05)
Master moved to
17345a48while this PR was open — two siblings landed: #696 (persistedCanonicalPrintingMetadata.layout) and #702 (collapsed the Stage C OCR attempt ladder from 3 tiers to 2; bumpedCOLLECTOR_LINE_OCR_EXTRACTOR_VERSION/COLLECTOR_LINE_TSV_EXTRACTOR_VERSIONv2→v3 andARTIST_OCR_EXTRACTOR_VERSIONv3→v4). Merged (not rebased, to avoid a force-push) at7757257b.Conflicts resolved in
golden_set.pyanddocs/troubleshooting.md— both sides were additive in each case, so both bodies of content were kept: master'scollector_line_ocrre-verification comment for the original 30 alongside this branch's two new pins and their provenance comment; this branch's rewritten trimmed-cohort closing sentence followed by master's newcollectstatic/staticfiles-manifest troubleshooting entry.docs/features/catalog-completion-plan.mdauto-merged cleanly (non-overlapping edits); one sentence was added to its golden-set bullet pointing at the open item below.Known gap, not fully closed by this reconcile:
35449/130028's OCR-derived expectations were recorded before #702's ladder collapse. #702's own re-verification pass covered only the original 30 cards — these two weren't inGOLDEN_CARD_IDSyet when it ran. Breaking down what's actually known:35449'scollector_line_ocr(already a full abstention under the pre-perf(image-evidence): collapse Stage C OCR attempt ladder from 3 tiers to 2 (issue #677) (re-land of #698) #702 8-attempt ladder — removing an attempt tier can only remove an opportunity to succeed, never create one) and both cards'artist_ocr(same argument, also an abstention).130028'scollector_line_ocrmatch ({"set_code": "not", "collector_number": "2024"}) and both cards'collector_line_tsv=True— these resolved at some attempt tier under the old ladder, and which tier is unknown without tracing a live run.MPCAutofill/.envpresent in the worktree (not recreated — out of scope to source one from elsewhere on the box), no production-DB-default Postgres port reachable on the host, andmpcautofill_django(which does have DB access) was off limits toexec/cpunder an open ruling for this task's duration. Seegolden_set.py's owncollector_line_ocrcomment for the full reasoning trail.extract_card_evidence()pass over cards 35449 and 130028 against current (post-perf(image-evidence): collapse Stage C OCR attempt ladder from 3 tiers to 2 (issue #677) (re-land of #698) #702) code and confirm or correctGOLDEN_EXPECTATIONS["collector_line_ocr"][130028]andGOLDEN_EXPECTATIONS["collector_line_tsv"][35449]/[130028].Test plan
cardpicker/tests/test_golden_set.pyalone — 29 passedcardpicker/tests/test_image_evidence.pyalone — 115 passed.github/scripts/check_extractor_ownership_totality.py— clean (19 declared contributors).github/scripts/check_extractor_manifest_sync.py— clean (11 manifest keys).github/scripts/tests/test_check_extractor_ownership_totality.py— 18 passedmanage.py makemigrations --check --dry-run— no changes detectedblack/ruff/isort/mypy/prettier— all passed via pre-commit on the merge commitextract_card_evidence()for cards 35449/130028 against current (perf(image-evidence): collapse Stage C OCR attempt ladder from 3 tiers to 2 (issue #677) (re-land of #698) #702) code — blocked in this task's environment, see "Reconcile with master" above; tracked as the one open itemDocs
docs/features/catalog-completion-plan.md's golden-set substrate paragraph anddocs/troubleshooting.md's issue #683 entry both updated in place to reflect the 32-card total and the refined finding; the golden-set bullet additionally notes the open re-verification item above.