Skip to content

feat(golden-set): pin genuine trimmed real-printing and sideways-layout cards (issue #683 follow-up) - #703

Merged
WilfordGrimley merged 4 commits into
masterfrom
golden-set-trimmed-sideways-coverage
Aug 6, 2026
Merged

feat(golden-set): pin genuine trimmed real-printing and sideways-layout cards (issue #683 follow-up)#703
WilfordGrimley merged 4 commits into
masterfrom
golden-set-trimmed-sideways-coverage

Conversation

@WilfordGrimley

@WilfordGrimley WilfordGrimley commented Aug 5, 2026

Copy link
Copy Markdown

What

Closes two known golden-set coverage gaps (cardpicker/golden_set.py), additively — the original 30 pinned cards and their expectations are untouched.

What was measured

Trimmed geometry. Issue #689's "the trimmed cohort is dominated by fan-made custom renders" finding was based on the 8 highest-volume trimmed sources (88.4% of the 2,786-card trimmed population). This PR sampled 7 real cards from 6 previously-unchecked long-tail sources (Monza81, Mursu, MrTeferi, Mithrane, CompC, TorskeRavn, Kevin Kneupper) and visually inspected each fetched image against the live Scryfall API.

Result: the population is not entirely custom renders. 3 of 7 sampled cards carried real, Scryfall-verifiable collector-line data. One of those — a "MH3-AI • EN Kevin Kneupper ft. Midjourney" credit line on "Witch-Blessed Meadow" — looked equally plausible but was confirmed fabricated: the real MH3 #239 printing is by Tyler Walpole (painted art), and "MH3-AI" does not exist as a Scryfall set. Not pinned.

The genuine one, card 35449 ("Fierce Guardianship"), verified against Scryfall's own c20/35 record (artist Randy Vargas, exact flavor-text match), is now pinned. It is a real printing but not a photographic scan — its dimensions match Scryfall's own border_crop image asset, not a camera/scanner capture of a physical card. No genuine photographic bleed-trimmed scan was found in this catalogue.

Sideways layout. Pinned card 130028 ("Bowie Base One"), a real Scryfall layout:planar card (verified via a live layout:planar or layout:scheme Scryfall search, not guessed from the name). Its real extraction run does not show the clean across-the-board abstention a naive prediction would expect: collector_line_ocr/legal_line confidently misparse the card's own sideways credit-line text into a false-positive (set_code="not", collector_number="2024") pair, while artist_ocr correctly abstains despite reading real (but irrelevant) rules-text noise from the same crop region.

What's covered / not covered

All 11 GOLDEN_EXPECTATIONS keys extended for both new ids, each with an inline comment explaining why the card is pinned and what regression it protects against — see golden_set.py's own GOLDEN_CARD_IDS comment for the full verification writeup.

Remains uncovered: a genuine photographic bleed-trimmed scan of a real card (none turned up in this sample — see the trimmed-cohort comment for what that does and doesn't mean). Sideways coverage is a single card from one cohort (planar); scheme layout is not separately represented.

Reconcile with master (2026-08-05)

Master moved to 17345a48 while this PR was open — two siblings landed: #696 (persisted CanonicalPrintingMetadata.layout) and #702 (collapsed the Stage C OCR attempt ladder from 3 tiers to 2; bumped COLLECTOR_LINE_OCR_EXTRACTOR_VERSION/COLLECTOR_LINE_TSV_EXTRACTOR_VERSION v2→v3 and ARTIST_OCR_EXTRACTOR_VERSION v3→v4). Merged (not rebased, to avoid a force-push) at 7757257b.

Conflicts resolved in golden_set.py and docs/troubleshooting.md — both sides were additive in each case, so both bodies of content were kept: master's collector_line_ocr re-verification comment for the original 30 alongside this branch's two new pins and their provenance comment; this branch's rewritten trimmed-cohort closing sentence followed by master's new collectstatic/staticfiles-manifest troubleshooting entry. docs/features/catalog-completion-plan.md auto-merged cleanly (non-overlapping edits); one sentence was added to its golden-set bullet pointing at the open item below.

Known gap, not fully closed by this reconcile: 35449/130028's OCR-derived expectations were recorded before #702's ladder collapse. #702's own re-verification pass covered only the original 30 cards — these two weren't in GOLDEN_CARD_IDS yet when it ran. Breaking down what's actually known:

  • Provably unaffected, no live run needed: 35449's collector_line_ocr (already a full abstention under the pre-perf(image-evidence): collapse Stage C OCR attempt ladder from 3 tiers to 2 (issue #677) (re-land of #698) #702 8-attempt ladder — removing an attempt tier can only remove an opportunity to succeed, never create one) and both cards' artist_ocr (same argument, also an abstention).
  • Not provably safe, and not independently re-verified against current code: 130028's collector_line_ocr match ({"set_code": "not", "collector_number": "2024"}) and both cards' collector_line_tsv=True — these resolved at some attempt tier under the old ladder, and which tier is unknown without tracing a live run.
  • This reconcile could not reach production data to run that check from its own environment: no MPCAutofill/.env present in the worktree (not recreated — out of scope to source one from elsewhere on the box), no production-DB-default Postgres port reachable on the host, and mpcautofill_django (which does have DB access) was off limits to exec/cp under an open ruling for this task's duration. See golden_set.py's own collector_line_ocr comment for the full reasoning trail.
  • Open item: someone with production DB access should run a real, no-persistence extract_card_evidence() pass over cards 35449 and 130028 against current (post-perf(image-evidence): collapse Stage C OCR attempt ladder from 3 tiers to 2 (issue #677) (re-land of #698) #702) code and confirm or correct GOLDEN_EXPECTATIONS["collector_line_ocr"][130028] and GOLDEN_EXPECTATIONS["collector_line_tsv"][35449]/[130028].

Test plan

Docs

docs/features/catalog-completion-plan.md's golden-set substrate paragraph and docs/troubleshooting.md's issue #683 entry both updated in place to reflect the 32-card total and the refined finding; the golden-set bullet additionally notes the open re-verification item above.

…ut cards (issue #683 follow-up)

Closes two known golden-set coverage gaps, additively (original 30 cards
and their expectations untouched).

Trimmed geometry: the trimmed cohort's dominant-custom-render finding
(issue #689) was based on the 8 highest-volume trimmed sources (88.4%
of the 2,786-card population). Sampled 7 cards from 6 previously-
unchecked long-tail sources and visually inspected each real fetched
image against the live Scryfall API. Found the population is NOT
entirely custom renders: 3 of 7 carried real, Scryfall-verifiable
collector-line data. One of those (a "MH3-AI" credit line) looked
equally plausible but was confirmed FABRICATED against Scryfall - not
pinned. The genuine one, card 35449 ("Fierce Guardianship", C20 #35,
artist Randy Vargas - verified including exact flavor-text match), is
now pinned. It is a real printing but not a photographic scan - its
dimensions match Scryfall's own border_crop asset, not a camera/
scanner capture, so no genuine photographic bleed-trimmed scan was
found in this catalogue.

Sideways layout: pinned card 130028 ("Bowie Base One"), a real
Scryfall layout:planar card (verified via a live Scryfall
layout:planar/scheme search), landscape-oriented with all text rotated
90° from what every extractor's fixed-fraction crop boxes assume. Its
real run does not show the clean across-the-board abstention a naive
prediction would expect: collector_line_ocr and legal_line confidently
misparse the card's own sideways credit-line text (a false-positive
"not"/"2024" set-code/collector-number pair), while artist_ocr
correctly abstains despite reading real (but irrelevant) rules-text
noise from the same crop.

All 11 GOLDEN_EXPECTATIONS keys extended for both new ids, each with
an inline comment stating why the card is pinned and what regression
it protects against. Values recorded against real,
fetch_and_compute_card_evidence_for_tests() runs over the live
production images (host venv, real network fetch, zero persistence),
same method as every existing pin.

Docs: docs/features/catalog-completion-plan.md's golden-set substrate
paragraph and docs/troubleshooting.md's issue #683 entry both updated
in place to reflect the 32-card total and the refined finding.

Verification: cardpicker/tests/test_golden_set.py - 29/29 passed
(host venv against the isolated pp_test_pg test database, no
production writes). black/ruff/isort clean on golden_set.py.
docs_lint.py clean.
…sideways-coverage

# Conflicts:
#	MPCAutofill/cardpicker/golden_set.py
#	docs/troubleshooting.md
…ured run (issue #683)

Fetched both new golden cards (35449, 130028) from the CDN at DEFAULT_FETCH_DPI=250
and ran compute_card_evidence() directly against current code - no DB, no
persistence. Every pinned value for both cards reproduced exactly, including the
two the prior analytical argument didn't cover (130028's collector_line_ocr match
and both cards' collector_line_tsv=True), confirming neither ever depended on the
tier 3 that #677 removed.
@WilfordGrimley
WilfordGrimley merged commit e2b2854 into master Aug 6, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant