feat(artbox-exemplar-index): self-referential artbox-phash exemplar index (issue #508 phase 1) - #701
Merged
Merged
Conversation
…ndex (issue #508 phase 1) Builds ArtboxPhashExemplar, the second illustration-deduction path's reference index, and its read-only backfill/retraction commands. Reuses the batched/resumable/idempotent shape from PR #694 (feat/local-backfill-canonical-hash) - only the source changes, from Scryfall CDN fetches to a scan of our own ImageEvidence; this backfill needs none of that branch's threaded fetch pipeline since every input is already a local DB read. Seeds from two sources, per the owner's 2026-08-05 extension of #508's original human-only design: human-backed printing resolutions (printing_tag_status == RESOLVED, always human-backed by the consensus gate) and unopposed join-key machine votes at 0.75/0.85 confidence. The artist-disagreement tier (0.65) and the no-match tier (0.6) are both excluded from seeding. Every exemplar records its own seed kind, confidence, and a seed_group_key that lets a bad seed retract together with everything it seeded in one filtered delete. Never fetches Scryfall images - phash comparability requires the same crop geometry our own extractor guarantees. Phase 1 scope only: no matching calculator, no vote, no change to consensus or the human-backed resolution gate. Documents the mechanism in docs/identification-pipeline.md's Parallel detectors section, stating explicitly what it does not do.
WilfordGrimley
added a commit
that referenced
this pull request
Aug 5, 2026
…03, repoint dependency at 0102_artbox_phash_exemplar (#696) Merged origin/master into this branch, which brought in 0102_artbox_phash_exemplar (#701) depending on 0101_delete_printingtagvote — the same parent this branch's migration used, producing two leaves. Renumbered this branch's migration to 0103 and repointed its dependency at 0102_artbox_phash_exemplar. Operation unchanged.
WilfordGrimley
added a commit
that referenced
this pull request
Aug 5, 2026
…hort (issue #693) (#696) * feat(printing-metadata): persist Scryfall layout, measure sideways cohort (issue #693) Step 1: add CanonicalPrintingMetadata.layout (migration 0102), wired from PrintingMetadataRow.layout (already parsed, previously discarded after the DOUBLE_FACED_LAYOUTS check). Follows the existing border_color/frame field conventions on that model exactly (blank-string sentinel, CharField, added to _METADATA_SYNC_FIELDS). Step 2: measured the sideways cohort (Scryfall layout planar/scheme) against the live catalogue via read-only docker exec queries. 41 of 230,488 catalogued cards (0.018%) resolve to a sideways layout; the cohort's own collector-line parse failure rate (2.4%, 1/41) is not elevated versus the whole-catalogue rate measured multiple ways (15.5%-38.4% depending on denominator). Gate result: cohort negligible, no elevated failure signal - stopping here per the brief's own explicit exit ramp. Step 3 (crop-box rotation) not built. None of golden_set.py's 30 pinned cards is a sideways layout (confirmed via live query) - the golden set cannot catch a regression in this path. Also documents a fresh-worktree makemigrations blocker (missing staticfiles manifest) in docs/troubleshooting.md, hit and resolved while verifying this migration. * fix(docs): correct urls.py path in favicon.ico troubleshooting entry MPCAutofill/urls.py -> MPCAutofill/MPCAutofill/urls.py (Django project's own urls module, which contains the static() call for the favicon redirect). Fixes docs_lint.py path-reference failure on PR #696. * docs(catalog-completion-plan): document CanonicalPrintingMetadata.layout and the sideways-cohort measurement (issue #693) * fix(catalog-completion-plan): retract biased sideways-cohort measurement, redo with rotation trial (issue #693) The 41-card/0.018% cohort was drawn only from cards already carrying a canonical_card link, which is acquired by successful identification - a sideways card whose crop lands on rotated content and fails identification could never appear in it. Redone by expected layout (Card.name join against CanonicalCard's Scryfall layout, independent of identification success): 1,093 cards (0.474%), 53.3% collector-line parse failure vs. 40.4% for the correct comparison group (unlinked population generally, not the whole-catalogue figure that mixes in the already-identified linked population). That elevated signal motivated a rotation trial: 30 real images sampled from the cohort's never-linked pool, existing collector-line and Illus.-anchor crops run unmodified then again with the image rotated 90 degrees both directions. Zero candidate-validated genuine matches in any orientation - a content problem (fan-made renders without a real collector line, overlapping issue #683's trimmed-cohort finding), not a crop-geometry problem. Rotation was evaluated and deliberately not built. PR #696 body patched to match via gh api PATCH (gh pr edit fails on this repo). * fix(migrations): renumber 0102_canonicalprintingmetadata_layout to 0103, repoint dependency at 0102_artbox_phash_exemplar (#696) Merged origin/master into this branch, which brought in 0102_artbox_phash_exemplar (#701) depending on 0101_delete_printingtagvote — the same parent this branch's migration used, producing two leaves. Renumbered this branch's migration to 0103 and repointed its dependency at 0102_artbox_phash_exemplar. Operation unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ArtboxPhashExemplar(+ArtboxPhashExemplarSeedKind): a labelled association from a card's own currentartbox_phash(issue Artbox perceptual-hash extractor: evidence-only, rides the next whole-catalog pass #480's evidence-only extractor) to theillustration_idof the printing that scan was identified as. Sourced exclusively from our own DB, never Scryfall images (PR feat(local_phash): local-only resumable backfill for CanonicalCard.image_hash #694 was closed and deferred to Deferred: cross-source phash reference corpus backfill (PR #694) — supplanted by #508's self-referential exemplar index #697 for exactly this reason — phash comparability needs identical crop geometry, which only our own extractor guarantees).cardpicker/artbox_exemplar_backfill.py: the seeding logic, reusing the batched/resumable/idempotent shape from PR feat(local_phash): local-only resumable backfill for CanonicalCard.image_hash #694 (feat/local-backfill-canonical-hash) — only the source changes, from Scryfall CDN fetches to a scan of our ownImageEvidence. Two seed sources, per the owner's 2026-08-05 extension of Artbox-phash illustration matching: resolution-seeded exemplar index (second deduction path) #508's original human-only design:printing_tag_status == RESOLVED— resolution always requires a human-backed vote perresolve_weighted_consensus's gate, so no per-vote inspection is needed).JOIN_KEY_SEED_CONFIDENCE_FLOOR). The artist-disagreement tier (0.65, 38 votes) and the no-match tier (0.6, 16,537 votes — not identifications at all) are both excluded; see that constant's own comment for the full measured distribution and reasoning.seed_group_key(mirroringprinting_consensus.md5_group_keyfor the human-resolution case, or the source vote's own pk for the machine case) — a bad seed retracts together with everything it seeded via one filtered delete on that key. Human and machine seeds stay permanently distinguishable (is_human_backed, enforced by a CheckConstraint againstseed_kind).backfill_artbox_phash_exemplars(seeding,--dry-runsupported) andretract_artbox_phash_exemplars(retraction by seed_group_key/card/illustration/run/source-vote, dry-run by default) management commands.measure_unresolved_coverage): of every currently-UNRESOLVED card carrying a currentartbox_phash, how many match the exemplar index at d=0 and separately at d≤2 — the honest phase-2 estimate, computed read-only every invocation.docs/identification-pipeline.md's "Parallel detectors" section (the neighbouring Artbox perceptual-hash extractor: evidence-only, rides the next whole-catalog pass #480/OCR-free-path section), stating explicitly what phase 1 does NOT do: no matching calculator, no vote, no consensus change, no touch to the human-backed resolution gate.0102_artbox_phash_exemplar.pyon top of0101_delete_printingtagvote.Coverage measurement (real, read-only, against the live catalogue)
Executed via a read-only script (mirroring this branch's exact query logic verbatim) inside the live
mpcautofill_djangocontainer — see DEVIATIONS in the session report for why the literal management command wasn't invoked against production directly. Results, 2026-08-05:artbox_phash: 43,400 match at d=0 (28.7%), 46,457 match at d≤2 (30.7%).This is a strong phase-2 signal — roughly 3 in 10 unresolved cards already have a perceptually-identical or near-identical exemplar in the index, entirely from resolutions and join-key votes that already exist today.
Test plan
pytest cardpicker/tests/test_artbox_exemplar_backfill.py— 33 tests, all passing (model constraints, both seed passes, confidence floor, staleness/no-match exclusion, idempotent resume, dry-run, retraction, coverage matching).pytest cardpicker/) — 3639 passed, 8 skipped (pre-existing, unrelated), 0 failed.pre-commit runon all touched/new files — ruff, isort, black, mypy, prettier all pass.makemigrations --check --dry-run— no changes detected (migration matches models exactly); confirmed single leaf on top of0101_delete_printingtagvote.