From 1393a0352e0bab5d9d0682b1cd78d8e9e89a88be Mon Sep 17 00:00:00 2001 From: wilfordgrimley <2397930+WilfordGrimley@users.noreply.github.com> Date: Wed, 5 Aug 2026 02:41:28 +0000 Subject: [PATCH] feat(artbox-exemplar-index): self-referential artbox-phash exemplar index (issue #508 phase 1) Builds ArtboxPhashExemplar, the second illustration-deduction path's reference index, and its read-only backfill/retraction commands. Reuses the batched/resumable/idempotent shape from PR #694 (feat/local-backfill-canonical-hash) - only the source changes, from Scryfall CDN fetches to a scan of our own ImageEvidence; this backfill needs none of that branch's threaded fetch pipeline since every input is already a local DB read. Seeds from two sources, per the owner's 2026-08-05 extension of #508's original human-only design: human-backed printing resolutions (printing_tag_status == RESOLVED, always human-backed by the consensus gate) and unopposed join-key machine votes at 0.75/0.85 confidence. The artist-disagreement tier (0.65) and the no-match tier (0.6) are both excluded from seeding. Every exemplar records its own seed kind, confidence, and a seed_group_key that lets a bad seed retract together with everything it seeded in one filtered delete. Never fetches Scryfall images - phash comparability requires the same crop geometry our own extractor guarantees. Phase 1 scope only: no matching calculator, no vote, no change to consensus or the human-backed resolution gate. Documents the mechanism in docs/identification-pipeline.md's Parallel detectors section, stating explicitly what it does not do. --- .../cardpicker/artbox_exemplar_backfill.py | 422 ++++++++++++++++ .../backfill_artbox_phash_exemplars.py | 106 ++++ .../retract_artbox_phash_exemplars.py | 93 ++++ .../migrations/0102_artbox_phash_exemplar.py | 75 +++ MPCAutofill/cardpicker/models.py | 109 ++++ .../tests/test_artbox_exemplar_backfill.py | 473 ++++++++++++++++++ docs/identification-pipeline.md | 27 + 7 files changed, 1305 insertions(+) create mode 100644 MPCAutofill/cardpicker/artbox_exemplar_backfill.py create mode 100644 MPCAutofill/cardpicker/management/commands/backfill_artbox_phash_exemplars.py create mode 100644 MPCAutofill/cardpicker/management/commands/retract_artbox_phash_exemplars.py create mode 100644 MPCAutofill/cardpicker/migrations/0102_artbox_phash_exemplar.py create mode 100644 MPCAutofill/cardpicker/tests/test_artbox_exemplar_backfill.py diff --git a/MPCAutofill/cardpicker/artbox_exemplar_backfill.py b/MPCAutofill/cardpicker/artbox_exemplar_backfill.py new file mode 100644 index 000000000..6eaac337f --- /dev/null +++ b/MPCAutofill/cardpicker/artbox_exemplar_backfill.py @@ -0,0 +1,422 @@ +""" +Issue #508 phase 1: seeds `ArtboxPhashExemplar` from our own DB, never from Scryfall images (see +that model's own docstring for the full design rationale). Two seed sources, owner-ratified +2026-08-05: + +- Human-backed printing resolutions (`Card.printing_tag_status == RESOLVED` - resolution always + requires a human-backed vote, see `vote_consensus.resolve_weighted_consensus`, so no per-vote + inspection is needed to classify one as such). +- High-confidence join-key machine votes (`local_calculate_verdicts`'s join-key calculator, at or + above `JOIN_KEY_SEED_CONFIDENCE_FLOOR`). + +Reuses the batched/resumable/idempotent shape `local_phash.run_canonical_hash_backfill` (PR #694, +branch `feat/local-backfill-canonical-hash`) established for a local, zero-network backfill: filter +on "not yet done" as the checkpoint (there, `image_hash=0`; here, no existing +`ArtboxPhashExemplar` row for the card), a `--dry-run` that reports without writing. This backfill +needs none of that branch's threaded fetch pipeline - every input here is already a local DB read +(`ImageEvidence.artbox_phash`, `CardPrintingTag.confidence`), so there is nothing to fetch and +nothing to parallelize; `Model.objects.bulk_create(..., batch_size=...)` does the insert-side +chunking on its own. +""" + +import itertools +import time +from dataclasses import dataclass +from typing import Any, Iterable, Iterator, Optional, TypedDict +from uuid import UUID + +from django.db.models import F, QuerySet + +from cardpicker.evidence_transfer import md5_currency_q +from cardpicker.models import ( + ArtboxPhashExemplar, + ArtboxPhashExemplarSeedKind, + Card, + CardPrintingTag, + ImageEvidence, + PrintingTagStatus, +) +from cardpicker.printing_consensus import md5_group_key + +# Duplicated as a literal rather than imported from `local_calculate_verdicts` - matching that +# module's own "avoid a hard import-time dependency between sibling engines over one constant" +# precedent (its own comment on `DEDUCTIVE_BACKFILL_ANONYMOUS_ID`/`OCR_CONFIDENCE_BOTH`). +JOIN_KEY_ANONYMOUS_ID = "stage-d-join-key-v1" + +# The floor between `local_calculate_verdicts.JOIN_KEY_CONFIDENCE_COLLECTOR_ONLY`/ +# `JOIN_KEY_CONFIDENCE_SYMBOL_TIEBREAK` (0.75) and `JOIN_KEY_CONFIDENCE_ARTIST_DISAGREEMENT` +# (0.65). Measured 2026-08-05 against the live join-key vote population: 41,585 votes at 0.85, +# 498 at 0.75, 38 at 0.65, 16,537 at 0.6 (`JOIN_KEY_NO_MATCH_CONFIDENCE` - `is_no_match=True` +# rows, never identifications, structurally excluded below regardless of this floor). +# +# 0.85 and 0.75 are both UNCONTRADICTED matches (both signals agree, or the collector-number/ +# symbol-tiebreak match stands with nothing disagreeing) - 42,083 votes admitted. 0.65 is a real +# match assertion that carries a KNOWN contradicting signal (the artist OCR read disagrees with +# the matched printing's own artist) - only 38 votes, and excluding them is the conservative +# choice per the owner's framing: a wrong exemplar propagates to every card it later matches, so +# a floor that admits a marginal, actively-contradicted identification is worse than a smaller +# index. See docs/identification-pipeline.md's "Parallel detectors" section for where this is +# documented for a reader who isn't reading this module's own source. +JOIN_KEY_SEED_CONFIDENCE_FLOOR = 0.75 + +DEFAULT_BACKFILL_BATCH_SIZE = 500 + + +def human_resolution_seed_group_key(card: Card) -> str: + """ + Stable retraction identity for a HUMAN_RESOLUTION seed - shares a value with every OTHER + exemplar seeded from the same resolved md5 identity group (`printing_consensus.md5_group_key` + is the same group `printing_consensus.resolve_printing` itself tallies as one unit), so + retracting a wrong resolution retracts every exemplar it seeded, across every group member, + in one filtered delete - not just the one card a caller happens to be looking at. + """ + return f"human:{md5_group_key(card)}" + + +def join_key_seed_group_key(vote_id: int) -> str: + """ + Stable retraction identity for a JOIN_KEY_MACHINE seed - one vote seeds exactly one exemplar, + so this is a 1:1 key, kept in the same `f"{kind}:{value}"` shape as + `human_resolution_seed_group_key` rather than reusing the vote's own pk as a bare int, so a + caller retracting "by seed" never needs to know which of the two seed kinds it's holding. + """ + return f"machine:vote:{vote_id}" + + +def _current_artbox_evidence_queryset() -> "QuerySet[ImageEvidence]": + """ + Bulk form of `image_evidence.current_evidence_queryset` (that function is single-card; + `evidence_transfer.md5_currency_q` is its own bulk F-expression half, reused here directly) - + every `ImageEvidence` row that is CURRENT for its own card (live `content_phash` match, and + the md5-currency rule `md5_currency_q` expresses) AND carries a populated `artbox_phash`. A + stale row (an evidence row from a since-replaced image) can never reach here, and neither can + a card with no `artbox_phash` yet (79,207 of 230,378 rows, per issue #508's brief) - both + excluded by construction, never by a downstream filter a future caller could forget. + """ + return ImageEvidence.objects.filter( + content_hash=F("card__content_phash"), + card__content_phash__isnull=False, + artbox_phash__isnull=False, + ).filter(md5_currency_q()) + + +class _ExemplarFields(TypedDict): + illustration_id: UUID + artbox_phash: int + card_id: int + printing_id: int + seed_kind: str + is_human_backed: bool + source_vote_id: Optional[int] + confidence: Optional[float] + seed_group_key: str + content_hash: int + + +def _chunked(iterable: Iterable[Any], size: int) -> Iterator[list[Any]]: + """Groups `iterable` into lists of at most `size` - used only for the join-key pass's bulk + evidence lookup (one query per chunk instead of one per vote), never for the insert side + (`bulk_create`'s own `batch_size` argument already chunks that).""" + iterator = iter(iterable) + while True: + chunk = list(itertools.islice(iterator, size)) + if not chunk: + return + yield chunk + + +def _human_resolution_candidates() -> Iterator[_ExemplarFields]: + """ + Every current-evidence card whose `printing_tag_status` is RESOLVED and whose resolved + printing carries an `illustration_id` - resolution is human-backed by construction (see this + module's own docstring), so no per-vote inspection is needed. + `.exclude(card__artbox_phash_exemplar__isnull=False)` is the checkpoint: a card that already + has an exemplar row is never re-visited, the same NULL/absence-filter resumability discipline + `local_phash.run_canonical_hash_backfill` uses for `image_hash=0`. + """ + queryset = ( + _current_artbox_evidence_queryset() + .filter( + card__printing_tag_status=PrintingTagStatus.RESOLVED, + card__inferred_canonical_card__isnull=False, + card__inferred_canonical_card__printing_metadata__illustration_id__isnull=False, + ) + .exclude(card__artbox_phash_exemplar__isnull=False) + .select_related("card", "card__inferred_canonical_card__printing_metadata") + .order_by("card_id") + ) + for evidence in queryset.iterator(chunk_size=DEFAULT_BACKFILL_BATCH_SIZE): + card = evidence.card + printing = card.inferred_canonical_card + # Both asserts state what the queryset's own filters above already guarantee at runtime + # (inferred_canonical_card__isnull=False, printing_metadata__illustration_id__isnull= + # False, artbox_phash__isnull=False) - mypy cannot see through a QuerySet filter, so + # these are for the type checker, not a real runtime possibility. + assert printing is not None + assert printing.printing_metadata.illustration_id is not None + assert evidence.artbox_phash is not None + yield _ExemplarFields( + illustration_id=printing.printing_metadata.illustration_id, + artbox_phash=evidence.artbox_phash, + card_id=card.pk, + printing_id=printing.pk, + seed_kind=ArtboxPhashExemplarSeedKind.HUMAN_RESOLUTION, + is_human_backed=True, + source_vote_id=None, + confidence=None, + seed_group_key=human_resolution_seed_group_key(card), + content_hash=evidence.content_hash, + ) + + +def _join_key_machine_candidates( + batch_size: int, already_seeded_card_ids: set[int] +) -> Iterator[tuple[list[_ExemplarFields], int]]: + """ + Yields `(batch, skipped_in_batch)` for every `CardPrintingTag` vote cast by the join-key + calculator at or above `JOIN_KEY_SEED_CONFIDENCE_FLOOR`. `already_seeded_card_ids` is the set + of cards the human-resolution pass just seeded (or would have, under `--dry-run`) - a card + resolved AND join-key-voted is seeded once, from the higher-trust human resolution, never + both; this in-memory exclusion is what keeps that true even inside a single `--dry-run` + invocation, where the DB-level `.exclude(card__artbox_phash_exemplar__isnull=False)` checkpoint + (correct across separate invocations) hasn't actually written anything yet to exclude against. + A vote whose card has no CURRENT `artbox_phash` (stale evidence, or evidence that was never + computed) is counted in `skipped_in_batch`, never silently dropped. + """ + votes_queryset = ( + CardPrintingTag.objects.filter( + anonymous_id=JOIN_KEY_ANONYMOUS_ID, + is_no_match=False, + confidence__gte=JOIN_KEY_SEED_CONFIDENCE_FLOOR, + printing__printing_metadata__illustration_id__isnull=False, + ) + .exclude(card__artbox_phash_exemplar__isnull=False) + .select_related("printing__printing_metadata") + .order_by("pk") + ) + for vote_chunk in _chunked(votes_queryset.iterator(chunk_size=batch_size), batch_size): + vote_chunk = [vote for vote in vote_chunk if vote.card_id not in already_seeded_card_ids] + if not vote_chunk: + continue + card_ids = [vote.card_id for vote in vote_chunk] + evidence_by_card_id = { + card_id: (artbox_phash, content_hash) + for card_id, artbox_phash, content_hash in _current_artbox_evidence_queryset() + .filter(card_id__in=card_ids) + .values_list("card_id", "artbox_phash", "content_hash") + } + batch: list[_ExemplarFields] = [] + skipped = 0 + for vote in vote_chunk: + evidence = evidence_by_card_id.get(vote.card_id) + if evidence is None: + skipped += 1 + continue + artbox_phash, content_hash = evidence + # Same rationale as _human_resolution_candidates' own asserts above - the queryset + # filters already guarantee these are non-null; mypy just can't see through them. + assert artbox_phash is not None + assert vote.printing is not None + assert vote.printing.printing_metadata.illustration_id is not None + batch.append( + _ExemplarFields( + illustration_id=vote.printing.printing_metadata.illustration_id, + artbox_phash=artbox_phash, + card_id=vote.card_id, + printing_id=vote.printing_id, + seed_kind=ArtboxPhashExemplarSeedKind.JOIN_KEY_MACHINE, + is_human_backed=False, + source_vote_id=vote.pk, + confidence=vote.confidence, + seed_group_key=join_key_seed_group_key(vote.pk), + content_hash=content_hash, + ) + ) + yield batch, skipped + + +def _bulk_create_exemplars(fields_list: list[_ExemplarFields], run_id: Optional[str], batch_size: int) -> None: + ArtboxPhashExemplar.objects.bulk_create( + [ + ArtboxPhashExemplar( + illustration_id=fields["illustration_id"], + artbox_phash=fields["artbox_phash"], + card_id=fields["card_id"], + printing_id=fields["printing_id"], + seed_kind=fields["seed_kind"], + is_human_backed=fields["is_human_backed"], + source_vote_id=fields["source_vote_id"], + confidence=fields["confidence"], + seed_group_key=fields["seed_group_key"], + content_hash=fields["content_hash"], + run_id=run_id, + ) + for fields in fields_list + ], + batch_size=batch_size, + ) + + +@dataclass(frozen=True) +class ArtboxExemplarBackfillResult: + dry_run: bool = False + human_backed_seeded: int = 0 + machine_seeded: int = 0 + machine_skipped_stale_or_missing_evidence: int = 0 + distinct_illustration_ids: int = 0 + elapsed_seconds: float = 0.0 + + +def run_artbox_exemplar_backfill( + dry_run: bool = False, + batch_size: int = DEFAULT_BACKFILL_BATCH_SIZE, + limit: Optional[int] = None, + run_id: Optional[str] = None, +) -> ArtboxExemplarBackfillResult: + """ + Human-backed resolutions are seeded FIRST, in full, before the join-key machine pass even + queries - the priority order the owner's design implies ("seed from high-confidence join-key + hits AS WELL AS human-backed resolutions"): a card that qualifies for both is seeded once, + from the higher-trust human resolution. + + `limit` bounds the TOTAL number of exemplar rows created across both passes combined (for + testing/sampling - mirrors `local_phash.run_canonical_hash_backfill`'s own `--limit`), not a + per-pass limit. Human-backed resolutions currently number 12 catalogue-wide (2026-08-05), so + materializing that pass's candidates as one list, rather than streaming it in batches the way + the (much larger) machine pass below does, is a deliberate, safe simplification - not an + oversight that stops scaling once that population is no longer small; growth in resolved + printings is governed by real human review throughput, which will not silently jump this + module's own memory footprint. + """ + start_time = time.monotonic() + illustration_ids: set[UUID] = set() + + human_fields = list(_human_resolution_candidates()) + if limit is not None: + human_fields = human_fields[:limit] + human_backed_seeded = len(human_fields) + illustration_ids.update(fields["illustration_id"] for fields in human_fields) + if not dry_run and human_fields: + _bulk_create_exemplars(human_fields, run_id, batch_size) + + already_seeded_card_ids = {fields["card_id"] for fields in human_fields} + remaining = None if limit is None else max(limit - human_backed_seeded, 0) + + machine_seeded = 0 + machine_skipped = 0 + if remaining is None or remaining > 0: + for batch, skipped_in_batch in _join_key_machine_candidates(batch_size, already_seeded_card_ids): + machine_skipped += skipped_in_batch + if remaining is not None: + batch = batch[:remaining] + if not batch: + if remaining is not None and remaining <= 0: + break + continue + if not dry_run: + _bulk_create_exemplars(batch, run_id, batch_size) + machine_seeded += len(batch) + illustration_ids.update(fields["illustration_id"] for fields in batch) + if remaining is not None: + remaining -= len(batch) + if remaining <= 0: + break + + return ArtboxExemplarBackfillResult( + dry_run=dry_run, + human_backed_seeded=human_backed_seeded, + machine_seeded=machine_seeded, + machine_skipped_stale_or_missing_evidence=machine_skipped, + distinct_illustration_ids=len(illustration_ids), + elapsed_seconds=time.monotonic() - start_time, + ) + + +def dry_run_candidate_exemplar_hashes(batch_size: int = DEFAULT_BACKFILL_BATCH_SIZE) -> list[int]: + """ + Every `artbox_phash` value `run_artbox_exemplar_backfill(dry_run=True, ...)` would have + seeded, re-derived from the live tables rather than read back from a persisted table (nothing + was written) - for a dry run's own coverage measurement (`measure_unresolved_coverage`), + which needs the candidate hash set regardless of whether this invocation is allowed to write + it anywhere. + """ + human_fields = list(_human_resolution_candidates()) + already_seeded_card_ids = {fields["card_id"] for fields in human_fields} + hashes = [fields["artbox_phash"] for fields in human_fields] + for batch, _skipped in _join_key_machine_candidates(batch_size, already_seeded_card_ids): + hashes.extend(fields["artbox_phash"] for fields in batch) + return hashes + + +def _hamming_radius_le_2_variants(value: int, bits: int = 64) -> Iterator[int]: + """Every 64-bit integer within Hamming distance <=2 of `value` (2,081 total: itself, each + single-bit flip, and each pair of bit flips) - a generator, not a materialized list, so a + caller checking membership against a small target set (`set.isdisjoint` short-circuits on + the first hit) never pays for the full 2,081 unless no match exists.""" + yield value + for i in range(bits): + yield value ^ (1 << i) + for i in range(bits): + for j in range(i + 1, bits): + yield value ^ (1 << i) ^ (1 << j) + + +@dataclass(frozen=True) +class UnresolvedCoverageResult: + unresolved_candidates_considered: int = 0 + matches_at_d0: int = 0 + matches_at_d_le_2: int = 0 + + +def measure_unresolved_coverage(exemplar_hashes: Iterable[int]) -> UnresolvedCoverageResult: + """ + The honest phase-2 estimate #508 asks for: of every currently-UNRESOLVED card carrying a + CURRENT `artbox_phash`, how many would match the exemplar index at perceptual identity (d=0) + and how many within the established narrowing radius (d<=2 - see `ArtboxPhashExemplar`'s own + docstring on why this radius, not `local_phash.find_best_match`'s unrelated 20/5 cross-source + cutoffs). `exemplar_hashes` is every seeded (or about-to-be-seeded, under `--dry-run`) + `artbox_phash` value - deduplicated into a plain Python `set` once, then checked per-candidate + rather than materializing every exemplar's own 2,081-hash expansion into one giant set: with + tens of thousands of exemplars, expanding the exemplar side once would already cost tens of + millions of entries; expanding each CANDIDATE on demand instead, with an early-exiting + `isdisjoint` check against the small exemplar set, costs at most 2,081 checks per candidate + and typically far fewer once a match is found (Hamming distance is symmetric, so "some + exemplar is within 2 of this candidate" and "this candidate is within 2 of some exemplar" are + the same question). + """ + exemplar_hash_set = set(exemplar_hashes) + # `values_list("artbox_phash", flat=True)` is typed `int | None` at the field level (mypy + # can't see that `_current_artbox_evidence_queryset` already filters artbox_phash__isnull= + # False) - the `if value is not None` filter is what narrows this list back to `list[int]`. + candidate_hashes: list[int] = [ + value + for value in _current_artbox_evidence_queryset() + .filter(card__printing_tag_status=PrintingTagStatus.UNRESOLVED) + .values_list("artbox_phash", flat=True) + if value is not None + ] + matches_at_d0 = sum(1 for candidate_hash in candidate_hashes if candidate_hash in exemplar_hash_set) + matches_at_d_le_2 = sum( + 1 + for candidate_hash in candidate_hashes + if not exemplar_hash_set.isdisjoint(_hamming_radius_le_2_variants(candidate_hash)) + ) + return UnresolvedCoverageResult( + unresolved_candidates_considered=len(candidate_hashes), + matches_at_d0=matches_at_d0, + matches_at_d_le_2=matches_at_d_le_2, + ) + + +__all__ = [ + "JOIN_KEY_ANONYMOUS_ID", + "JOIN_KEY_SEED_CONFIDENCE_FLOOR", + "DEFAULT_BACKFILL_BATCH_SIZE", + "human_resolution_seed_group_key", + "join_key_seed_group_key", + "ArtboxExemplarBackfillResult", + "run_artbox_exemplar_backfill", + "dry_run_candidate_exemplar_hashes", + "UnresolvedCoverageResult", + "measure_unresolved_coverage", +] diff --git a/MPCAutofill/cardpicker/management/commands/backfill_artbox_phash_exemplars.py b/MPCAutofill/cardpicker/management/commands/backfill_artbox_phash_exemplars.py new file mode 100644 index 000000000..1fa3c908a --- /dev/null +++ b/MPCAutofill/cardpicker/management/commands/backfill_artbox_phash_exemplars.py @@ -0,0 +1,106 @@ +from typing import Any + +from django.core.management.base import BaseCommand + +from cardpicker.artbox_exemplar_backfill import ( + DEFAULT_BACKFILL_BATCH_SIZE, + dry_run_candidate_exemplar_hashes, + measure_unresolved_coverage, + run_artbox_exemplar_backfill, +) +from cardpicker.models import ArtboxPhashExemplar + + +class Command(BaseCommand): + help = ( + "Issue #508 phase 1: seeds ArtboxPhashExemplar from our own DB - every current-evidence " + "card with a human-backed printing resolution, plus every card carrying a join-key " + "machine vote at or above the confidence floor (see artbox_exemplar_backfill.py's own " + "JOIN_KEY_SEED_CONFIDENCE_FLOOR comment). Never fetches Scryfall images. Idempotent and " + "resumable by construction (filters out cards that already have an exemplar row, so a " + "plain re-invocation after a kill just picks up where it left off) - no separate " + "--resume flag needed. Also reports the phase-2 coverage estimate: of every currently-" + "UNRESOLVED card carrying a current artbox_phash, how many would match the exemplar " + "index at d=0 and at d<=2 - computed read-only, every invocation, dry-run or not." + ) + + def add_arguments(self, parser: Any) -> None: + parser.add_argument( + "--dry-run", + action="store_true", + default=False, + help="Compute and report every counter, including the coverage estimate, without " + "writing any ArtboxPhashExemplar row.", + ) + parser.add_argument( + "--batch-size", + type=int, + default=DEFAULT_BACKFILL_BATCH_SIZE, + help=f"Rows persisted per bulk_create flush, and rows per join-key evidence-lookup " + f"chunk. Default: {DEFAULT_BACKFILL_BATCH_SIZE}.", + ) + parser.add_argument( + "--limit", + type=int, + default=None, + help="Only create this many exemplar rows total, across both seed passes combined " + "(for testing/sampling). Default: no limit, process the entire backlog.", + ) + parser.add_argument( + "--run-id", + type=str, + default=None, + help="Stamped onto every row this invocation creates, for later retraction via " + "retract_artbox_phash_exemplars --run-id. Default: none stamped.", + ) + parser.add_argument( + "--skip-coverage", + action="store_true", + default=False, + help="Skip the phase-2 coverage measurement pass (unresolved-card matching) - " + "useful for a fast seeding-only run; the coverage pass is a full scan of every " + "UNRESOLVED card's current artbox_phash and is the more expensive half of this " + "command.", + ) + # --skip-checks is deliberately NOT defined here - Django's BaseCommand already adds it + # natively (see local_backfill_canonical_hash.py's own matching comment). + + def handle(self, *args: Any, **kwargs: Any) -> None: + dry_run = kwargs["dry_run"] + batch_size = kwargs["batch_size"] + limit = kwargs["limit"] + run_id = kwargs["run_id"] + skip_coverage = kwargs["skip_coverage"] + + mode = "DRY RUN" if dry_run else "WRITE" + self.stdout.write( + f"[{mode}] backfill_artbox_phash_exemplars --batch-size={batch_size} " f"--limit={limit} --run-id={run_id}" + ) + + result = run_artbox_exemplar_backfill(dry_run=dry_run, batch_size=batch_size, limit=limit, run_id=run_id) + + self.stdout.write( + f"Seeded human_backed={result.human_backed_seeded} machine={result.machine_seeded} " + f"(machine_skipped_stale_or_missing_evidence=" + f"{result.machine_skipped_stale_or_missing_evidence}), " + f"distinct_illustration_ids={result.distinct_illustration_ids}." + ) + self.stdout.write(f"Elapsed {result.elapsed_seconds:.1f}s.") + if dry_run: + self.stdout.write("Dry run - nothing written.") + + if skip_coverage: + self.stdout.write("Coverage measurement skipped (--skip-coverage).") + return + + if dry_run: + exemplar_hashes = dry_run_candidate_exemplar_hashes(batch_size) + else: + exemplar_hashes = list(ArtboxPhashExemplar.objects.values_list("artbox_phash", flat=True)) + + coverage = measure_unresolved_coverage(exemplar_hashes) + self.stdout.write( + f"COVERAGE (phase-2 estimate): unresolved_candidates_considered=" + f"{coverage.unresolved_candidates_considered} matches_at_d0={coverage.matches_at_d0} " + f"matches_at_d_le_2={coverage.matches_at_d_le_2}." + ) diff --git a/MPCAutofill/cardpicker/management/commands/retract_artbox_phash_exemplars.py b/MPCAutofill/cardpicker/management/commands/retract_artbox_phash_exemplars.py new file mode 100644 index 000000000..7a1c14307 --- /dev/null +++ b/MPCAutofill/cardpicker/management/commands/retract_artbox_phash_exemplars.py @@ -0,0 +1,93 @@ +from typing import Any + +from django.core.management.base import BaseCommand, CommandError + +from cardpicker.models import ArtboxPhashExemplar + + +class Command(BaseCommand): + help = ( + "Issue #508 phase 1: retracts ArtboxPhashExemplar rows. Nothing downstream reads this " + "table yet (no matching calculator, no vote), so retraction here is a plain filtered " + "delete - no consensus resync, no safety gate against a live consensus outcome (compare " + "retract_stage_d_by_run_id, which needs both because its target table casts votes). " + "--seed-group-key is the primary path (ArtboxPhashExemplar's own docstring): every " + "exemplar traced to the SAME source resolution or the SAME source join-key vote shares " + "one seed_group_key, so retracting a bad seed 'together with everything it seeded' is " + "exactly this filter. --card-id/--illustration-id/--run-id/--source-vote-id are narrower " + "or broader alternatives for a caller who already knows one of those instead. Exactly one " + "selector is required per invocation; combining more than one is a CommandError, not a " + "silent AND, since a caller reaching for two selectors at once is very likely trying to " + "express something this command doesn't support rather than a genuine narrowing. Dry-run " + "by default - --write required to actually delete anything." + ) + + def add_arguments(self, parser: Any) -> None: + parser.add_argument( + "--seed-group-key", type=str, default=None, help="Retract every exemplar sharing this seed_group_key." + ) + parser.add_argument( + "--card-id", type=int, default=None, help="Retract the single exemplar seeded from this card." + ) + parser.add_argument( + "--illustration-id", + type=str, + default=None, + help="Retract every exemplar pointing at this illustration_id (UUID).", + ) + parser.add_argument( + "--run-id", + type=str, + default=None, + help="Retract every exemplar stamped with this backfill run_id " + "(backfill_artbox_phash_exemplars --run-id).", + ) + parser.add_argument( + "--source-vote-id", + type=int, + default=None, + help="Retract the single exemplar seeded from this CardPrintingTag vote (JOIN_KEY_MACHINE seeds only).", + ) + parser.add_argument( + "--write", + action="store_true", + default=False, + help="Actually delete the matched rows. Default is dry-run: report the count and " + "the affected illustration_ids without deleting anything.", + ) + + def handle(self, *args: Any, **kwargs: Any) -> None: + selectors = { + "seed_group_key": kwargs["seed_group_key"], + "card_id": kwargs["card_id"], + "illustration_id": kwargs["illustration_id"], + "run_id": kwargs["run_id"], + "source_vote_id": kwargs["source_vote_id"], + } + given = {key: value for key, value in selectors.items() if value is not None} + if len(given) != 1: + raise CommandError( + "Exactly one of --seed-group-key/--card-id/--illustration-id/--run-id/" + f"--source-vote-id is required (got {len(given)}: {sorted(given)})." + ) + + write = kwargs["write"] + mode = "WRITE" if write else "DRY RUN" + + queryset = ArtboxPhashExemplar.objects.filter(**given) + count = queryset.count() + illustration_ids = sorted({str(value) for value in queryset.values_list("illustration_id", flat=True)}) + + self.stdout.write(f"[{mode}] retract_artbox_phash_exemplars {given}") + self.stdout.write( + f"Matched {count} exemplar row(s) across {len(illustration_ids)} distinct illustration_id(s)" + f"{': ' + ', '.join(illustration_ids[:20]) if illustration_ids else ''}" + f"{' (truncated)' if len(illustration_ids) > 20 else ''}." + ) + + if not write: + self.stdout.write("Dry run - nothing deleted.") + return + + deleted_count, _ = queryset.delete() + self.stdout.write(f"Deleted {deleted_count} row(s).") diff --git a/MPCAutofill/cardpicker/migrations/0102_artbox_phash_exemplar.py b/MPCAutofill/cardpicker/migrations/0102_artbox_phash_exemplar.py new file mode 100644 index 000000000..79a8d1905 --- /dev/null +++ b/MPCAutofill/cardpicker/migrations/0102_artbox_phash_exemplar.py @@ -0,0 +1,75 @@ +# Generated by Django 4.2.30 on 2026-08-05 02:13 + +import django.db.models.deletion +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ("cardpicker", "0101_delete_printingtagvote"), + ] + + operations = [ + migrations.CreateModel( + name="ArtboxPhashExemplar", + fields=[ + ("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")), + ("illustration_id", models.UUIDField(db_index=True)), + ("artbox_phash", models.BigIntegerField(db_index=True)), + ( + "seed_kind", + models.CharField( + choices=[ + ("human_resolution", "Human-backed printing resolution"), + ("join_key_machine", "High-confidence join-key vote"), + ], + max_length=32, + ), + ), + ("is_human_backed", models.BooleanField()), + ("confidence", models.FloatField(blank=True, null=True)), + ("seed_group_key", models.CharField(db_index=True, max_length=128)), + ("content_hash", models.BigIntegerField()), + ("run_id", models.CharField(blank=True, db_index=True, max_length=64, null=True)), + ("created_at", models.DateTimeField(auto_now_add=True)), + ( + "card", + models.OneToOneField( + on_delete=django.db.models.deletion.CASCADE, + related_name="artbox_phash_exemplar", + to="cardpicker.card", + ), + ), + ( + "printing", + models.ForeignKey( + on_delete=django.db.models.deletion.CASCADE, + related_name="artbox_phash_exemplars", + to="cardpicker.canonicalcard", + ), + ), + ( + "source_vote", + models.ForeignKey( + blank=True, + null=True, + on_delete=django.db.models.deletion.SET_NULL, + related_name="+", + to="cardpicker.cardprintingtag", + ), + ), + ], + ), + migrations.AddConstraint( + model_name="artboxphashexemplar", + constraint=models.CheckConstraint( + check=models.Q( + models.Q(("is_human_backed", True), ("seed_kind", "human_resolution")), + models.Q(("is_human_backed", False), ("seed_kind", "join_key_machine")), + _connector="OR", + ), + name="artboxphashexemplar_seed_kind_matches_human_backed", + ), + ), + ] diff --git a/MPCAutofill/cardpicker/models.py b/MPCAutofill/cardpicker/models.py index 623e3ad90..f41447013 100755 --- a/MPCAutofill/cardpicker/models.py +++ b/MPCAutofill/cardpicker/models.py @@ -1390,6 +1390,115 @@ def __str__(self) -> str: return f"[{self.source}] {self.card.name} -> illustration {outcome}" +class ArtboxPhashExemplarSeedKind(models.TextChoices): + """ + How an `ArtboxPhashExemplar` row was seeded (issue #508 phase 1) - the provenance distinction + the owner made mandatory at seeding time (2026-08-05): a machine-derived seed and a + human-backed one must stay distinguishable forever, so a later decision to trust only the + latter is a query over this field, not a migration. + """ + + HUMAN_RESOLUTION = "human_resolution", gettext_lazy("Human-backed printing resolution") + JOIN_KEY_MACHINE = "join_key_machine", gettext_lazy("High-confidence join-key vote") + + +class ArtboxPhashExemplar(models.Model): + """ + Second illustration-deduction path's reference index (issue #508 phase 1, "self-referential + exemplar index" - owner-shaped 2026-07-28, seeding extended by the owner 2026-08-05 to include + machine seeds). An exemplar is a labelled association: a card's own CURRENT `artbox_phash` + (`ImageEvidence.artbox_phash` - see that field's own docstring, issue #480) -> the + `illustration_id` of the printing that scan was identified as (via + `CanonicalPrintingMetadata.illustration_id`, reached through the resolved/matched + `CanonicalCard`). + + NEVER SOURCED FROM SCRYFALL IMAGES (binding, #508's design section). phash comparability + requires identical crop geometry/preprocessing - our own `artbox_phash` extractor is + self-consistent; Scryfall's `art_crop` framing differs and would make cross-source Hamming + distances unreliable (this is also why PR #694 was closed and deferred to #697 - do not + reintroduce a Scryfall fetch anywhere a seed for this table is computed). + + SEED SOURCES (owner decision 2026-08-05, extending #508's original human-only spec, which + would have left this index dormant - only 12 human-backed resolutions exist catalogue-wide + at seeding time): + + - `HUMAN_RESOLUTION`: `card.printing_tag_status == RESOLVED`. Resolution ALWAYS requires a + human-backed vote (`vote_consensus.resolve_weighted_consensus`'s non-machine-alone gate, + untouched by this work) - so every RESOLVED card is human-backed by construction, and no + per-vote inspection is needed to classify one as such. + - `JOIN_KEY_MACHINE`: an individual `CardPrintingTag` vote cast by the join-key calculator + (`local_calculate_verdicts.JOIN_KEY_ANONYMOUS_ID`) at or above + `artbox_exemplar_backfill.JOIN_KEY_SEED_CONFIDENCE_FLOOR` - see that constant's own comment + for why the floor excludes the artist-disagreement confidence tier (0.65) along with the + no-match tier (0.6, which is not an identification at all and can never seed regardless of + any floor). + + `is_human_backed` is a plain denormalised copy of `seed_kind`'s own implication (never + `HUMAN_RESOLUTION` with `is_human_backed=False` or vice versa - enforced by the CheckConstraint + below), kept as its own column so a reader who only needs the human/machine split never has to + know the seed-kind vocabulary. + + RETRACTION (owner directive 2026-08-05: "a bad seed must be retractable together with + everything it seeded"). `seed_group_key` is the stable identity of the SOURCE EVENT that + produced this row, not of the row itself: every exemplar traceable to the same md5-identity- + group resolution, or to the same source `CardPrintingTag` vote, shares one key, so retracting + a bad seed is `ArtboxPhashExemplar.objects.filter(seed_group_key=...).delete()` - one query, no + per-row reasoning about what else that source touched. See `artbox_exemplar_backfill. + human_resolution_seed_group_key`/`join_key_seed_group_key` for the exact format (the human- + resolution case mirrors `printing_consensus.md5_group_key`'s own group identity, so retracting + "this resolved identity group" here means the same set of cards `printing_consensus` itself + would call one group). `source_vote` is `SET_NULL` on the vote's own deletion (a purge doesn't + orphan this row's retractability - `seed_group_key` carries it independently of the FK's + referential integrity). + + INDEX-NOT-STORE (CLAUDE.md's governing premise): this table holds a hash and a UUID, nothing + fetched or decodable back into pixels - `content_hash` records the source card's own + `content_phash` AT SEED TIME purely as a staleness audit trail (so a later reader can tell + whether the source card's image has since changed), never a second copy of anything + image-shaped. + + PHASE 1 SCOPE: this table is read by nothing yet. No matching calculator, no vote, no + consensus, no change to `resolve_weighted_consensus`/the human-backed gate - see + `docs/identification-pipeline.md`'s "Parallel detectors" section for what this deliberately + does NOT do. + """ + + illustration_id = models.UUIDField(db_index=True) + artbox_phash = models.BigIntegerField(db_index=True) + card = models.OneToOneField(to=Card, on_delete=models.CASCADE, related_name="artbox_phash_exemplar") + printing = models.ForeignKey(to=CanonicalCard, on_delete=models.CASCADE, related_name="artbox_phash_exemplars") + seed_kind = models.CharField(max_length=32, choices=ArtboxPhashExemplarSeedKind.choices) + is_human_backed = models.BooleanField() + # SET_NULL, not CASCADE - see class docstring's RETRACTION section for why losing this FK + # on the source vote's own deletion is fine (seed_group_key carries retractability instead). + source_vote = models.ForeignKey( + to=CardPrintingTag, on_delete=models.SET_NULL, null=True, blank=True, related_name="+" + ) + # Purely informational, mirroring `CardPrintingTag.confidence`'s own "not read by any + # resolution math" convention (`JOIN_KEY_CONFIDENCE_BOTH`'s comment in + # local_calculate_verdicts.py makes the identical point for that field). Null for + # HUMAN_RESOLUTION seeds - a resolution is a consensus outcome, not a single confidence value. + confidence = models.FloatField(null=True, blank=True) + seed_group_key = models.CharField(max_length=128, db_index=True) + content_hash = models.BigIntegerField() + run_id = models.CharField(max_length=64, null=True, blank=True, db_index=True) + created_at = models.DateTimeField(auto_now_add=True) + + class Meta: + constraints = [ + models.CheckConstraint( + check=( + models.Q(seed_kind=ArtboxPhashExemplarSeedKind.HUMAN_RESOLUTION, is_human_backed=True) + | models.Q(seed_kind=ArtboxPhashExemplarSeedKind.JOIN_KEY_MACHINE, is_human_backed=False) + ), + name="artboxphashexemplar_seed_kind_matches_human_backed", + ), + ] + + def __str__(self) -> str: + return f"[{self.seed_kind}] card={self.card_id} -> illustration {self.illustration_id}" + + class TagModerationClass(models.TextChoices): """ Whether consensus on this tag resolves like any other (STANDARD) or requires a privileged diff --git a/MPCAutofill/cardpicker/tests/test_artbox_exemplar_backfill.py b/MPCAutofill/cardpicker/tests/test_artbox_exemplar_backfill.py new file mode 100644 index 000000000..af33c6614 --- /dev/null +++ b/MPCAutofill/cardpicker/tests/test_artbox_exemplar_backfill.py @@ -0,0 +1,473 @@ +""" +Tests for cardpicker.artbox_exemplar_backfill (issue #508 phase 1) and the model/management +commands it backs: ArtboxPhashExemplar's own constraints, the two seed passes (human-backed +resolution, join-key machine), the confidence floor, staleness exclusion, idempotent resume, +retraction, and the coverage measurement's Hamming-distance matching. +""" + +import uuid +from typing import Any + +import pytest + +from django.core.management import call_command +from django.db import IntegrityError, transaction + +from cardpicker.artbox_exemplar_backfill import ( + JOIN_KEY_ANONYMOUS_ID, + JOIN_KEY_SEED_CONFIDENCE_FLOOR, + dry_run_candidate_exemplar_hashes, + human_resolution_seed_group_key, + join_key_seed_group_key, + measure_unresolved_coverage, + run_artbox_exemplar_backfill, +) +from cardpicker.models import ( + ArtboxPhashExemplar, + ArtboxPhashExemplarSeedKind, + PrintingTagStatus, +) +from cardpicker.tests.factories import ( + CanonicalCardFactory, + CanonicalPrintingMetadataFactory, + CardFactory, + CardPrintingTagFactory, + ImageEvidenceFactory, +) + + +def _resolved_card_with_evidence(illustration_id: Any = None, artbox_phash: int = 111, content_phash: int = 111): + """A card whose printing_tag_status is RESOLVED, resolved to a printing carrying + `illustration_id` (a fresh uuid4 if not given), with a CURRENT ImageEvidence row carrying + `artbox_phash`.""" + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=illustration_id or uuid.uuid4()) + card = CardFactory( + content_phash=content_phash, + printing_tag_status=PrintingTagStatus.RESOLVED, + inferred_canonical_card=printing, + ) + ImageEvidenceFactory(card=card, content_hash=content_phash, artbox_phash=artbox_phash) + return card, printing + + +def _join_key_vote_with_evidence( + confidence: float, artbox_phash: int = 222, content_phash: int = 222, is_no_match: bool = False +): + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=uuid.uuid4()) + card = CardFactory(content_phash=content_phash) + ImageEvidenceFactory(card=card, content_hash=content_phash, artbox_phash=artbox_phash) + vote = CardPrintingTagFactory( + card=card, + printing=None if is_no_match else printing, + is_no_match=is_no_match, + anonymous_id=JOIN_KEY_ANONYMOUS_ID, + confidence=confidence, + ) + return card, printing, vote + + +class TestModelConstraints: + def test_one_exemplar_per_card_enforced(self, db: Any) -> None: + card, printing = _resolved_card_with_evidence() + ArtboxPhashExemplar.objects.create( + illustration_id=printing.printing_metadata.illustration_id, + artbox_phash=111, + card=card, + printing=printing, + seed_kind=ArtboxPhashExemplarSeedKind.HUMAN_RESOLUTION, + is_human_backed=True, + seed_group_key=human_resolution_seed_group_key(card), + content_hash=111, + ) + with pytest.raises(IntegrityError): + with transaction.atomic(): + ArtboxPhashExemplar.objects.create( + illustration_id=uuid.uuid4(), + artbox_phash=222, + card=card, + printing=printing, + seed_kind=ArtboxPhashExemplarSeedKind.HUMAN_RESOLUTION, + is_human_backed=True, + seed_group_key=human_resolution_seed_group_key(card), + content_hash=111, + ) + + def test_seed_kind_must_match_is_human_backed(self, db: Any) -> None: + card, printing = _resolved_card_with_evidence() + with pytest.raises(IntegrityError): + with transaction.atomic(): + ArtboxPhashExemplar.objects.create( + illustration_id=printing.printing_metadata.illustration_id, + artbox_phash=111, + card=card, + printing=printing, + seed_kind=ArtboxPhashExemplarSeedKind.HUMAN_RESOLUTION, + is_human_backed=False, # mismatched - must be True for HUMAN_RESOLUTION + seed_group_key=human_resolution_seed_group_key(card), + content_hash=111, + ) + + +class TestSeedGroupKey: + def test_human_resolution_key_shares_across_md5_group(self, db: Any) -> None: + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=uuid.uuid4()) + card_a = CardFactory(md5_checksum="same-md5", content_phash=1, printing_tag_status=PrintingTagStatus.RESOLVED) + card_b = CardFactory(md5_checksum="same-md5", content_phash=2, printing_tag_status=PrintingTagStatus.RESOLVED) + + assert human_resolution_seed_group_key(card_a) == human_resolution_seed_group_key(card_b) + + def test_human_resolution_key_differs_for_checksum_less_cards(self, db: Any) -> None: + card_a = CardFactory(md5_checksum=None) + card_b = CardFactory(md5_checksum=None) + + assert human_resolution_seed_group_key(card_a) != human_resolution_seed_group_key(card_b) + + def test_join_key_group_key_is_per_vote(self, db: Any) -> None: + assert join_key_seed_group_key(1) != join_key_seed_group_key(2) + assert join_key_seed_group_key(1) == join_key_seed_group_key(1) + + +class TestHumanResolutionSeeding: + def test_resolved_card_with_current_evidence_is_seeded(self, db: Any) -> None: + illustration_id = uuid.uuid4() + card, printing = _resolved_card_with_evidence(illustration_id=illustration_id, artbox_phash=333) + + result = run_artbox_exemplar_backfill() + + assert result.human_backed_seeded == 1 + assert result.machine_seeded == 0 + exemplar = ArtboxPhashExemplar.objects.get(card=card) + assert exemplar.illustration_id == illustration_id + assert exemplar.artbox_phash == 333 + assert exemplar.seed_kind == ArtboxPhashExemplarSeedKind.HUMAN_RESOLUTION + assert exemplar.is_human_backed is True + assert exemplar.confidence is None + assert exemplar.source_vote_id is None + + def test_printing_with_no_illustration_id_is_not_seeded(self, db: Any) -> None: + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=None) + card = CardFactory( + content_phash=444, printing_tag_status=PrintingTagStatus.RESOLVED, inferred_canonical_card=printing + ) + ImageEvidenceFactory(card=card, content_hash=444, artbox_phash=444) + + result = run_artbox_exemplar_backfill() + + assert result.human_backed_seeded == 0 + assert not ArtboxPhashExemplar.objects.filter(card=card).exists() + + def test_unresolved_card_is_not_seeded(self, db: Any) -> None: + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=uuid.uuid4()) + card = CardFactory(content_phash=555, printing_tag_status=PrintingTagStatus.UNRESOLVED) + ImageEvidenceFactory(card=card, content_hash=555, artbox_phash=555) + + result = run_artbox_exemplar_backfill() + + assert result.human_backed_seeded == 0 + assert not ArtboxPhashExemplar.objects.filter(card=card).exists() + + def test_stale_evidence_cannot_seed(self, db: Any) -> None: + """content_hash on the ImageEvidence row disagrees with the card's LIVE content_phash - + the row is stale (an old image's evidence) and must never seed an exemplar.""" + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=uuid.uuid4()) + card = CardFactory( + content_phash=999, printing_tag_status=PrintingTagStatus.RESOLVED, inferred_canonical_card=printing + ) + ImageEvidenceFactory(card=card, content_hash=111, artbox_phash=111) # stale: != card.content_phash + + result = run_artbox_exemplar_backfill() + + assert result.human_backed_seeded == 0 + assert not ArtboxPhashExemplar.objects.filter(card=card).exists() + + def test_no_artbox_phash_is_not_seeded(self, db: Any) -> None: + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=uuid.uuid4()) + card = CardFactory( + content_phash=222, printing_tag_status=PrintingTagStatus.RESOLVED, inferred_canonical_card=printing + ) + ImageEvidenceFactory(card=card, content_hash=222, artbox_phash=None) + + result = run_artbox_exemplar_backfill() + + assert result.human_backed_seeded == 0 + + def test_second_invocation_only_seeds_what_the_first_missed(self, db: Any) -> None: + _resolved_card_with_evidence(artbox_phash=1) + card_b, _ = _resolved_card_with_evidence(artbox_phash=2, content_phash=2) + + first = run_artbox_exemplar_backfill() + assert first.human_backed_seeded == 2 + + second = run_artbox_exemplar_backfill() + assert second.human_backed_seeded == 0 + assert ArtboxPhashExemplar.objects.count() == 2 + + def test_dry_run_writes_nothing(self, db: Any) -> None: + _resolved_card_with_evidence() + + result = run_artbox_exemplar_backfill(dry_run=True) + + assert result.human_backed_seeded == 1 + assert ArtboxPhashExemplar.objects.count() == 0 + + +class TestJoinKeyMachineSeeding: + def test_vote_at_both_confidence_is_seeded(self, db: Any) -> None: + card, printing, vote = _join_key_vote_with_evidence(confidence=0.85, artbox_phash=1) + + result = run_artbox_exemplar_backfill() + + assert result.machine_seeded == 1 + exemplar = ArtboxPhashExemplar.objects.get(card=card) + assert exemplar.seed_kind == ArtboxPhashExemplarSeedKind.JOIN_KEY_MACHINE + assert exemplar.is_human_backed is False + assert exemplar.confidence == 0.85 + assert exemplar.source_vote_id == vote.pk + + def test_vote_at_collector_only_confidence_is_seeded(self, db: Any) -> None: + _join_key_vote_with_evidence(confidence=JOIN_KEY_SEED_CONFIDENCE_FLOOR, artbox_phash=2) + + result = run_artbox_exemplar_backfill() + + assert result.machine_seeded == 1 + + def test_vote_below_floor_is_not_seeded(self, db: Any) -> None: + card, _, _ = _join_key_vote_with_evidence(confidence=0.65, artbox_phash=3) + + result = run_artbox_exemplar_backfill() + + assert result.machine_seeded == 0 + assert not ArtboxPhashExemplar.objects.filter(card=card).exists() + + def test_no_match_vote_never_seeds_regardless_of_confidence(self, db: Any) -> None: + card, _, _ = _join_key_vote_with_evidence(confidence=0.6, artbox_phash=4, is_no_match=True) + + result = run_artbox_exemplar_backfill() + + assert result.machine_seeded == 0 + assert not ArtboxPhashExemplar.objects.filter(card=card).exists() + + def test_stale_evidence_is_skipped_and_counted(self, db: Any) -> None: + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=uuid.uuid4()) + card = CardFactory(content_phash=777) + ImageEvidenceFactory(card=card, content_hash=111, artbox_phash=111) # stale + CardPrintingTagFactory(card=card, printing=printing, anonymous_id=JOIN_KEY_ANONYMOUS_ID, confidence=0.85) + + result = run_artbox_exemplar_backfill() + + assert result.machine_seeded == 0 + assert result.machine_skipped_stale_or_missing_evidence == 1 + + def test_card_already_human_resolved_is_not_double_seeded_by_machine_pass(self, db: Any) -> None: + illustration_id = uuid.uuid4() + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=illustration_id) + card = CardFactory( + content_phash=555, printing_tag_status=PrintingTagStatus.RESOLVED, inferred_canonical_card=printing + ) + ImageEvidenceFactory(card=card, content_hash=555, artbox_phash=555) + CardPrintingTagFactory(card=card, printing=printing, anonymous_id=JOIN_KEY_ANONYMOUS_ID, confidence=0.85) + + result = run_artbox_exemplar_backfill() + + assert result.human_backed_seeded == 1 + assert result.machine_seeded == 0 + exemplar = ArtboxPhashExemplar.objects.get(card=card) + assert exemplar.seed_kind == ArtboxPhashExemplarSeedKind.HUMAN_RESOLUTION + + def test_dry_run_does_not_double_count_human_and_machine_for_same_card(self, db: Any) -> None: + illustration_id = uuid.uuid4() + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=illustration_id) + card = CardFactory( + content_phash=666, printing_tag_status=PrintingTagStatus.RESOLVED, inferred_canonical_card=printing + ) + ImageEvidenceFactory(card=card, content_hash=666, artbox_phash=666) + CardPrintingTagFactory(card=card, printing=printing, anonymous_id=JOIN_KEY_ANONYMOUS_ID, confidence=0.85) + + result = run_artbox_exemplar_backfill(dry_run=True) + + assert result.human_backed_seeded == 1 + assert result.machine_seeded == 0 + + def test_second_invocation_only_seeds_what_the_first_missed(self, db: Any) -> None: + _join_key_vote_with_evidence(confidence=0.85, artbox_phash=1) + card_b, printing_b, vote_b = _join_key_vote_with_evidence(confidence=0.85, artbox_phash=2, content_phash=2) + + first = run_artbox_exemplar_backfill() + assert first.machine_seeded == 2 + + second = run_artbox_exemplar_backfill() + assert second.machine_seeded == 0 + + +class TestDistinguishability: + def test_human_and_machine_exemplars_are_queryable_separately(self, db: Any) -> None: + _resolved_card_with_evidence(artbox_phash=1) + _join_key_vote_with_evidence(confidence=0.85, artbox_phash=2) + + run_artbox_exemplar_backfill() + + assert ArtboxPhashExemplar.objects.filter(is_human_backed=True).count() == 1 + assert ArtboxPhashExemplar.objects.filter(is_human_backed=False).count() == 1 + assert ArtboxPhashExemplar.objects.filter(seed_kind=ArtboxPhashExemplarSeedKind.HUMAN_RESOLUTION).count() == 1 + assert ArtboxPhashExemplar.objects.filter(seed_kind=ArtboxPhashExemplarSeedKind.JOIN_KEY_MACHINE).count() == 1 + + +class TestRunIdStamping: + def test_run_id_is_stamped_on_created_rows(self, db: Any) -> None: + _resolved_card_with_evidence() + _join_key_vote_with_evidence(confidence=0.85, artbox_phash=2) + + run_artbox_exemplar_backfill(run_id="run-abc") + + assert ArtboxPhashExemplar.objects.filter(run_id="run-abc").count() == 2 + + +class TestRetraction: + def test_retract_by_seed_group_key_removes_every_group_member(self, db: Any) -> None: + illustration_id = uuid.uuid4() + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=illustration_id) + card_a = CardFactory( + md5_checksum="grp-1", + content_phash=1, + printing_tag_status=PrintingTagStatus.RESOLVED, + inferred_canonical_card=printing, + ) + card_b = CardFactory( + md5_checksum="grp-1", + content_phash=2, + printing_tag_status=PrintingTagStatus.RESOLVED, + inferred_canonical_card=printing, + ) + ImageEvidenceFactory(card=card_a, content_hash=1, artbox_phash=1) + ImageEvidenceFactory(card=card_b, content_hash=2, artbox_phash=2) + + run_artbox_exemplar_backfill() + assert ArtboxPhashExemplar.objects.count() == 2 + seed_group_key = ArtboxPhashExemplar.objects.get(card=card_a).seed_group_key + assert seed_group_key == ArtboxPhashExemplar.objects.get(card=card_b).seed_group_key + + call_command("retract_artbox_phash_exemplars", f"--seed-group-key={seed_group_key}", "--write") + + assert ArtboxPhashExemplar.objects.count() == 0 + + def test_retract_by_source_vote_id_removes_only_that_exemplar(self, db: Any) -> None: + card_a, _, vote_a = _join_key_vote_with_evidence(confidence=0.85, artbox_phash=1) + card_b, _, vote_b = _join_key_vote_with_evidence(confidence=0.85, artbox_phash=2, content_phash=2) + + run_artbox_exemplar_backfill() + assert ArtboxPhashExemplar.objects.count() == 2 + + call_command("retract_artbox_phash_exemplars", f"--source-vote-id={vote_a.pk}", "--write") + + assert ArtboxPhashExemplar.objects.count() == 1 + assert ArtboxPhashExemplar.objects.filter(card=card_b).exists() + assert not ArtboxPhashExemplar.objects.filter(card=card_a).exists() + + def test_dry_run_deletes_nothing(self, db: Any) -> None: + card, printing = _resolved_card_with_evidence() + run_artbox_exemplar_backfill() + seed_group_key = ArtboxPhashExemplar.objects.get(card=card).seed_group_key + + call_command("retract_artbox_phash_exemplars", f"--seed-group-key={seed_group_key}") + + assert ArtboxPhashExemplar.objects.count() == 1 + + def test_requires_exactly_one_selector(self, db: Any) -> None: + from django.core.management.base import CommandError + + with pytest.raises(CommandError): + call_command("retract_artbox_phash_exemplars", "--write") + + with pytest.raises(CommandError): + call_command( + "retract_artbox_phash_exemplars", "--card-id=1", "--illustration-id=" + str(uuid.uuid4()), "--write" + ) + + +class TestCoverageMeasurement: + def test_exact_match_counts_as_both_d0_and_d_le_2(self, db: Any) -> None: + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=uuid.uuid4()) + card = CardFactory(content_phash=42, printing_tag_status=PrintingTagStatus.UNRESOLVED) + ImageEvidenceFactory(card=card, content_hash=42, artbox_phash=42) + + result = measure_unresolved_coverage([42]) + + assert result.unresolved_candidates_considered == 1 + assert result.matches_at_d0 == 1 + assert result.matches_at_d_le_2 == 1 + + def test_distance_one_and_two_count_only_toward_d_le_2(self, db: Any) -> None: + base = 0 + distance_one = base ^ (1 << 3) + distance_two = base ^ (1 << 3) ^ (1 << 7) + distance_three = base ^ (1 << 3) ^ (1 << 7) ^ (1 << 11) + + for offset, value in enumerate([distance_one, distance_two, distance_three]): + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=uuid.uuid4()) + card = CardFactory(content_phash=100 + offset, printing_tag_status=PrintingTagStatus.UNRESOLVED) + ImageEvidenceFactory(card=card, content_hash=100 + offset, artbox_phash=value) + + result = measure_unresolved_coverage([base]) + + assert result.unresolved_candidates_considered == 3 + assert result.matches_at_d0 == 0 + assert result.matches_at_d_le_2 == 2 # distance_one and distance_two, not distance_three + + def test_resolved_and_no_match_cards_are_excluded_from_candidates(self, db: Any) -> None: + for status, offset in [(PrintingTagStatus.RESOLVED, 0), (PrintingTagStatus.NO_MATCH, 1)]: + printing = CanonicalCardFactory() + CanonicalPrintingMetadataFactory(canonical_card=printing, illustration_id=uuid.uuid4()) + card = CardFactory(content_phash=200 + offset, printing_tag_status=status) + ImageEvidenceFactory(card=card, content_hash=200 + offset, artbox_phash=200 + offset) + + result = measure_unresolved_coverage([200, 201]) + + assert result.unresolved_candidates_considered == 0 + assert result.matches_at_d0 == 0 + + def test_dry_run_candidate_exemplar_hashes_matches_what_a_real_run_would_seed(self, db: Any) -> None: + _resolved_card_with_evidence(artbox_phash=1) + _join_key_vote_with_evidence(confidence=0.85, artbox_phash=2) + + dry_run_hashes = sorted(dry_run_candidate_exemplar_hashes()) + + assert dry_run_hashes == [1, 2] + + +class TestBackfillCommandCLI: + def test_real_cli_invocation_with_no_flags(self, db: Any) -> None: + _resolved_card_with_evidence() + + call_command("backfill_artbox_phash_exemplars", "--skip-checks", "--skip-coverage") + + assert ArtboxPhashExemplar.objects.count() == 1 + + def test_dry_run_flag_reports_without_writing(self, db: Any, capsys: Any) -> None: + _resolved_card_with_evidence() + + call_command("backfill_artbox_phash_exemplars", "--skip-checks", "--dry-run", "--skip-coverage") + captured = capsys.readouterr() + + assert "Dry run - nothing written." in captured.out + assert ArtboxPhashExemplar.objects.count() == 0 + + def test_coverage_measurement_runs_by_default(self, db: Any, capsys: Any) -> None: + _resolved_card_with_evidence() + + call_command("backfill_artbox_phash_exemplars", "--skip-checks") + captured = capsys.readouterr() + + assert "COVERAGE (phase-2 estimate)" in captured.out diff --git a/docs/identification-pipeline.md b/docs/identification-pipeline.md index a0873fcf1..337b373fe 100644 --- a/docs/identification-pipeline.md +++ b/docs/identification-pipeline.md @@ -394,6 +394,33 @@ printing votes, so it would have fired on the first `-v2` run. votes (ordinary consensus since #292). Detect-and-tag only. - **"Marked as proxy"** (#291, planned): marker presence → tag; **absence** → moderation flag, batched by source (the counterfeit-risk framing). +- **Artbox-phash exemplar index** (#508 phase 1, `artbox_exemplar_backfill.py`, + `ArtboxPhashExemplar`): a second, OCR-free illustration-deduction path, + built but not yet wired to anything. A card's own `artbox_phash` (the + evidence-only extractor from #480) is labelled with the `illustration_id` + of the printing that scan was identified as, sourced two ways — a + human-backed printing resolution (`printing_tag_status == RESOLVED`, + always human-backed by the g5 gate above), or an unopposed join-key + machine vote at 0.75/0.85 confidence (the artist-disagreement tier, 0.65, + and the no-match tier, 0.6, are both excluded — a wrong exemplar + propagates to every card it later matches). Every row records its own + seed kind and confidence, human and machine seeds stay permanently + distinguishable, and a bad seed retracts together with everything it + seeded via a shared `seed_group_key` (one md5-identity-group resolution, + or one source vote). **Never sourced from Scryfall images** — phash + comparability needs the same crop geometry our own extractor guarantees + and Scryfall's `art_crop` framing does not. + + **What this deliberately does NOT do, in phase 1**: it does not compare + any unresolved card against the index, does not resolve anything, does + not cast an illustration or printing vote, and does not touch + `resolve_weighted_consensus` or the human-backed gate. The index exists + and is measured; nothing reads it yet. A phase 2 matching calculator + (comparing unresolved scans against this index at d=0/d≤2 and reusing the + illustration→printings vote logic the design docstring in + `ArtboxPhashExemplar` describes) is gated on the coverage this index + actually measures once seeded, the same measurement-gate discipline + #693 established. ## Why a bad identification is hard