Skip to content

Commit 08da4ad

Browse files
Declare local_phash's skip reasons at source (protected-core exception)
`local_phash.find_best_match` returned "no-hashable-candidates" and "no-clear-winner" as bare inline literals. The roster tether added in #567 derives the skip-reason roster from module-level `*_SKIP_REASON = "<literal>"` declarations, and it cannot enumerate literals it cannot see - so a NEW literal added inside `find_best_match` would have reached `CardScanLog.skip_reason` (~2.7M rows, no `choices` list, no FK) with nothing to catch it. #567 could not close that: `local_phash.py` is PROTECTED CORE (docs/upstreaming/license-provenance.md §2). It mirrored the two constants in the consuming module instead and documented the residual gap. The owner granted a narrow exception on 2026-07-29 to close it. - declare PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON and PHASH_NO_CLEAR_WINNER_SKIP_REASON in `local_phash.py`, export them, and return them by name - delete the mirror in `local_identify_printing_tags.py`, which now imports the one constant it uses; one declaration per value - record the exception, its reasoning and - the part that matters - its LIMITS in license-provenance.md §2.1, a new exception log. It permits declaring skip-reason constants in this one file. It is not a licence to edit protected core; the next such change needs its own ruling. `local_phash.py` stays on the protected list. - two guards in test_skip_reason_roster.py for the regressions the tether is structurally blind to: the mirror coming back, and a bare literal returning at the origin Naming-only. The string VALUES are untouched - and so are the constant NAMES, so the roster's pinning test and the doc's Constant column needed no edit at all. A `CardScanLog` row written after this is byte-identical to one written before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
1 parent 3df1322 commit 08da4ad

5 files changed

Lines changed: 248 additions & 42 deletions

File tree

MPCAutofill/cardpicker/local_identify_printing_tags.py

Lines changed: 17 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,7 @@
8282
local_phash,
8383
)
8484
from cardpicker.local_fallback import FALLBACK_ANONYMOUS_ID
85+
from cardpicker.local_phash import PHASH_NO_CLEAR_WINNER_SKIP_REASON
8586
from cardpicker.models import (
8687
CanonicalCard,
8788
Card,
@@ -167,15 +168,18 @@
167168

168169
# The phash engine's own outcomes (`run_phash_for_card`).
169170
PHASH_TOO_MANY_CANDIDATES_SKIP_REASON = "too-many-candidates"
170-
# `local_phash.find_best_match` is PROTECTED CORE (docs/upstreaming/license-provenance.md §2) and
171-
# returns these two strings as its own inline literals; they cannot be declared at their true
172-
# source without editing a protected file. They are MIRRORED here - the one roster entry whose
173-
# declaration is not co-located with its origin - and used for the equality test below so the
174-
# coupling is at least named rather than anonymous. NO_CLEAR_WINNER is never written to
175-
# `CardScanLog` by this module today (`_classify_no_clear_winner` always refines it into one of
176-
# the two variants below); HISTORICAL rows predating that refinement still carry it.
177-
PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON = "no-hashable-candidates"
178-
PHASH_NO_CLEAR_WINNER_SKIP_REASON = "no-clear-winner"
171+
# `no-hashable-candidates` and `no-clear-winner` are NOT declared here. They originate inside
172+
# `local_phash.find_best_match` and, as of the 2026-07-29 protected-core exception
173+
# (docs/upstreaming/license-provenance.md §2), are declared THERE as
174+
# `PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON` / `PHASH_NO_CLEAR_WINNER_SKIP_REASON`. They were
175+
# briefly mirrored here because that file is protected core and could not be edited; the mirror is
176+
# gone, so there is ONE declaration per value and nothing that can drift. This module IMPORTS the
177+
# one it needs (see the import block above) rather than re-declaring it.
178+
#
179+
# The two refinements below ARE this module's own: `_classify_no_clear_winner` splits phash's
180+
# undifferentiated `no-clear-winner` into a threshold miss and a margin miss, so plain
181+
# `no-clear-winner` is never written to `CardScanLog` by this module today - HISTORICAL rows
182+
# predating that refinement still carry it.
179183
PHASH_NO_CLEAR_WINNER_DISTANCE_SKIP_REASON = "no-clear-winner-distance"
180184
PHASH_NO_CLEAR_WINNER_MARGIN_SKIP_REASON = "no-clear-winner-margin"
181185

@@ -1994,8 +1998,10 @@ def verify_zero_resolutions(card_ids: list[int], batch_size: int = 2000) -> list
19941998
"OCR_UNKNOWN_SET_CODE_SKIP_REASON",
19951999
"PARSED_BUT_NO_MATCH_SKIP_REASON",
19962000
"PHASH_TOO_MANY_CANDIDATES_SKIP_REASON",
1997-
"PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON",
1998-
"PHASH_NO_CLEAR_WINNER_SKIP_REASON",
2001+
# `PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON` / `PHASH_NO_CLEAR_WINNER_SKIP_REASON` are
2002+
# deliberately absent: they are declared and exported by `cardpicker.local_phash`, their
2003+
# origin. Re-exporting them from here would recreate the two-names-one-value ambiguity that
2004+
# removing the mirror was meant to end.
19992005
"PHASH_NO_CLEAR_WINNER_DISTANCE_SKIP_REASON",
20002006
"PHASH_NO_CLEAR_WINNER_MARGIN_SKIP_REASON",
20012007
"RESCANNABLE_SKIP_REASONS",

MPCAutofill/cardpicker/local_phash.py

Lines changed: 37 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,27 @@
7272
# win either one already gets over full resolution.
7373
INGEST_HASH_FETCH_DPI = 40
7474

75+
# THIS MODULE'S SKIP VOCABULARY (2026-07-29, see docs/reference/skip-reasons.md). `find_best_match`
76+
# below returns these two strings; via `local_identify_printing_tags.run_phash_for_card` they reach
77+
# `CardScanLog.skip_reason`, so they are roster members and must be statically enumerable - the
78+
# docs_lint roster tether derives the roster from module-level `*_SKIP_REASON = "<literal>"`
79+
# declarations and CANNOT see a bare inline literal, which is exactly the hole a new literal added
80+
# inside `find_best_match` would have fallen through.
81+
#
82+
# THIS FILE IS PROTECTED CORE (docs/upstreaming/license-provenance.md section 2). These two
83+
# declarations exist under a NARROW, EXPLICIT owner exception granted 2026-07-29, recorded in that
84+
# section - it authorises declaring skip-reason constants HERE and nothing else. The file remains
85+
# protected; any other change to it still needs its own ruling.
86+
#
87+
# Naming: the `PHASH_` prefix is kept even though this module is already the phash engine, because
88+
# these names moved here VERBATIM from `local_identify_printing_tags`, where they were mirrored,
89+
# and because the consuming module reads them alongside its own
90+
# `PHASH_NO_CLEAR_WINNER_{DISTANCE,MARGIN}_SKIP_REASON` refinements as one family. Keeping the
91+
# names byte-identical (not only the values) is what makes this change nothing but a move: the
92+
# roster's pinning test and the doc's Constant column needed no edit at all.
93+
PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON = "no-hashable-candidates"
94+
PHASH_NO_CLEAR_WINNER_SKIP_REASON = "no-clear-winner"
95+
7596

7697
def _hash_to_int(image_hash: "imagehash.ImageHash") -> int:
7798
return twos_complement(str(image_hash), _HASH_BITS)
@@ -376,15 +397,20 @@ def find_best_match(
376397
margin: int = DEFAULT_MARGIN,
377398
) -> tuple[Optional[PhashMatch], str]:
378399
"""
379-
Returns (match, skip_reason). skip_reason is "no-hashable-candidates" (every candidate
380-
failed to fetch/hash), "no-clear-winner" (best distance is over threshold, or the runner-up
381-
is too close behind it), or "" (matched). Requires at least 2 hashed candidates to compute a
382-
margin at all when there's more than one name-candidate in the first place; a genuinely
383-
single-candidate name (already excluded by the orchestrator's selection - phash only runs on
384-
multi-candidate names in practice) would just need the threshold.
400+
Returns (match, skip_reason). skip_reason is PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON
401+
("no-hashable-candidates" - every candidate failed to fetch/hash),
402+
PHASH_NO_CLEAR_WINNER_SKIP_REASON ("no-clear-winner" - best distance is over threshold, or the
403+
runner-up is too close behind it), or "" (matched). Requires at least 2 hashed candidates to
404+
compute a margin at all when there's more than one name-candidate in the first place; a
405+
genuinely single-candidate name (already excluded by the orchestrator's selection - phash only
406+
runs on multi-candidate names in practice) would just need the threshold.
407+
408+
Any NEW skip reason added here must be declared as a module-level `*_SKIP_REASON` constant
409+
above and documented in docs/reference/skip-reasons.md - a bare literal returned from here
410+
reaches `CardScanLog` without the roster tether ever seeing it.
385411
"""
386412
if not candidates_with_hashes:
387-
return None, "no-hashable-candidates"
413+
return None, PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON
388414

389415
# card_hash and each candidate hash are both plain ints (the DB storage representation) -
390416
# ImageHash's `-` operator (Hamming distance) needs two ImageHash objects, not raw ints.
@@ -400,9 +426,9 @@ def find_best_match(
400426
runner_up_distance = scored[1][1] if len(scored) > 1 else None
401427

402428
if best_distance > distance_threshold:
403-
return None, "no-clear-winner"
429+
return None, PHASH_NO_CLEAR_WINNER_SKIP_REASON
404430
if runner_up_distance is not None and (runner_up_distance - best_distance) <= margin:
405-
return None, "no-clear-winner"
431+
return None, PHASH_NO_CLEAR_WINNER_SKIP_REASON
406432

407433
return PhashMatch(candidate=best_candidate, distance=best_distance, runner_up_distance=runner_up_distance), ""
408434

@@ -412,6 +438,8 @@ def find_best_match(
412438
"DEFAULT_MARGIN",
413439
"ART_CROP_BOX",
414440
"INGEST_HASH_FETCH_DPI",
441+
"PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON",
442+
"PHASH_NO_CLEAR_WINNER_SKIP_REASON",
415443
"DEFAULT_BACKFILL_BATCH_SIZE",
416444
"DEFAULT_BACKFILL_WORKERS",
417445
"DEFAULT_PIPELINE_QUEUE_DEPTH_BATCHES",

MPCAutofill/cardpicker/tests/test_skip_reason_roster.py

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
hand-written set, so a value reintroduced as a bare inline literal (which
2222
the derivation cannot see) fails here even if nothing else notices.
2323
"""
24+
import ast
2425
import re
2526
from pathlib import Path
2627

@@ -204,3 +205,82 @@ def test_docs_roster_tether_is_clean():
204205
spec.loader.exec_module(module)
205206

206207
assert module.check_skip_reason_roster_tether() == []
208+
209+
210+
# ---------------------------------------------------------------------------
211+
# The protected-core exception's own guard rails (2026-07-29,
212+
# docs/upstreaming/license-provenance.md section 2.1).
213+
#
214+
# `local_phash.find_best_match` produces two roster values. Until the owner
215+
# granted a narrow exception, that file could not be edited, so the constants
216+
# were MIRRORED in `local_identify_printing_tags.py` — which meant a NEW bare
217+
# literal returned from `find_best_match` reached `CardScanLog` with nothing to
218+
# catch it, because the roster tether cannot enumerate literals it cannot see.
219+
#
220+
# The exception was granted to close exactly that hole, and these two tests are
221+
# what keep it closed. They fail if the mirror comes back (two declarations that
222+
# can drift) or if a bare literal is reintroduced at the origin (a roster member
223+
# no derivation can find). Neither failure mode is visible to the tether itself,
224+
# which is the whole reason they are pinned here.
225+
# ---------------------------------------------------------------------------
226+
227+
PHASH_ORIGIN_SKIP_REASONS = {
228+
"PHASH_NO_HASHABLE_CANDIDATES_SKIP_REASON": "no-hashable-candidates",
229+
"PHASH_NO_CLEAR_WINNER_SKIP_REASON": "no-clear-winner",
230+
}
231+
232+
233+
def _module_level_str_constants(path: Path) -> dict[str, str]:
234+
tree = ast.parse(path.read_text())
235+
return {
236+
node.targets[0].id: node.value.value
237+
for node in tree.body
238+
if isinstance(node, ast.Assign)
239+
and len(node.targets) == 1
240+
and isinstance(node.targets[0], ast.Name)
241+
and isinstance(node.value, ast.Constant)
242+
and isinstance(node.value.value, str)
243+
}
244+
245+
246+
def test_phash_skip_reasons_are_declared_at_their_origin_and_nowhere_else():
247+
"""One declaration per value, in the module that produces it."""
248+
declared_in_phash = _module_level_str_constants(CARDPICKER_DIR / "local_phash.py")
249+
for name, value in PHASH_ORIGIN_SKIP_REASONS.items():
250+
assert declared_in_phash.get(name) == value, (
251+
f"{name} must be declared in local_phash.py, where find_best_match produces it, "
252+
f"with the value {value!r} — see license-provenance.md section 2.1."
253+
)
254+
255+
for py in sorted(CARDPICKER_DIR.glob("*.py")):
256+
if py.name == "local_phash.py":
257+
continue
258+
for name, value in _module_level_str_constants(py).items():
259+
assert value not in set(PHASH_ORIGIN_SKIP_REASONS.values()), (
260+
f"{py.name} re-declares a local_phash skip reason as {name}={value!r}. "
261+
f"That mirror was removed on purpose: two declarations of one value can drift. "
262+
f"Import it from cardpicker.local_phash instead."
263+
)
264+
265+
266+
def test_find_best_match_returns_no_bare_skip_reason_literal():
267+
"""Every skip reason `find_best_match` returns must be a NAME bound to one of
268+
its module's own constants. A bare literal here is invisible to the roster
269+
derivation and would reach `CardScanLog` unnoticed — the exact defect the
270+
protected-core exception was granted to fix."""
271+
path = CARDPICKER_DIR / "local_phash.py"
272+
tree = ast.parse(path.read_text())
273+
constants = _module_level_str_constants(path)
274+
fn = next(n for n in tree.body if isinstance(n, ast.FunctionDef) and n.name == "find_best_match")
275+
276+
returned = [n.value.elts[1] for n in ast.walk(fn) if isinstance(n, ast.Return) and isinstance(n.value, ast.Tuple)]
277+
assert returned, "find_best_match no longer returns a (match, skip_reason) tuple"
278+
for node in returned:
279+
if isinstance(node, ast.Constant) and node.value == "":
280+
continue # the "matched, not a skip" sentinel, not a roster member
281+
assert isinstance(node, ast.Name), (
282+
f"line {node.lineno}: find_best_match returns a bare skip-reason literal. "
283+
f"Declare it as a module-level *_SKIP_REASON constant and document it in "
284+
f"docs/reference/skip-reasons.md."
285+
)
286+
assert node.id in constants, f"line {node.lineno}: {node.id} is not a module-level string constant"

docs/reference/skip-reasons.md

Lines changed: 32 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -140,11 +140,17 @@ longer exists in the code; it wrote `fetch_failed` like the rest.
140140
| `no-clear-winner-margin` | `PHASH_NO_CLEAR_WINNER_MARGIN_SKIP_REASON` | `local-phash-v1` | A candidate cleared the threshold, but the runner-up was too close behind it. | Live |
141141

142142
`no-hashable-candidates` and `no-clear-winner` are the two values whose
143-
strings physically originate inside `MPCAutofill/cardpicker/local_phash.py`,
144-
which is PROTECTED CORE (`docs/upstreaming/license-provenance.md` section 2) and therefore cannot be edited to declare them at their true source.
145-
They are **mirrored** in the consuming module, which is the one place in
146-
the roster where the declaration is not co-located with the origin. See
147-
"The values that are not declared at their origin" below.
143+
strings physically originate inside `MPCAutofill/cardpicker/local_phash.py`
144+
(`find_best_match`), and that is where their constants are **declared**
145+
not in this section's own module. That file is PROTECTED CORE
146+
(`docs/upstreaming/license-provenance.md` section 2); the two declarations
147+
sit there under the narrow owner exception granted 2026-07-29 and recorded
148+
in that section. They were briefly mirrored in
149+
`local_identify_printing_tags.py`, while that file was the only editable
150+
one; the mirror is gone, so every roster value now has exactly one
151+
declaration and it is co-located with its origin.
152+
`local_identify_printing_tags` imports `PHASH_NO_CLEAR_WINNER_SKIP_REASON`
153+
from `local_phash` for its `_classify_no_clear_winner` refinement test.
148154

149155
## Stage D join-key calculator — `MPCAutofill/cardpicker/local_calculate_verdicts.py`
150156

@@ -266,27 +272,32 @@ persisted values with no further work.
266272

267273
Stated explicitly rather than forced, per the sweep's own brief.
268274

269-
**`local_phash.find_best_match`'s two return literals**
270-
(`no-hashable-candidates`, `no-clear-winner`) are emitted from
275+
**`local_phash.find_best_match`'s two return values — CLOSED 2026-07-29.**
276+
`no-hashable-candidates` and `no-clear-winner` were the sweep's one
277+
exception: they are produced inside
271278
`MPCAutofill/cardpicker/local_phash.py`, which is PROTECTED CORE
272-
(`docs/upstreaming/license-provenance.md` section 2) and cannot be edited
273-
to declare them at source without an owner exception. They are mirrored as
274-
constants in the consuming module,
275-
`MPCAutofill/cardpicker/local_identify_printing_tags.py`, and that module's
276-
equality test reads the mirrored constant, so the coupling is named rather
277-
than anonymous. The roster is still complete — but the guarantee is
278-
"declared where it is consumed", not "declared where it is produced", for
279-
these two alone. A NEW literal added to `find_best_match` would flow
280-
through to `CardScanLog` without any lint failure. That residual gap closes
281-
only by editing a protected file.
279+
(`docs/upstreaming/license-provenance.md` section 2), so the sweep could
280+
not declare them at source and mirrored them in the consuming module
281+
instead. That left a real hole — a NEW literal added inside
282+
`find_best_match` would have reached `CardScanLog` without the tether ever
283+
seeing it, because the tether cannot enumerate literals it cannot see.
284+
The owner granted a narrow exception on 2026-07-29 (recorded in
285+
`license-provenance.md` section 2, which also states its limits): the two
286+
constants are now declared in `local_phash.py` itself, the mirror in
287+
`local_identify_printing_tags.py` is removed, and the tether reports
288+
`local_phash.py` as the declaration site. **Every roster value is now
289+
declared where it is produced.** The exception covers skip-reason constants
290+
in that one file only; the file remains protected.
282291

283292
**The lands module's phash-branch composition.** That module reports
284293
`f"{LANDS_PHASH_SKIP_REASON_PREFIX}{reason}"` — a `phash-` prefix
285294
concatenated onto whatever `find_best_match` returned. The prefix is a
286-
constant and the routing test reads it, but the composed value is not a
287-
declared string and cannot be made one without enumerating protected-core
288-
returns. This is harmless today precisely because nothing in that module is
289-
persisted; the constant's own comment records that a `CardScanLog` write
295+
constant and the routing test reads it, but the composed value is still not
296+
a declared string — the concatenation is what the tether cannot see, and
297+
that is true regardless of the two `find_best_match` returns now being
298+
named constants. This is harmless today precisely because nothing in that
299+
module is persisted; the constant's own comment records that a
300+
`CardScanLog` write
290301
must not be added there until the composition is replaced with explicit
291302
per-outcome constants.
292303

0 commit comments

Comments
 (0)