feat(stage-d): wire four FREE evidence-only vote channels into the conveyor - #724
Merged
Conversation
…nveyor Closes a gap where four calculator identities (ai-art-detector-v1, lands-artist-decomp-v1, residual-classify-v1, art-hash-artist-v1) had a real, card_ids-scoped, purge_and_write_votes-backed code path but no caller reachable from stage_e_dispatch._run_stage_d or stream_full_catalog.py - a channel that never runs and one that runs and casts zero votes are indistinguishable from the outside. All four read only already-stored evidence (ImageEvidence OCR text, Card.content_phash, already-resolved artist/printing chains) and fetch no image; snip run_lands_identify/run_frame_mismatch_recovery are called with every live-fetch budget forced to 0, each module's own docstring already documenting that as the scoped, genuinely free path. New stage_e_dispatch._run_evidence_only_calculators, called from _run_stage_d after the attribute-chip casters. New DispatchOutcome counters per channel, folded into both the streaming dispatch's PilotRunLedger.counters and run_pipeline's bulk/streaming result dicts (also closes a pre-existing gap where the three attribute-chip counters were missing from the streaming ledger.counters merge, though not from DispatchOutcome itself). Six other identities were investigated and left deliberately unwired, each with a reason recorded in docs/pipeline-fidelity-gate.md's new "Wiring status" section and a tracked issue: - art-edge-continuity-v1: gated behind its own stated validation precondition (#721) - deductive-backfill-v1, local-name-frequency-v1: FREE data source but no card_ids scoping - would rebuild a whole-catalogue index/census on every micro-batch (#722) - local-ocr-v1, local-phash-v1, local-fallback-v1: primary caster is the dormant live-fetch pilot engine, no stored-evidence reconstruction path (#723) Roster count correction: the derived calculator roster is 17 real vote-casting identities (19 declared, 2 allowlisted), not ~28 as previously circulated. 11 of 17 are now wired (7 before this change, 4 added here), 6 deliberately deferred. Tests: new TestEvidenceOnlyCalculators in test_stage_e_dispatch.py, one per channel, asserting on dispatch_micro_batch's own DispatchOutcome/ PilotRunLedger - not on the calculator function in isolation. Verified each test goes red when the new call is removed from _run_stage_d (mutation check), then restored and re-verified green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires four FREE-data vote-casting calculators into the streaming conveyor's
stage_e_dispatch._run_stage_d(and therefore intorun_pipeline's bulkmonolith, which calls the same function):
ai-art-detector-v1,lands-artist-decomp-v1,residual-classify-v1,art-hash-artist-v1. Eachhad a real, tested,
card_ids-scoped code path already (issue #533's ownbatch-scoping prerequisite already applied to each) but no caller reachable
from the conveyor — a channel that never runs and a channel that runs and
casts zero votes are indistinguishable from the outside, which is the
defect this closes.
Roster count correction: the derived calculator roster
(
docs_lint.py::check_calculator_roster_tether) is 17 real vote-castingidentities (19 declared, 2 allowlisted as non-calculators), not the ~28
previously circulated. 11 of 17 are now invoked by a real call site (7
before this PR, 4 added here); snip 6 are deliberately left unwired with a
stated reason and a tracked issue.
Roster table (the answer to "does a pass invoke this")
stage-d-join-key-v1stage-d-fallback-v1stage-d-illustration-v2stage-d-slow-path-v1CardScanLogonlylayout-class-cast-v1frame-style-cast-v1bleed-edge-cast-v1ai-art-detector-v1lands-artist-decomp-v1fetch_budget=0)residual-classify-v1art-hash-artist-v1art-edge-continuity-v1deductive-backfill-v1card_idsparam; snip rebuilds 113k-row index/call — #722local-name-frequency-v1card_idsparam; snip catalogue-wide census/call — #722local-ocr-v1run_pilot) needs live fetch+tesseract — #723local-phash-v1run_pilot) needs live fetch+hash — #723local-fallback-v1(
evidence-transfer-v1/question-feed-hypothetical-voteare allowlisted —not vote-casting calculators, see
CALCULATOR_ROSTER_ALLOWLIST.)What shipped
stage_e_dispatch.py: new_run_evidence_only_calculators(run_id, card_ids, outcome, dry_run), called from_run_stage_dafter theattribute-chip casters, with a new
seam("stage-d:evidence-only")envelope-check point matching the existing convention. Calls
run_ai_art_detector,run_frame_mismatch_recovery(
ocr_refetch_budget=0, fallback_refetch_budget=0),run_d0_sibling_artist_propagation,run_lands_identify(
sample_size=None, fetch_budget=0) — every live-fetch budget forced to0, exactly the "scoped, genuinely free" path each module's own docstring
already documents.
run_ai_art_detector'sRuntimeError(missing"AI-Generated"
Tagseed) is caught and logged, matching_run_attribute_chip_casters' own established pattern — a missingoperator seed must not fail votes the printing calculators already
wrote this batch. Six new
DispatchOutcomecounters.stage_e_dispatch.dispatch_micro_batch: the six new counters foldedinto the streaming dispatch's
PilotRunLedger.counters. Also added thethree PRE-EXISTING
stage_d_border_chip_votes/frame_chip_votes/bleed_chip_votescounters to this same merge — they were present onDispatchOutcomeand tested against it directly, but were neversurfaced into the streaming ledger's own
countersdict (bulk mode'srun_pipeline.pyalready reported them). Minor, directly adjacent fix,folded into the same merge dict this PR was already touching.
run_pipeline.py: the 6 new + (belatedly) 3 chip counters added toboth the bulk-mode
resultdict and the streaming-modeaccaccumulator, plus the STAGE D log line text updated to name the four
new calculators.
test_stage_e_dispatch.py: newTestEvidenceOnlyCalculators, onetest per channel, each asserting on
dispatch_micro_batch's ownDispatchOutcome/PilotRunLedger— never on the calculator function inisolation (that coverage already exists in each calculator's own test
module). A fifth test proves
run_ai_art_detector's missing-tag-seedRuntimeErrordoes not halt the other three channels in the samebatch.
docs/pipeline-fidelity-gate.md: new "Wiring status" section (theroster table above, in durable form) plus issue links added to the
existing
art-edge-continuity-v1/local-name-frequency-v1rosterentries.
deferred channels: #721 (art-edge-continuity-v1),
#722 (deductive-backfill-v1 + local-name-frequency-v1, shared root cause),
#723 (local-ocr-v1/local-phash-v1/local-fallback-v1, shared root cause).
Deviations from spec
shares one root cause (a missing
card_idsscoping prerequisite fordeductive-backfill/name-frequency; snip the dormant live-fetch pilot engine
for ocr/phash/fallback) and one fix would resolve both/all three members
at once — a separate issue per identity would fragment one piece of work
across multiple trackers. Each issue still states per-channel acceptance
criteria.
local_art_edge.py/deductive_backfill.py/local_identify_printing_tags.pyto point at the new issues from insidetheir own docstrings. The brief asks for a reason "in a durable place
(the module's own docstring, OR a doc under docs/)" —
docs/ pipeline-fidelity-gate.md's new section satisfies this without touchingthree additional source files for a documentation-only cross-reference.
numbers, not just the wired count — the real roster is 17, not ~28.
Reported as found, not corrected silently.
run_slow_path_calculator/no attribute-chip casters) was already stalerelative to current master — PR Wire the attribute-chip casters into an engine; strip the vote from the uncalled wrapper #654 (2026-07-30, already on
masterbefore this branch was cut) had already wired
run_slow_path_calculatorand the three attribute-chip casters. This PR's roster table reports the
true current state rather than the brief's premise.
Test plan
python -m pytest cardpicker/tests/test_stage_e_dispatch.py— 93passed (individually)
python -m pytest cardpicker/tests/test_run_pipeline.py— 32passed (individually)
python -m pytest cardpicker/tests/test_stage_e_dispatch.py cardpicker/tests/test_run_pipeline.py— 125 passed (together)python -m pytest cardpicker/tests/test_local_detect_ai_art.py cardpicker/tests/test_local_lands_identify.py cardpicker/tests/test_local_residual_classify.py— 175 passed(individually and together; snip these calculator modules were not
modified, verified as a due-diligence check)
to go red when
_run_evidence_only_calculators(...)is removedfrom
_run_stage_d, then the call restored and the same 5 verifiedgreen again
pre-commit run(ruff, isort, black, mypy, prettier) — clean on all4 changed files
python .github/scripts/docs_lint.py— clean (roster tether intact)test_artbox_exemplar_backfill.pyerrors atSETUP with
AttributeError: type object 'PytestDjangoTestCase' has no attribute '_pre_setup_ran_eagerly'— a documented pre-existingenvironment issue, not touched by this PR, not run as part of the
verification above
Backfill (NOT run as part of this PR)
Wiring affects future passes only. The existing catalogue needs a
catch-up run per newly-wired channel, over stored evidence:
ai-art-detector-v1:management/commands/local_detect_ai_art.pyalready exists and calls
run_ai_art_detectorin exactly the samecard_ids=None / stored-evidence-only shape this PR wires per-batch — it
suffices as the backfill command, no new command needed.
lands-artist-decomp-v1:management/commands/local_lands_identify.pyalready exists and calls
run_lands_identify, but its CLI defaults(
sample_size=300, non-zerofetch_budget) target the pilot's ownsampled/fetch-budgeted mode, not the free evidence-backed path this PR
wires. A catch-up run needs
--sample-size=<None-equivalent> --fetch-budget=0(or equivalent full-pool, zero-fetch flags) to matchwhat the conveyor now does per batch — check the command's current flag
surface before running.
residual-classify-v1/art-hash-artist-v1:management/commands/local_residual_classify.pyalready exists andcalls both
run_frame_mismatch_recovery/run_d0_sibling_artist_propagation.For the phash-only free path this PR wires, run with
--ocr-refetch-budget=0 --fallback-refetch-budget=0(or equivalent) tomatch; snip a full run with non-zero refetch budgets would ALSO cover the
OCR/fallback-flagged rows this PR intentionally leaves for the live-fetch
pilot decision tracked in local-ocr-v1 / local-phash-v1 / local-fallback-v1: the live-fetch pilot engine channels remain unreachable from the conveyor #723, and is a separate, larger-scope backfill
decision, not what this PR's wiring covers.
Sequencing caveat carried over from the brief:
ai-art-detector-v1and the evidence-backed branches of
lands-artist-decomp-v1/residual-classify-v1all read Stage C OCR text. A full-catalogue passwas reported live on this box re-deriving that same OCR text under new
extractor versions at the time this branch was cut — a backfill run for
any of these three channels started before that pass completes would
produce verdicts against stale extractor output that would need
re-running once it finishes. Do not run any backfill from this PR
without first confirming that pass has completed.
Open items
art-edge-continuity-v1's validation pass (art-edge-continuity-v1: run the stated extended-art validation pass before wiring the vote #721) run before orafter this PR merges? It needs no live fetch (pure DB read against
already-imported Scryfall
frame_effects), so it could run independentof the concurrent full-catalogue pass mentioned in the brief — owner
call.
deductive-backfill-v1/local-name-frequency-v1(deductive-backfill-v1 / local-name-frequency-v1: add card_ids batch scoping so these FREE-data channels can be wired into Stage D #722): is addingcard_idsscoping + the shared process-cache todeductive-backfill-v1worth doing, or should this channel be retired instead (issue deductive backfill: the first tier's "cross-verified against Scryfall" condition excludes nothing (137 -> 137) #600
already flags that its D1 tier's advertised external corroboration was
never implemented)?
local-name-frequency-v1's own roster entry alreadysays "may be retired" — this PR does not resolve that question, only
sharpens why it's currently unwireable.
local-ocr-v1/local-phash-v1/local-fallback-v1(local-ocr-v1 / local-phash-v1 / local-fallback-v1: the live-fetch pilot engine channels remain unreachable from the conveyor #723): is thedormant
run_pilotengine still a channel this project wants live atall, given
local_calculate_verdicts' three calculators already coverthe equivalent ground via Stage C's evidence-first design? This is
the largest open question of the three issues and needs an owner
decision before any further work on it.
Live state
wire-free-vote-channelspushed to origin, 1 commit ahead ofmaster(17345a48).ProxyPrints/ProxyPrints.github.io, all open, no other action taken onthem.
database was run at any point in this session.