Audit: does the pipeline still catch what it aims to? The composition check - #616
Merged
Merged
Conversation
This was referenced Jul 29, 2026
Open
WilfordGrimley
force-pushed
the
docs/pipeline-coverage-audit
branch
from
July 30, 2026 08:41
16a0e92 to
805566f
Compare
… production Every change of the last three days was verified in isolation; none was verified in composition. This enumerates every vote channel, calculator identity, attribute-chip channel and Stage C extractor from code, measures each against production, and traces each to the engine that can reach it. Four findings, ranked: 1. Frame-style (Old/Modern Border) and bleed-edge (appropriate-bleed) chips are at ZERO machine rows with no substitute channel AND no wiring into either engine. 142,633 + 2,786 votes are re-derivable from stored ImageEvidence with no image fetch. Border chips survived the same purge only because they were being computed twice. 2. image_evidence.extract_card_evidence has zero production callers - only tests. Both engines call compute_card_evidence directly. It is the only non-pilot caller of cast_border_attribute_vote. Same defect shape as the artist recovery PR #581 fixed. 3. _slow_path_eligible_cards_queryset has no exclusion for the illustration calculator's identity; the code comment says the wiring "would need" it. A card illustration resolves is still routed to a human. PR #604 does not close this. 4. Four Stage D readers gate on one extractor key and then read six extractors' fields ungated. Two degrade in the strict direction, one into a permanently non-rescannable frame-mismatch skip. Also: local-name-frequency-v1 and scryfall-tagger-v1 have zero PilotRunLedger rows ever - never run, not merely dormant. docs/reference/skip-reasons.md was checked value by value against production and holds unchanged. READ-ONLY: no write, no management command, no migration, no deploy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
WilfordGrimley
force-pushed
the
docs/pipeline-coverage-audit
branch
from
July 30, 2026 09:00
805566f to
5c1ec23
Compare
This was referenced Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Read-only audit. No production write, no management command, no migration, no deploy. One new document under
docs/reports/, indexed,docs_lint.py --strictclean. No pipeline code changed.The owner's question: "have we verified that with all of our deletions that the pipeline actually still catches all that it aims to? do we need to do any more reordering or culling? readd anything we dropped?"
Every change of the last three days was verified in isolation. Nothing had checked them in composition. This does: every vote channel, calculator identity, attribute-chip channel and Stage C extractor enumerated from code, measured against production, and traced to the engine that can reach it.
The answers
Q1 — does it still catch everything? No. Three channels are down.
Old Border/Modern Border) and bleed-edge chips (appropriate-bleed) are at zero machine rows, have no substitute channel, and are unreachable from both the pooled runner and the conveyor. Three independent failures stacked on one channel. 142,633 + 2,786 votes are re-derivable from storedImageEvidencewith no image fetch at all.local-fallback-v1(now 0) andlayout-class-cast-v1(216,476). Had the duplication been culled first, border colour would be zero too.Q2 — more reordering or culling? Yes to both.
_slow_path_eligible_cards_querysethas no exclusion for the illustration calculator's identity. The code comment admits the wiring "would need" it. A card illustration resolves is still routed to a human reviewer in the same run. PR Run-scoped eligibility, the value-comparing split, and an archive for superseded votes #604 does not close this; it fires on the firststage-d-illustration-v2run.extractor_versionskey and then read six extractors' fields ungated. Two degrade in the strict direction — a missingartist_ocrmakesillus_anchor_firedreadFalse, classifies every framemodern, and can veto a genuine old-frame card asframe-mismatch, which is deliberately not rescannable. Permanent wrong skip. The correct pattern (REQUIRED_EXTRACTOR_KEYS) already exists three times in the same codebase.image_evidence.extract_card_evidencehas zero production callers — only tests. Both engines callcompute_card_evidencedirectly. It is the only non-pilot caller ofcast_border_attribute_vote. Exactly the defect shape PR Wire collector-line artist recovery into the pooled Stage C cohort runner #581 fixed for artist recovery.Q3 — what to re-add? Frame-style chips (142,633), bleed-edge chips (2,786), border-chip arrears (~2,648). Nothing else — no other purge removed something nothing re-derives.
Also established
local-name-frequency-v1andscryfall-tagger-v1have zeroPilotRunLedgerrows, ever. Never run, not merely dormant — that closes the open retire-or-keep diagnosis with a fact.docs/reference/skip-reasons.mdwas checked value by value against production. It holds unchanged — the one roster in this audit that survived contact with production.Cardrows have noImageEvidencerow at all.Every zero is classified into one of the four states the brief asked to be kept apart (purged-with-no-substitute / purged-but-superseded / wired-but-never-run / deliberately-retired-and-replaced), and every finding is marked LIVE or LATENT against the deployed build.
🤖 Generated with Claude Code
https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN