From bc63ec9cb08d6e3256b99b2a23e18e7a245f832a Mon Sep 17 00:00:00 2001 From: wilfordgrimley <2397930+WilfordGrimley@users.noreply.github.com> Date: Mon, 27 Jul 2026 21:49:31 +0000 Subject: [PATCH 1/5] docs: document local_calculate_verdicts --diff-report and per-calculator counters (PR #494) --- docs/features/stage-e-operations.md | 49 +++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/docs/features/stage-e-operations.md b/docs/features/stage-e-operations.md index 83504cc01..9dfa061e2 100644 --- a/docs/features/stage-e-operations.md +++ b/docs/features/stage-e-operations.md @@ -1014,6 +1014,55 @@ of §3-§5 as a whole): the resulting ledger data (issue #463) is still an owner-polled action, gated on the §8 Tron pass. +## `local_calculate_verdicts` observability additions (PR #494) + +Two observability additions landed in `local_calculate_verdicts.py`, motivated +by run `20260726T165343-3e8301db` where `counters={}` appeared in the ledger +row because stdout was severed before the terminal summary printed. + +### `--diff-report PATH` + +Writes one JSONL line per card the run **would act on** (skipped cards +excluded) to the given path. Each line: + +```json +{"card_id": "...", "calculator": "join_key|fallback|slow_path", "would_cast": , "existing_votes": [...]} +``` + +- `existing_votes`: every `CardPrintingTag` vote for the card across all + `anonymous_id` values — the complete prior-vote picture at the time of the + run. +- Stream-written: appended and flushed after each calculator completes, never + buffered in memory. +- The path is **truncated on open** (not appended to); an unwritable path + errors **before** the ledger row is created. +- Passing `--diff-report` lifts the default audit sample cap from 20 to + `sys.maxsize` — use it when you need complete per-card coverage of a batch. + +### Per-calculator counters in the ledger row + +`PilotRunLedger.counters` now carries a sub-dict per calculator: +`counters["join_key"]`, `counters["fallback"]`, `counters["slow_path"]`. Each +has the same shape: + +```json +{ + "considered": 0, + "would_cast": 0, + "votes_written": 0, + "already_voted": 0, + "skip_counts": {} +} +``` + +For `slow_path`, `would_cast`/`votes_written` count its **routed** +`CardScanLog` rows (the slow-path calculator casts no `CardPrintingTag` votes); +`skip_counts` is the `reason_counts` breakdown from that calculator. + +Counters are persisted to the ledger row **before** the terminal summary is +printed (counters-before-output discipline), so a severed stdout no longer +produces an empty `counters={}` record. + ## See also - [`docs/proposals/stage-e-streaming.md`](../proposals/stage-e-streaming.md) From 665eb50df2bc7852889dda636fcce160752fd456 Mon Sep 17 00:00:00 2001 From: wilfordgrimley <2397930+WilfordGrimley@users.noreply.github.com> Date: Mon, 27 Jul 2026 21:50:07 +0000 Subject: [PATCH 2/5] docs: document rejudge_fallback_channel compare-and-retract command (PR #495) --- docs/features/stage-e-operations.md | 62 +++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/docs/features/stage-e-operations.md b/docs/features/stage-e-operations.md index 9dfa061e2..ba7d5f488 100644 --- a/docs/features/stage-e-operations.md +++ b/docs/features/stage-e-operations.md @@ -1063,6 +1063,68 @@ Counters are persisted to the ledger row **before** the terminal summary is printed (counters-before-output discipline), so a severed stdout no longer produces an empty `counters={}` record. +## `rejudge_fallback_channel` — fallback-channel compare-and-retract (PR #495) + +`MPCAutofill/management/commands/rejudge_fallback_channel.py` re-evaluates the +`stage-d-fallback-v1` channel's existing conclusions against **current stored** +`ImageEvidence` and retracts the rows where the conclusion changed, making +those cards eligible for a fresh `local_calculate_verdicts` pass. + +### What it does + +For each card in the fallback channel, the command re-derives the fallback +calculator's conclusion from the card's current `ImageEvidence` +(`layout_class`, `artist_ocr_name`, `symbol_phash`) — zero image fetches, zero +re-parse. If the conclusion differs from the stored vote or skip, it +**retracts** (deletes) the `CardPrintingTag` vote and any `CardScanLog` skip +rows for that card. + +The command **never** touches `stage-d-join-key-v1` rows. Fallback eligibility +is gated on a join-key no-hit; retraction removes only the fallback-channel +layer, leaving the join-key layer untouched. + +### Safety gate + +A card is **never** retracted if `resolve_printing(card) is not None`. This +covers both a resolved printing and a resolved NO_MATCH consensus. Gated cards +are counted and their PKs are recorded in `counters["gate_refused_card_ids"]` +for human review before any manual intervention. + +### Flags + +| Flag | Notes | +| -------------------------- | ------------------------------------------------- | +| `--selector all-channel` | Operate on the entire fallback-channel population | +| `--card-ids-file PATH` | Operate on a newline-delimited list of card IDs | +| `--write` | Required to retract (dry-run by default) | +| `--skip-dryrun-check` | Skip the dry-run-window guard | +| `--dry-run-window-hours N` | Override the dry-run-window length | + +`--selector` and `--card-ids-file` are mutually exclusive; exactly one is +required. + +### Counters and ledger row + +`PilotRunLedger.votes_written` is repurposed to record the **retracted** count +(same convention as `reparse_collector_evidence`). Full counter set: + +``` +considered, no_evidence, no_prior_fallback_state, unchanged, +changed, retracted, gate_refused, transitions +``` + +`transitions` is a string-keyed breakdown of conclusion changes, e.g. +`"skip:ambiguous -> vote:12345": 42`. + +### Runbook + +Two-step sequence to bring retracted cards back into the active population: + +1. `python manage.py rejudge_fallback_channel --selector all-channel` (dry-run + → review counters and `gate_refused_card_ids`) → rerun with `--write` +2. `python manage.py local_calculate_verdicts` (unchanged) to fill the + retracted cards through the normal calculator chain. + ## See also - [`docs/proposals/stage-e-streaming.md`](../proposals/stage-e-streaming.md) From 66bbb3acf8d98705925d05ff776b5c8e65616525 Mon Sep 17 00:00:00 2001 From: wilfordgrimley <2397930+WilfordGrimley@users.noreply.github.com> Date: Mon, 27 Jul 2026 21:56:06 +0000 Subject: [PATCH 3/5] docs: Stage C full-catalog completion record (218,108/218,516, 99.8%) --- docs/pipeline-fidelity-gate.md | 36 ++++---- ...26-07-26-stagec-full-catalog-completion.md | 92 +++++++++++++++++++ docs/reports/README.md | 5 + 3 files changed, 117 insertions(+), 16 deletions(-) create mode 100644 docs/reports/2026-07-26-stagec-full-catalog-completion.md diff --git a/docs/pipeline-fidelity-gate.md b/docs/pipeline-fidelity-gate.md index ef4beb52b..4a2c3ed4c 100644 --- a/docs/pipeline-fidelity-gate.md +++ b/docs/pipeline-fidelity-gate.md @@ -435,24 +435,28 @@ one flattened figure: | `20260724T001154-d3986cfc` (post-fire calculator re-pass write) | **1** | **§14 "Five further passes" write** — `local_calculate_verdicts` re-run over the lexicon-gate-retracted pool; 52,348 `unknown-set-code` + 16 `no-evidence` skips, 117,442 `to-review` — see §14 | **Stage C run history** (`ImageEvidence.run_id`, current last-writer row -count per run — canaries first, then the main leg): - -| run_id | date | rows | -| ------------------------------------------ | ------------- | ------: | -| `stagec-canary-20260720T1659Z` | 2026-07-20 | 223 | -| `stagec-canary-decoupled-20260720T235127Z` | 2026-07-20 | 206 | -| `ntx-0721` | 2026-07-20/21 | 9,675 | -| `stagec-20k-20260721T0227Z` | 2026-07-20/21 | 10,696 | -| `stagec-remainder-0721` | 2026-07-21/22 | 197,470 | - -Full chain: the two decoupling canaries (Jul 20) validated the -fetch/compute-decoupled architecture at small scale, then `ntx-0721` + -`stagec-20k-20260721T0227Z` (Jul 20–21) ran a combined ~20k-card -extraction pass, then `stagec-remainder-0721` (Jul 21–22, 197,470-row -main leg) extracted the bulk of the remaining catalog. See +count per run — updated 2026-07-27 after the full-catalog re-extraction): + +| run_id | date | rows | +| --------------------------------------- | ------------- | ------: | +| `pass-pilot-20260725` | 2026-07-25 | 100 | +| `stage-e-stream-20260725T233633221123Z` | 2026-07-25 | 22 | +| `stage-e-stream-20260725T233633687349Z` | 2026-07-25 | 23 | +| `pass-full-20260725` | 2026-07-25/26 | 194,831 | +| `pass-full-20260725-r2` | 2026-07-25/26 | 23,132 | +| **Total** | | 218,108 | + +The 2026-07-25 full-catalog re-extraction (`pass-full-20260725` + +`pass-full-20260725-r2`) superseded all prior Stage C last-writer rows — +the original canaries (`stagec-canary-20260720T1659Z`, +`stagec-canary-decoupled-20260720T235127Z`), `ntx-0721`, +`stagec-20k-20260721T0227Z`, and `stagec-remainder-0721` no longer appear +as last-writer for any `ImageEvidence` row. See [`reports/2026-07-20-decoupled-canary-confirm.md`](reports/2026-07-20-decoupled-canary-confirm.md) and [`reports/2026-07-21-stagec-20k-extraction.md`](reports/2026-07-21-stagec-20k-extraction.md) -for the canary/20k narrative detail. None of these Stage C runs have a +for the original canary/20k narrative detail, and +[`reports/2026-07-26-stagec-full-catalog-completion.md`](reports/2026-07-26-stagec-full-catalog-completion.md) +for the full-catalog completion record. None of the Stage C runs have a `PilotRunLedger` row of their own — see §11 for what that does and doesn't affect. diff --git a/docs/reports/2026-07-26-stagec-full-catalog-completion.md b/docs/reports/2026-07-26-stagec-full-catalog-completion.md new file mode 100644 index 000000000..1f7d8d268 --- /dev/null +++ b/docs/reports/2026-07-26-stagec-full-catalog-completion.md @@ -0,0 +1,92 @@ +# Stage C full-catalog extraction — completion record (2026-07-26) + +Full re-extraction of the entire card catalog under the decoupled +fetch/compute architecture, superseding all prior Stage C runs (canaries, +20k cohort, and the Jul 21–22 remainder leg). Preceded by +[`2026-07-20-decoupled-canary-confirm.md`](2026-07-20-decoupled-canary-confirm.md) +and +[`2026-07-21-stagec-20k-extraction.md`](2026-07-21-stagec-20k-extraction.md). + +This session did not execute the extraction — it records the result against +the live DB (`mpcautofill_django`) as of 2026-07-27. All figures below come +from `SELECT`/`count()` queries; no writes were made. + +## Run parameters + +| run_id | last write | rows (last-writer) | +| --------------------------------------- | ---------- | -----------------: | +| `pass-pilot-20260725` | 2026-07-25 | 100 | +| `stage-e-stream-20260725T233633221123Z` | 2026-07-25 | 22 | +| `stage-e-stream-20260725T233633687349Z` | 2026-07-25 | 23 | +| `pass-full-20260725` | 2026-07-26 | 194,831 | +| `pass-full-20260725-r2` | 2026-07-26 | 23,132 | +| **Total** | | **218,108** | + +A 100-card pilot (`pass-pilot-20260725`) preceded the main legs. Two +`stage-e-stream-*` entries represent Stage E streaming activity on 45 cards +total. The two main legs (`pass-full-20260725` + `-r2`) together cover +217,963 cards. + +The `pass-full-20260725` pair is a **full re-extraction**: every prior +last-writer row (`stagec-remainder-0721`, `stagec-20k-20260721T0227Z`, +`ntx-0721`, and the two Jul-20 canaries) has been replaced — none of those +run_ids appear as last-writer for any `ImageEvidence` row in the current +DB. The Stage C run history table in +[`../pipeline-fidelity-gate.md`](../pipeline-fidelity-gate.md) §6 reflects +this updated state. + +## Coverage + +- Total cards in DB: **218,516** +- Cards with any `ImageEvidence` row: **218,108** (99.8%) +- Cards with no `ImageEvidence` row: **408** (0.2%) — all `GOOGLE_DRIVE`-sourced + +Of the 218,108 cards with evidence: + +- `fetch_ok=True`: 216,494 (99.3% of those with evidence — successfully extracted) +- `fetch_ok=False`: 1,614 (0.7% — fetch attempted, image unavailable, evidence row + written with failure recorded) + +The 408 cards with no evidence at all were never successfully started during any +pass; all are `GOOGLE_DRIVE`-sourced. + +## Fetch-failure rate vs. prior runs + +| run | cards | fetch_ok=False | rate | +| ------------------------------------------------------------- | ----------: | -------------: | --------: | +| Bundled canary (`stagec-canary-20260720T1659Z`) | 400 | 0 | 0.0% | +| Decoupled canary (`stagec-canary-decoupled-20260720T235127Z`) | 400 | 6 | 1.5% | +| 20k cohort (`stagec-20k-20260721T0227Z`) | 20,000 | 105 | 0.53% | +| **Full catalog (`pass-full-20260725` + r2)** | **218,108** | **1,614** | **0.74%** | + +The full-catalog failure rate (0.74%) is higher than the 20k cohort's 0.525%. +This is consistent with broader catalog coverage: the 20k cohort used +edhrec_rank-ascending ordering (popular cards first, cold tail deferred), while +the full pass covered the entire cold tail — where per-file breakage across +independent Drive folders is more prevalent. + +## No-pixels invariant + +Not re-checked in this session — the schema inspection in +[`2026-07-21-stagec-20k-extraction.md`](2026-07-21-stagec-20k-extraction.md) +applies unchanged. `ImageEvidence` holds hashes, measurements, classification +strings, OCR text, and pixel coordinates; no field is capable of holding image +bytes. + +## Stage completion + +Stage C extraction is at **99.8% catalog coverage** (218,108 / 218,516). +The original compute profile +([`2026-07-20-pipeline-compute-profile.md`](2026-07-20-pipeline-compute-profile.md)) +projected 116.2h single-threaded against a 6.2h reference budget (BLOCKING). +The decoupled fetch/compute architecture resolved that gap; this record +confirms the full catalog has now been extracted under that architecture in +production. + +## Open items + +1. The 408 cards with no `ImageEvidence` are unreachable by Stage D unless a + targeted re-extraction pass succeeds for them. No such pass is scheduled; + they are expected to remain unresolved unless their Drive sources are + repaired or re-indexed. All are Google Drive — no other source type + contributes to the gap. diff --git a/docs/reports/README.md b/docs/reports/README.md index 1ea6161ec..f75572390 100644 --- a/docs/reports/README.md +++ b/docs/reports/README.md @@ -73,6 +73,11 @@ a `detail` object read only when the summary signals it's needed. `stage-d-join-key-v1` totals reconciled to 11,905, gate re-derived at 0/12,684 across both the printing- and tag-consensus engines this arc touched. +- `2026-07-26-stagec-full-catalog-completion.md` — Stage C full-catalog + extraction completion: 218,108 / 218,516 cards (99.8%), run_ids + `pass-full-20260725` + `-r2` superseding all prior Stage C last-writer rows, + 1,614 fetch_ok=False rows + 408 cards with no evidence (all Google Drive), + full-catalog fetch-failure rate 0.74%. - `2026-07-22-knowledge-inventory.md` — pipeline-fidelity gate artifact 2 (knowledge-inventory sweep): a constant-by-constant table of every pilot-era value's current home, 3 confirmed MISSING items, 3 open From 0d1c91d6b919775c81366f1a307147ce17b708b8 Mon Sep 17 00:00:00 2001 From: wilfordgrimley <2397930+WilfordGrimley@users.noreply.github.com> Date: Mon, 27 Jul 2026 21:56:46 +0000 Subject: [PATCH 4/5] docs: document web-ci.yml per-surface CI gating (PR #466) --- docs/infrastructure.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/infrastructure.md b/docs/infrastructure.md index 254c5f3ab..5ac2714b6 100644 --- a/docs/infrastructure.md +++ b/docs/infrastructure.md @@ -247,6 +247,18 @@ directly. `coverage-ack: — ` line (same tether discipline as `docs_lint.py`'s in-file `ALLOWLIST`). New tests and un-skipping are always fine. +- `web-ci.yml` per-surface change gating (PR #466): a `changes` job runs + first on every push and diffs `HEAD` against the push's pre-image SHA using + plain `git diff --name-only` (same pattern as other workflows in this repo, + not a paths-filter action). It sets two boolean outputs — `backend` + (`MPCAutofill/**` prefix) and `frontend` (`frontend/**` prefix) — and every + downstream job gates itself on `needs.changes.outputs. == 'true'`. + `cloudflare-static-site/**` is not wired to any surface. Fallback: on + `workflow_dispatch`, a new branch push, or a force-push where no before SHA + is available, both outputs are set to `true` (run everything). A + `test-pre-commit` job that had been duplicated in `web-ci.yml` was removed + in this PR rather than scoped — `test-pre-commit.yml` already runs it + unconditionally. ## Push policy From aa149fe6a6872bf3c0152f4d53281fc6a88f349d Mon Sep 17 00:00:00 2001 From: wilfordgrimley <2397930+WilfordGrimley@users.noreply.github.com> Date: Mon, 27 Jul 2026 21:57:50 +0000 Subject: [PATCH 5/5] docs: fix rejudge_fallback_channel command path (cardpicker/management/commands/) --- docs/features/stage-e-operations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/features/stage-e-operations.md b/docs/features/stage-e-operations.md index ba7d5f488..83ea39be7 100644 --- a/docs/features/stage-e-operations.md +++ b/docs/features/stage-e-operations.md @@ -1065,7 +1065,7 @@ produces an empty `counters={}` record. ## `rejudge_fallback_channel` — fallback-channel compare-and-retract (PR #495) -`MPCAutofill/management/commands/rejudge_fallback_channel.py` re-evaluates the +`MPCAutofill/cardpicker/management/commands/rejudge_fallback_channel.py` re-evaluates the `stage-d-fallback-v1` channel's existing conclusions against **current stored** `ImageEvidence` and retracts the rows where the conclusion changed, making those cards eligible for a fresh `local_calculate_verdicts` pass.