md5 identity groups: checksum substrate, evidence transfer, and group-level vote pooling

[WilfordGrimley]

Owner-ratified design (2026-07-25, conversation record — the four rulings)

1. Group tally semantics: consensus tallies at md5-group level. MACHINE evidence events dedupe to once-per-group (same anonymous_id across siblings = ONE event — max, never sum). HUMAN votes are genuinely independent events and SUM across members. Resolution state propagates to every member.
2. Human dissent within a group is a visible contest on ONE identification target, handled by the standard vote-weight matrix at group level — nothing new invented; byte-identical images are treated as identical, full stop.
3. Singleton degenerate case: a card with null or unique md5 is a group of one; every pooling change must be a provable no-op for it (this is also the core test strategy — today's behavior is the degenerate case).
4. Sequencing: build now, drain the Bug-A tail only after all three PRs land (owner: running extraction before the complete md5 machinery wastes the largest remaining compute spend). The Stage E shakedown driver: route the Bug-A tail through the streaming conveyor #465 shakedown drain is PAUSED at ~1,282/6,531 pending this issue.

Binding soundness rule (already recorded here, restated): an md5 group is ONE identification target; a transferred vote counts ONCE per group. The theory.md independence-assumptions amendment ships IN THE SAME PR as pooling (PR-3), never separately.

Build plan — three sequential PRs

PR-1 — checksum substrate (#442 items 1-2, mechanical): Card.md5_checksum (CharField, db_index, null=True; source = Drive API md5Checksum from the listings we already fetch — read the crawler/update path to find the seam). Backfill management command via a listings re-walk (NO image fetches), forced dry-run-before-write per #373 convention, PilotRunLedger row, prints the measured dupe factor for cross-check against #442's 18.87%/12,275-group walk. LOCAL_FILE and checksum-less sources stay null (= singleton, per ruling 3).

PR-2 — evidence transfer + staleness + fetch fallback (#451 items 1-2): At Stage C (both BULK resume filter seam and streaming _run_stage_c), before fetching: if an md5 sibling holds CURRENT full-manifest evidence, create this card's ImageEvidence row from the sibling's field values (keyed (card, content_hash) as normal — byte-identical file implies identical content_phash; assert, don't assume, at transfer time), run_id marking the transfer provenance. Stamp md5 onto ImageEvidence at extraction AND transfer time; evidence currency additionally requires stored md5 == Card.md5_checksum (closes the silent in-place-file-replacement hole). Fetch fallback: on 404/lockout, try an md5 sibling's source URL — same bytes. INTERIM GUARD until PR-3 lands: cards holding TRANSFERRED evidence are excluded from Stage D machine voting (prevents fabricated-independence votes in the window between PR-2 and PR-3 deploys; the guard and its removal are both part of the record).

PR-3 — group-level vote pooling (PROTECTED CORE) + theory amendment: resolve_weighted_consensus/resolve_and_persist_printing become group-aware per ruling 1 (machine dedupe by anonymous_id → one event at max weight; human votes sum; matrix contest logic runs on the pooled tally; resolution/status caches written for ALL members). consensus_recompute iterates groups. question_feed.is_likely_resolve_printing evaluates the group tally (a group one human vote from resolution is likely-resolve for every member; the feed should serve ONE member per group, not N). Removes PR-2's interim Stage D guard. theory.md: independence-assumptions amendment (same PR) — pooling only DEDUPES weight, never fabricates it, so the false-accept bound is preserved or tightened; state it in §4's mechanism list. Tron §8-style gate MANDATORY before merge; owner reviews the theory text personally.

Out of scope (stay on #451's ranked list)

Grid-selector collapse (frontend, item 4), federation join key (item 5), origin-ordering/source-calibration analytics.

Acceptance

• PR-1: backfill dry-run numbers reconcile with Ingest Drive file checksums; dedupe extraction via byte-identical equivalence classes #442's walk before --write.
• PR-2: transfer produces byte-identical evidence fields; singleton path untouched (full suite green unchanged); staleness invalidation test.
• PR-3: singleton no-op proven (existing consensus tests pass UNMODIFIED); group cases (dedupe/sum/contest/propagation); zero machine-only group resolutions (the human-backed gate at group level); recompute idempotence.
• Drain resumes only after PR-3 deploys.

[WilfordGrimley]

Owner-approved addition (2026-07-25 evening): PR-1 also stores Card.sha256_checksum (same listings walk, same migration). BINDING consequence for PR-2: evidence transfer requires md5 AND sha256 to match whenever sha256 is present on both cards — md5 collisions are constructible, and transfer's 'identical bytes => identical evidence' premise must be cryptographic, not assumed. A sha256 mismatch on an md5 match is a loud anomaly (log + skip transfer + flag), never a silent fallback to md5-only. Groups still key on md5; sha256 is the transfer safety pairing and the future federation join key (#451 item 5).

[WilfordGrimley]

Recompute pre-flight ADDITIONS (Tron delta gate, 2026-07-25, binding on the first group-semantics recompute): (1) the dry-run report must include resolved->unresolved transition counts (pooling can retract per-card resolutions when a self-contradicting voter is withheld), not only unresolved->resolved; (2) size the single-machine-agent blast radius before --apply: count groups where one machine identity (e.g. stage-d-join-key-v1) holds differing outcomes across members — all its machine weight withholds for those groups, which moves likely-resolve/review-queue counts without changing resolutions; get the number into the dry-run report so a queue shift is expected, not a surprise-regression. Both are dry-run reads after PR-1+PR-2 land, before any --apply, which stays owner-gated.

[WilfordGrimley]

OWNER RULING (2026-07-25): evidence transfer runs ENABLED (STAGE_C_EVIDENCE_TRANSFER_ENABLED=True) for the first whole-catalog pass, WITH the strict-source condition (transfer source requires non-null, agreeing md5 stamp — Tron condition 4's fix, not the ruling alternative). Rationale recorded: strict sources make every first-pass transfer source freshly-stamped this pass; residual copy-bug risk is covered by transfer tests + the pilot slice; ON saves 20-40k Google fetches, materially reducing lockout-trip exposure for the overnight run. BINDING pilot condition: the ~100-card pilot slice before the full pass must include transfer cases and verify transferred rows field-identical to donors before the full-pass GO.