Owner, 2026-07-30: "once we nail down the md5 dedupe, I think the image phash should follow the same logic — but put on the board, not a blocker."
Context
Two grouping mechanisms exist today and they are keyed differently:
- md5 (
evidence_transfer) — an md5-identical sibling's evidence is transferred instead of re-fetching. Saves a fetch.
- phash distance-0 (
local_clustering.compute_two_threshold_clusters + build_propagated_cluster_votes) — a distance-0 member inherits a vote. Saves a deduction.
Those draw different sets. Two byte-identical files always share a phash; two files sharing a phash are not necessarily byte-identical. So the group that gets a fetch saved and the group that gets a vote propagated are not the same group, which is hard to reason about and easy to get wrong.
The in-flight work unifies this on md5, the stricter and more defensible key: one fetch per md5 group, one deduction, votes applied across the group.
What this issue is for
Once that lands, revisit whether phash distance-0 should follow the same shape — a single grouping abstraction with the key as a parameter, rather than two mechanisms with different semantics.
Not a blocker, and deliberately second. md5 is exact-identity and its correctness argument is trivial. phash distance-0 is a near-identity claim: local_clustering's own docstring says a distance-0 match "most plausibly means a duplicate/near-identical upload" — plausibly, not certainly. Propagating a vote across a phash group is a stronger claim than propagating across an md5 group, and it deserves its own argument rather than inheriting one.
Open question for whoever picks this up: should phash d=0 propagate votes at all once md5 grouping exists, or is its value only as a narrowing prior for visual disambiguation (which is what the d<=2 band is already used for)? Those are different roles and the answer may be "md5 propagates, phash only narrows."
Related: #660 (the monolith, which wires cluster propagation), #618.
Owner, 2026-07-30: "once we nail down the md5 dedupe, I think the image phash should follow the same logic — but put on the board, not a blocker."
Context
Two grouping mechanisms exist today and they are keyed differently:
evidence_transfer) — an md5-identical sibling's evidence is transferred instead of re-fetching. Saves a fetch.local_clustering.compute_two_threshold_clusters+build_propagated_cluster_votes) — a distance-0 member inherits a vote. Saves a deduction.Those draw different sets. Two byte-identical files always share a phash; two files sharing a phash are not necessarily byte-identical. So the group that gets a fetch saved and the group that gets a vote propagated are not the same group, which is hard to reason about and easy to get wrong.
The in-flight work unifies this on md5, the stricter and more defensible key: one fetch per md5 group, one deduction, votes applied across the group.
What this issue is for
Once that lands, revisit whether phash distance-0 should follow the same shape — a single grouping abstraction with the key as a parameter, rather than two mechanisms with different semantics.
Not a blocker, and deliberately second. md5 is exact-identity and its correctness argument is trivial. phash distance-0 is a near-identity claim:
local_clustering's own docstring says a distance-0 match "most plausibly means a duplicate/near-identical upload" — plausibly, not certainly. Propagating a vote across a phash group is a stronger claim than propagating across an md5 group, and it deserves its own argument rather than inheriting one.Open question for whoever picks this up: should phash d=0 propagate votes at all once md5 grouping exists, or is its value only as a narrowing prior for visual disambiguation (which is what the d<=2 band is already used for)? Those are different roles and the answer may be "md5 propagates, phash only narrows."
Related: #660 (the monolith, which wires cluster propagation), #618.