diff --git a/docs/superpowers/plans/2026-08-17-merge-revamp-slice-1-engine.md b/docs/superpowers/plans/2026-08-17-merge-revamp-slice-1-engine.md new file mode 100644 index 00000000..1b53acf0 --- /dev/null +++ b/docs/superpowers/plans/2026-08-17-merge-revamp-slice-1-engine.md @@ -0,0 +1,2422 @@ +# Merge Revamp — Slice 1: Resolver, Engine and Bug Fixes — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace the two-contact merge engine with an N-way `merge_cluster/4` backed by a shared resolution module, fixing the two data-loss bugs, with no user-visible UI change. + +**Architecture:** A field registry (`MergeFields`) declares what is mergeable and how. A pure resolver (`MergeResolution`) turns a list of member contacts into a complete field map plus conflict and attribution metadata for later UI use. The engine (`Contacts.merge_cluster/4`) takes that map, validates every value against the members, and applies it inside one `Ecto.Multi`. The existing wizard and REST API keep working through a thin `merge_contacts/3` shim, so nothing user-facing changes in this slice. + +**Tech Stack:** Elixir, Ecto, PostgreSQL, Oban, ExUnit. + +**Spec:** `docs/superpowers/specs/2026-08-17-duplicate-merge-revamp-design.md` + +## Global Constraints + +- Account-scoped multitenancy: every query goes through the account. Members must all belong to `scope.account.id`. +- Soft-delete: losers get `deleted_at` set; they are never hard-deleted. +- Never add `Co-Authored-By` lines to commits. +- Run `mix test` before every commit and ensure 0 failures. +- `mix format` after each task; the repo uses `Phoenix.LiveView.HTMLFormatter` for `.heex`. +- Oban is disabled in test env; assert enqueued jobs with `Oban.Testing`. +- One deviation from the spec, deliberate: §3 writes the resolver as `resolve(members)`. It is implemented as `resolve(members, survivor_id)` because the Immich default rule and the `first_met_through_id` self-reference clear both need to know which member survives. + +## File Structure + +| File | Responsibility | +|---|---| +| `lib/kith/contacts/merge_fields.ex` (create) | Declares which contact fields are mergeable and how each is treated. No logic beyond classification. | +| `lib/kith/contacts/merge_resolution.ex` (create) | Pure function: members → resolved field map + conflicts + attributions. No DB writes. | +| `lib/kith/contacts/merge.ex` (create) | The `Ecto.Multi` engine. Called by `Contacts.merge_cluster/4`. Kept out of `contacts.ex`, which is already 2224 lines. | +| `lib/kith/contacts.ex` (modify) | Thin delegation: `merge_cluster/4` → `Merge.run/4`, and `merge_contacts/3` becomes a shim. | +| `lib/kith/duplicate_detection.ex` (modify) | Pair resolution and repointing helpers used by the engine. | +| `lib/kith_web/controllers/api/contact_controller.ex:218` (modify) | Adapt the API merge action to the resolver + engine. | +| `test/kith/contacts/merge_fields_test.exs` (create) | Registry classification and the non-clearable derivation guard. | +| `test/kith/contacts/merge_resolution_test.exs` (create) | Every resolution rule. | +| `test/kith/contacts_merge_test.exs` (modify) | Extended for N-way, the six missing remaps, drop, pairs. | + +--- + +### Task 1: Field registry + +**Files:** +- Create: `lib/kith/contacts/merge_fields.ex` +- Test: `test/kith/contacts/merge_fields_test.exs` + +**Interfaces:** +- Consumes: `Kith.Contacts.Contact` (existing schema). +- Produces: `MergeFields.choice_fields/0`, `policy_fields/0`, `array_fields/0`, `immich_fields/0`, `all/0`, `non_clearable/0`, `non_clearable?/1` — all returning lists of atoms except the last, which returns a boolean. + +- [ ] **Step 1: Write the failing test** + +Create `test/kith/contacts/merge_fields_test.exs`: + +```elixir +defmodule Kith.Contacts.MergeFieldsTest do + use Kith.DataCase, async: true + + alias Kith.Contacts.MergeFields + + describe "classification" do + test "display_name is never mergeable" do + refute :display_name in MergeFields.all() + end + + test "the fields the old wizard exposed are all choice fields" do + for field <- [ + :first_name, + :last_name, + :nickname, + :birthdate, + :description, + :occupation, + :company, + :avatar + ] do + assert field in MergeFields.choice_fields(), "#{field} missing" + end + end + + test "the fields the old wizard hid are now covered" do + for field <- [ + :middle_name, + :gender_id, + :currency_id, + :first_met_at, + :first_met_where, + :first_met_through_id + ] do + assert field in MergeFields.choice_fields(), "#{field} missing" + end + end + + test "policy, array and immich fields are separate from choice fields" do + assert :favorite in MergeFields.policy_fields() + assert :aliases in MergeFields.array_fields() + assert :immich_person_id in MergeFields.immich_fields() + + refute :favorite in MergeFields.choice_fields() + refute :aliases in MergeFields.choice_fields() + refute :immich_person_id in MergeFields.choice_fields() + end + + test "all/0 is the union of the four groups" do + expected = + MergeFields.choice_fields() ++ + MergeFields.policy_fields() ++ + MergeFields.array_fields() ++ MergeFields.immich_fields() + + assert Enum.sort(MergeFields.all()) == Enum.sort(expected) + end + end + + describe "non_clearable" do + test "matches the contact update changeset's required fields" do + required = + %Kith.Contacts.Contact{} + |> Kith.Contacts.Contact.update_changeset(%{}) + |> Map.fetch!(:required) + + assert Enum.sort(MergeFields.non_clearable()) == Enum.sort(required) + end + + test "first_name cannot be cleared" do + assert MergeFields.non_clearable?(:first_name) + refute MergeFields.non_clearable?(:nickname) + end + end +end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts/merge_fields_test.exs` +Expected: FAIL with `module Kith.Contacts.MergeFields is not available`. + +- [ ] **Step 3: Write minimal implementation** + +Create `lib/kith/contacts/merge_fields.ex`: + +```elixir +defmodule Kith.Contacts.MergeFields do + @moduledoc """ + Declares which `Kith.Contacts.Contact` fields participate in a merge and how + each is treated. + + Four groups: + + * `choice_fields` — ordinary scalars the user picks between. + * `policy_fields` — booleans and their companions, resolved by policy + rather than by choice (see `Kith.Contacts.MergeResolution`). + * `array_fields` — array columns resolved as a deduplicated union. + * `immich_fields` — integration state that moves as one unit. + + `display_name` is deliberately absent: it is computed by + `Contact.compute_display_name/1` and recomputed asynchronously by + `Kith.Workers.DisplayNameRecomputeWorker`. + """ + + alias Kith.Contacts.Contact + + @choice_fields ~w( + first_name middle_name last_name nickname + birthdate birthdate_year_unknown + description avatar occupation company + gender_id currency_id + first_met_at first_met_year_unknown first_met_where + first_met_through_id first_met_additional_info + )a + + @policy_fields ~w(favorite is_archived deceased deceased_at)a + @array_fields ~w(aliases)a + @immich_fields ~w(immich_person_id immich_person_url immich_status immich_last_synced_at)a + + @doc "Scalars the user picks between." + def choice_fields, do: @choice_fields + + @doc "Fields resolved by policy rather than by user choice." + def policy_fields, do: @policy_fields + + @doc "Array columns resolved as a deduplicated union." + def array_fields, do: @array_fields + + @doc "Immich integration state, moved as a single unit." + def immich_fields, do: @immich_fields + + @doc "Every mergeable field." + def all, do: @choice_fields ++ @policy_fields ++ @array_fields ++ @immich_fields + + @doc """ + Fields that cannot be set to `:clear`, derived from the contact update + changeset's `validate_required/2` list rather than maintained by hand. + """ + def non_clearable do + %Contact{} + |> Contact.update_changeset(%{}) + |> Map.fetch!(:required) + end + + @doc "Whether `field` may be cleared." + def non_clearable?(field), do: field in non_clearable() +end +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/contacts/merge_fields_test.exs` +Expected: PASS, 6 tests. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/contacts/merge_fields.ex test/kith/contacts/merge_fields_test.exs +git commit -m "feat(contacts): add merge field registry" +``` + +--- + +### Task 2: Resolver — scalar fields + +**Files:** +- Create: `lib/kith/contacts/merge_resolution.ex` +- Test: `test/kith/contacts/merge_resolution_test.exs` + +**Interfaces:** +- Consumes: `MergeFields.choice_fields/0`. +- Produces: `MergeResolution.resolve(members, survivor_id)` returning + `%MergeResolution{fields: map, conflicts: map, attributions: map}`. + `fields` maps field atom → concrete value or `:clear`. + `conflicts` maps field atom → list of `%{value: term, member_ids: [integer], count: integer}`, present only for contested fields. + `attributions` maps field atom → `:all_agree | {:only, contact_id} | {:some, count} | :none`. + +- [ ] **Step 1: Write the failing test** + +Create `test/kith/contacts/merge_resolution_test.exs`: + +```elixir +defmodule Kith.Contacts.MergeResolutionTest do + use Kith.DataCase, async: false + + alias Kith.Contacts.MergeResolution + alias Kith.ContactsFixtures + alias Kith.AccountsFixtures + + setup do + ContactsFixtures.seed_reference_data!() + user = AccountsFixtures.user_fixture() + %{account_id: user.account_id} + end + + defp contact(account_id, attrs), do: ContactsFixtures.contact_fixture(account_id, attrs) + + describe "scalar fields" do + test "no member holds a value — field is cleared, no attribution", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", occupation: nil}) + b = contact(ctx.account_id, %{first_name: "Sarah", occupation: nil}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.occupation == :clear + assert res.attributions.occupation == :none + refute Map.has_key?(res.conflicts, :occupation) + end + + test "every member agrees — value kept, attributed to all", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah"}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.first_name == "Sarah" + assert res.attributions.first_name == :all_agree + end + + test "only one member holds a value — gap is filled", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", middle_name: nil}) + b = contact(ctx.account_id, %{first_name: "Sarah", middle_name: "Jiyoung"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.middle_name == "Jiyoung" + assert res.attributions.middle_name == {:only, b.id} + refute Map.has_key?(res.conflicts, :middle_name) + end + + test "whitespace-only differences are not a conflict", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Figma"}) + b = contact(ctx.account_id, %{first_name: "Sarah", company: " Figma "}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.company == "Figma" + assert res.attributions.company == :all_agree + end + + test "disagreement is a conflict, defaulting to the most-held value", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Figma"}) + b = contact(ctx.account_id, %{first_name: "Sarah", company: "Stripe"}) + c = contact(ctx.account_id, %{first_name: "Sarah", company: "Stripe"}) + + res = MergeResolution.resolve([a, b, c], a.id) + + assert res.fields.company == "Stripe" + assert res.attributions.company == {:some, 2} + + candidates = res.conflicts.company + assert length(candidates) == 2 + stripe = Enum.find(candidates, &(&1.value == "Stripe")) + assert stripe.count == 2 + assert Enum.sort(stripe.member_ids) == Enum.sort([b.id, c.id]) + end + + test "a tied conflict breaks toward the most recently updated member", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Figma"}) + b = contact(ctx.account_id, %{first_name: "Sarah", company: "Stripe"}) + + # Make b unambiguously newer. + b = + b + |> Ecto.Changeset.change(%{updated_at: DateTime.add(a.updated_at, 3600, :second)}) + |> Repo.update!() + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.company == "Stripe" + end + + test "resolving a single member is a no-op copy", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Figma"}) + + res = MergeResolution.resolve([a], a.id) + + assert res.fields.first_name == "Sarah" + assert res.fields.company == "Figma" + assert res.conflicts == %{} + end + end +end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts/merge_resolution_test.exs` +Expected: FAIL with `module Kith.Contacts.MergeResolution is not available`. + +- [ ] **Step 3: Write minimal implementation** + +Create `lib/kith/contacts/merge_resolution.ex`: + +```elixir +defmodule Kith.Contacts.MergeResolution do + @moduledoc """ + Turns a set of member contacts into the complete field map a merge will apply. + + Pure: reads the structs it is given and writes nothing. This is the single + implementation of merge resolution — the cluster LiveView calls it to render, + the REST API adapter calls it to build its payload, and the engine never calls + it at all. That separation is what makes a merge apply exactly what the user + approved: the engine validates a concrete map rather than re-deriving one at + transaction time. + + Resolution is recomputed from scratch whenever the member selection changes; + explicit user choices are not carried across a change. + """ + + alias Kith.Contacts.MergeFields + + defstruct fields: %{}, conflicts: %{}, attributions: %{} + + @type t :: %__MODULE__{ + fields: %{atom() => term() | :clear}, + conflicts: %{atom() => [%{value: term(), member_ids: [integer()], count: integer()}]}, + attributions: %{atom() => :all_agree | {:only, integer()} | {:some, integer()} | :none} + } + + @doc """ + Resolves `members` into a complete field map. + + `survivor_id` must be the id of one of `members`. It is needed by rules that + are relative to the surviving record rather than to the set as a whole. + """ + def resolve(members, _survivor_id) when is_list(members) and members != [] do + Enum.reduce(MergeFields.choice_fields(), %__MODULE__{}, fn field, acc -> + put_resolution(acc, field, resolve_scalar(members, field)) + end) + end + + defp put_resolution(acc, field, {value, attribution, candidates}) do + acc = %{ + acc + | fields: Map.put(acc.fields, field, value), + attributions: Map.put(acc.attributions, field, attribution) + } + + if candidates == [] do + acc + else + %{acc | conflicts: Map.put(acc.conflicts, field, candidates)} + end + end + + # Returns {resolved_value, attribution, conflict_candidates} + defp resolve_scalar(members, field) do + held = + members + |> Enum.map(fn member -> {member.id, normalize(Map.fetch!(member, field))} end) + |> Enum.reject(fn {_id, value} -> is_nil(value) end) + + case held |> Enum.map(&elem(&1, 1)) |> Enum.uniq() do + [] -> + {:clear, :none, []} + + [only] -> + {only, attribution(members, held), []} + + _many -> + candidates = candidates(held) + {default_value(members, candidates), attribution(members, held), candidates} + end + end + + defp normalize(value) when is_binary(value) do + case String.trim(value) do + "" -> nil + trimmed -> trimmed + end + end + + defp normalize(value), do: value + + defp attribution(members, held) do + case length(held) do + 0 -> :none + 1 -> {:only, held |> hd() |> elem(0)} + n when n == length(members) -> :all_agree + n -> {:some, n} + end + end + + defp candidates(held) do + held + |> Enum.group_by(fn {_id, value} -> value end, fn {id, _value} -> id end) + |> Enum.map(fn {value, ids} -> %{value: value, member_ids: ids, count: length(ids)} end) + end + + # Most-held value wins; ties break toward the most recently updated member + # holding it. + defp default_value(members, candidates) do + updated_at = Map.new(members, &{&1.id, &1.updated_at}) + + candidates + |> Enum.max_by(fn %{member_ids: ids, count: count} -> + newest = + ids + |> Enum.map(&Map.fetch!(updated_at, &1)) + |> Enum.max(DateTime) + |> DateTime.to_unix() + + {count, newest} + end) + |> Map.fetch!(:value) + end +end +``` + +`survivor_id` is unused at this stage — hence the underscore. Task 3 introduces +the two rules that need it (the Immich default and the `first_met_through_id` +self-reference clear) and drops the underscore then. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/contacts/merge_resolution_test.exs` +Expected: PASS, 7 tests. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/contacts/merge_resolution.ex test/kith/contacts/merge_resolution_test.exs +git commit -m "feat(contacts): resolve scalar fields across merge members" +``` + +--- + +### Task 3: Resolver — arrays, associations, policy flags and Immich + +**Files:** +- Modify: `lib/kith/contacts/merge_resolution.ex` +- Test: `test/kith/contacts/merge_resolution_test.exs` + +**Interfaces:** +- Consumes: `MergeResolution.resolve/2` from Task 2. +- Produces: the same struct, with `fields` additionally covering `aliases`, the four policy fields and the four Immich fields, and with `first_met_through_id` cleared when it points at a merged member. + +- [ ] **Step 1: Write the failing test** + +Append these describe blocks to `test/kith/contacts/merge_resolution_test.exs`: + +```elixir + describe "array fields" do + test "aliases are unioned and deduplicated", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", aliases: ["Sarah K.", "SK"]}) + b = contact(ctx.account_id, %{first_name: "Sarah", aliases: ["SK", "김지영"]}) + + res = MergeResolution.resolve([a, b], a.id) + + assert Enum.sort(res.fields.aliases) == Enum.sort(["Sarah K.", "SK", "김지영"]) + end + + test "no aliases anywhere resolves to an empty list", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah"}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.aliases == [] + end + end + + describe "association ids" do + test "gender_id resolves like any other scalar", ctx do + gender = Repo.one!(from(g in Kith.Contacts.Gender, limit: 1)) + a = contact(ctx.account_id, %{first_name: "Sarah", gender_id: nil}) + b = contact(ctx.account_id, %{first_name: "Sarah", gender_id: gender.id}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.gender_id == gender.id + end + + test "first_met_through pointing at a merged member is cleared", ctx do + b = contact(ctx.account_id, %{first_name: "Sarah"}) + a = contact(ctx.account_id, %{first_name: "Sarah", first_met_through_id: b.id}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.first_met_through_id == :clear + end + + test "first_met_through pointing outside the cluster is kept", ctx do + outsider = contact(ctx.account_id, %{first_name: "Dana"}) + a = contact(ctx.account_id, %{first_name: "Sarah", first_met_through_id: outsider.id}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.first_met_through_id == outsider.id + end + end + + describe "policy fields" do + test "favorite is true if any member is favorited", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", favorite: false}) + b = contact(ctx.account_id, %{first_name: "Sarah", favorite: true}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.favorite == true + end + + test "is_archived is false if any member is active", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", is_archived: true}) + b = contact(ctx.account_id, %{first_name: "Sarah", is_archived: false}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.is_archived == false + end + + test "deceased_at takes the earliest non-nil date", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", deceased: true, deceased_at: ~D[2024-05-01]}) + b = contact(ctx.account_id, %{first_name: "Sarah", deceased: true, deceased_at: ~D[2023-02-11]}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.deceased == true + assert res.fields.deceased_at == ~D[2023-02-11] + end + + test "deceased with no dates anywhere leaves deceased_at nil", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", deceased: true, deceased_at: nil}) + b = contact(ctx.account_id, %{first_name: "Sarah", deceased: false}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.deceased == true + assert res.fields.deceased_at == :clear + end + end + + describe "immich fields" do + test "the survivor's link wins when it has one", ctx do + a = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "survivor-person", + immich_status: "linked" + }) + + b = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "loser-person", + immich_status: "linked" + }) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.immich_person_id == "survivor-person" + end + + test "an unlinked survivor adopts the only linked member's whole group", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", immich_person_id: nil}) + + b = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "loser-person", + immich_person_url: "https://immich.example/people/loser-person", + immich_status: "linked" + }) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.immich_person_id == "loser-person" + assert res.fields.immich_person_url == "https://immich.example/people/loser-person" + assert res.fields.immich_status == "linked" + end + + test "with several linked members the most recently synced wins", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", immich_person_id: nil}) + + b = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "older", + immich_status: "linked", + immich_last_synced_at: ~U[2025-01-01 00:00:00Z] + }) + + c = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "newer", + immich_status: "linked", + immich_last_synced_at: ~U[2026-01-01 00:00:00Z] + }) + + res = MergeResolution.resolve([a, b, c], a.id) + + assert res.fields.immich_person_id == "newer" + end + + test "nobody linked leaves the group cleared", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah"}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.immich_person_id == :clear + end + end +``` + +Add `import Ecto.Query` to the top of the test module, below `use Kith.DataCase`. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts/merge_resolution_test.exs` +Expected: FAIL — `res.fields.aliases` raises `KeyError` because only choice fields are resolved so far. + +- [ ] **Step 3: Write minimal implementation** + +In `lib/kith/contacts/merge_resolution.ex`, replace `resolve/2` and add the new +private functions below it: + +```elixir + def resolve(members, survivor_id) when is_list(members) and members != [] do + member_ids = MapSet.new(members, & &1.id) + + MergeFields.choice_fields() + |> Enum.reduce(%__MODULE__{}, fn field, acc -> + put_resolution(acc, field, resolve_scalar(members, field)) + end) + |> clear_self_reference(member_ids) + |> resolve_arrays(members) + |> resolve_policy_fields(members) + |> resolve_immich(members, survivor_id) + end + + # A contact cannot be met through a record it just absorbed. + defp clear_self_reference(acc, member_ids) do + case Map.get(acc.fields, :first_met_through_id) do + id when is_integer(id) -> + if MapSet.member?(member_ids, id) do + %{acc | fields: Map.put(acc.fields, :first_met_through_id, :clear)} + else + acc + end + + _ -> + acc + end + end + + defp resolve_arrays(acc, members) do + Enum.reduce(MergeFields.array_fields(), acc, fn field, acc -> + union = + members + |> Enum.flat_map(&(Map.fetch!(&1, field) || [])) + |> Enum.uniq() + + %{acc | fields: Map.put(acc.fields, field, union)} + end) + end + + defp resolve_policy_fields(acc, members) do + favorite = Enum.any?(members, & &1.favorite) + is_archived = not Enum.any?(members, &(not &1.is_archived)) + deceased = Enum.any?(members, & &1.deceased) + + deceased_at = + members + |> Enum.filter(& &1.deceased) + |> Enum.map(& &1.deceased_at) + |> Enum.reject(&is_nil/1) + |> case do + [] -> :clear + dates -> Enum.min(dates, Date) + end + + fields = + acc.fields + |> Map.put(:favorite, favorite) + |> Map.put(:is_archived, is_archived) + |> Map.put(:deceased, deceased) + |> Map.put(:deceased_at, deceased_at) + + %{acc | fields: fields} + end + + # The four Immich columns move together: an id from one record paired with + # another record's sync timestamp is corrupt state. + defp resolve_immich(acc, members, survivor_id) do + linked = Enum.filter(members, &(not is_nil(&1.immich_person_id))) + survivor = Enum.find(members, &(&1.id == survivor_id)) + + source = + cond do + survivor && survivor.immich_person_id -> survivor + linked == [] -> nil + true -> Enum.max_by(linked, &sync_key/1) + end + + fields = + Enum.reduce(MergeFields.immich_fields(), acc.fields, fn field, fields -> + value = if source, do: Map.fetch!(source, field), else: nil + Map.put(fields, field, value || :clear) + end) + + %{acc | fields: fields} + end + + defp sync_key(%{immich_last_synced_at: nil}), do: 0 + defp sync_key(%{immich_last_synced_at: at}), do: DateTime.to_unix(at) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/contacts/merge_resolution_test.exs` +Expected: PASS, 19 tests. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/contacts/merge_resolution.ex test/kith/contacts/merge_resolution_test.exs +git commit -m "feat(contacts): resolve arrays, associations, flags and immich state" +``` + +--- + +### Task 4: Engine skeleton — lock, validate, apply scalars, soft-delete + +**Files:** +- Create: `lib/kith/contacts/merge.ex` +- Modify: `lib/kith/contacts.ex` (add `merge_cluster/4` delegation) +- Test: `test/kith/contacts_merge_test.exs` + +**Interfaces:** +- Consumes: `MergeFields.non_clearable?/1`, `MergeResolution` struct shape. +- Produces: `Kith.Contacts.merge_cluster(scope, survivor_id, loser_ids, resolution)` returning `{:ok, %Contact{}}` or `{:error, reason}` where reason is one of `:not_found`, `:trashed`, `:different_accounts`, `:survivor_in_losers`, `:no_losers`, `{:unknown_value, field}`, `{:not_clearable, field}`. `resolution` is a plain map `%{fields: map, drop: map}`. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith/contacts_merge_test.exs`: + +```elixir + describe "merge_cluster/4 validation" do + setup ctx do + scope = Kith.Accounts.Scope.for_user(ctx.user) + %{scope: scope} + end + + defp resolution(fields \\ %{}), do: %{fields: fields, drop: %{}} + + test "merges three contacts into one survivor", ctx do + c = + Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Alice", + nickname: "Al" + }) + + {:ok, survivor} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id, c.id], + resolution(%{first_name: "Alice", nickname: "Al", company: "New Corp"}) + ) + + assert survivor.id == ctx.contact_a.id + assert survivor.nickname == "Al" + assert survivor.company == "New Corp" + + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id).deleted_at != nil + assert Repo.get!(Kith.Contacts.Contact, c.id).deleted_at != nil + end + + test "clears a field set to :clear", ctx do + {:ok, survivor} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution(%{occupation: :clear}) + ) + + assert survivor.occupation == nil + end + + test "rejects clearing a required field", ctx do + assert {:error, {:not_clearable, :first_name}} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution(%{first_name: :clear}) + ) + end + + test "rejects a value no member holds", ctx do + assert {:error, {:unknown_value, :company}} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution(%{company: "Never Corp"}) + ) + end + + test "rejects an empty loser list", ctx do + assert {:error, :no_losers} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [], resolution()) + end + + test "rejects the survivor appearing among the losers", ctx do + assert {:error, :survivor_in_losers} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_a.id], + resolution() + ) + end + + test "rejects a contact from another account", ctx do + other_user = Kith.AccountsFixtures.user_fixture() + + stranger = + Kith.ContactsFixtures.contact_fixture(other_user.account_id, %{first_name: "Zed"}) + + assert {:error, :different_accounts} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [stranger.id], + resolution() + ) + end + + test "rejects an already-trashed member", ctx do + {:ok, _} = + ctx.contact_b + |> Ecto.Changeset.change(%{deleted_at: DateTime.utc_now(:second)}) + |> Repo.update() + + assert {:error, :trashed} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution() + ) + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts_merge_test.exs` +Expected: FAIL with `function Kith.Contacts.merge_cluster/4 is undefined`. + +- [ ] **Step 3: Write minimal implementation** + +Create `lib/kith/contacts/merge.ex`: + +```elixir +defmodule Kith.Contacts.Merge do + @moduledoc """ + The N-way contact merge engine. + + Applies a resolution it is handed; it never derives one. Everything the merge + writes was computed by `Kith.Contacts.MergeResolution` and approved by the + caller, so a concurrent edit between resolution and submission fails + validation rather than silently changing the result. + """ + + import Ecto.Query, warn: false + + alias Ecto.Multi + alias Kith.Contacts.{Contact, MergeFields} + alias Kith.Repo + + @doc """ + Merges `loser_ids` into `survivor_id` inside one transaction. + + `resolution` is `%{fields: %{atom => value | :clear}, drop: %{atom => [id]}}`. + """ + def run(scope, survivor_id, loser_ids, resolution) do + with {:ok, members} <- lock_and_load(scope, survivor_id, loser_ids), + survivor = Enum.find(members, &(&1.id == survivor_id)), + :ok <- validate_fields(members, resolution) do + losers = Enum.reject(members, &(&1.id == survivor_id)) + + Multi.new() + |> Multi.run(:survivor, fn _repo, _changes -> + apply_fields(survivor, resolution) + end) + |> Multi.run(:soft_delete_losers, fn repo, _changes -> + now = DateTime.utc_now(:second) + ids = Enum.map(losers, & &1.id) + + {count, _} = + repo.update_all(from(c in Contact, where: c.id in ^ids), set: [deleted_at: now]) + + {:ok, count} + end) + |> Repo.transaction() + |> case do + {:ok, %{survivor: survivor}} -> {:ok, survivor} + {:error, _step, reason, _changes} -> {:error, reason} + end + end + end + + defp lock_and_load(scope, survivor_id, loser_ids) do + account_id = scope.account.id + ids = [survivor_id | loser_ids] + + cond do + loser_ids == [] -> + {:error, :no_losers} + + survivor_id in loser_ids -> + {:error, :survivor_in_losers} + + true -> + # FOR UPDATE, not just a liveness check: two sessions merging + # overlapping clusters must serialise rather than interleave. + members = + from(c in Contact, where: c.id in ^ids, lock: "FOR UPDATE") + |> Repo.all() + + cond do + length(members) != length(Enum.uniq(ids)) -> {:error, :not_found} + Enum.any?(members, &(&1.account_id != account_id)) -> {:error, :different_accounts} + Enum.any?(members, &(&1.deleted_at != nil)) -> {:error, :trashed} + true -> {:ok, members} + end + end + end + + defp validate_fields(members, %{fields: fields}) do + Enum.reduce_while(fields, :ok, fn {field, value}, :ok -> + cond do + value == :clear and MergeFields.non_clearable?(field) -> + {:halt, {:error, {:not_clearable, field}}} + + value == :clear -> + {:cont, :ok} + + held_by_member?(members, field, value) -> + {:cont, :ok} + + true -> + {:halt, {:error, {:unknown_value, field}}} + end + end) + end + + # Array and policy fields are computed rather than picked, so they are not + # required to match a single member's stored value. + defp held_by_member?(_members, field, _value) + when field in [:favorite, :is_archived, :deceased, :deceased_at, :aliases], + do: true + + defp held_by_member?(members, field, value) do + Enum.any?(members, fn member -> + stored = Map.fetch!(member, field) + stored == value or (is_binary(stored) and String.trim(stored) == value) + end) + end + + defp apply_fields(survivor, %{fields: fields}) do + changes = + Map.new(fields, fn + {field, :clear} -> {field, nil} + {field, value} -> {field, value} + end) + + survivor + |> Contact.update_changeset(changes) + |> Repo.update() + end +end +``` + +In `lib/kith/contacts.ex`, add to the alias block (`Kith.Contacts.{...}`) the +entry `Merge`, then add this function immediately above the existing +`def merge_contacts(` definition at line 1699: + +```elixir + @doc """ + Merges `loser_ids` into `survivor_id`, applying `resolution`. + + See `Kith.Contacts.MergeResolution` for how a resolution is produced. + """ + def merge_cluster(scope, survivor_id, loser_ids, resolution) do + Merge.run(scope, survivor_id, loser_ids, resolution) + end +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/contacts_merge_test.exs` +Expected: PASS — the 8 new tests plus the existing `merge_contacts/3` tests still green. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/contacts/merge.ex lib/kith/contacts.ex test/kith/contacts_merge_test.exs +git commit -m "feat(contacts): add N-way merge engine with locking and validation" +``` + +--- + +### Task 5: Remap every owned record, including the six Bug 1 misses + +**Files:** +- Modify: `lib/kith/contacts/merge.ex` +- Test: `test/kith/contacts_merge_test.exs` + +**Interfaces:** +- Consumes: `Merge.run/4` from Task 4. +- Produces: no new public function; `run/4` now moves every record owned by a loser onto the survivor. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith/contacts_merge_test.exs`: + +```elixir + describe "merge_cluster/4 remapping" do + setup ctx do + scope = Kith.Accounts.Scope.for_user(ctx.user) + %{scope: scope} + end + + test "moves the six record types the old engine orphaned", ctx do + account_id = ctx.account_id + loser_id = ctx.contact_b.id + + Repo.insert!(%Kith.Contacts.Debt{ + account_id: account_id, + contact_id: loser_id, + amount: Decimal.new("25.00"), + direction: "owed_to_user", + status: "outstanding" + }) + + Repo.insert!(%Kith.Contacts.Gift{ + account_id: account_id, + contact_id: loser_id, + name: "Book", + direction: "given", + status: "given" + }) + + Repo.insert!(%Kith.Contacts.Pet{ + account_id: account_id, + contact_id: loser_id, + name: "Mochi" + }) + + Repo.insert!(%Kith.Tasks.Task{ + account_id: account_id, + contact_id: loser_id, + title: "Call back" + }) + + Repo.insert!(%Kith.Conversations.Conversation{ + account_id: account_id, + contact_id: loser_id, + happened_at: ~D[2026-01-05] + }) + + Repo.insert!(%Kith.Contacts.ImmichCandidate{ + account_id: account_id, + contact_id: loser_id, + immich_person_id: "person-1" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [loser_id], %{ + fields: %{}, + drop: %{} + }) + + for schema <- [ + Kith.Contacts.Debt, + Kith.Contacts.Gift, + Kith.Contacts.Pet, + Kith.Tasks.Task, + Kith.Conversations.Conversation, + Kith.Contacts.ImmichCandidate + ] do + assert Repo.aggregate( + from(r in schema, where: r.contact_id == ^survivor.id), + :count + ) == 1, + "#{inspect(schema)} was not remapped" + + assert Repo.aggregate( + from(r in schema, where: r.contact_id == ^loser_id), + :count + ) == 0, + "#{inspect(schema)} left behind on the loser" + end + end + + test "repoints inbound first_met_through references", ctx do + admirer = + Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Dana", + first_met_through_id: ctx.contact_b.id + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.get!(Kith.Contacts.Contact, admirer.id).first_met_through_id == survivor.id + end + + test "moves notes and addresses from every loser", ctx do + c = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Alice"}) + + Kith.ContactsFixtures.note_fixture(ctx.contact_b, ctx.user.id) + Kith.ContactsFixtures.note_fixture(c, ctx.user.id) + Kith.ContactsFixtures.address_fixture(c) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id, c.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.aggregate( + from(n in Kith.Contacts.Note, where: n.contact_id == ^survivor.id), + :count + ) == 2 + + assert Repo.aggregate( + from(a in Kith.Contacts.Address, where: a.contact_id == ^survivor.id), + :count + ) == 1 + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts_merge_test.exs -k "remapping"` +Expected: FAIL — `Kith.Contacts.Debt was not remapped`, because `run/4` does no remapping yet. + +- [ ] **Step 3: Write minimal implementation** + +In `lib/kith/contacts/merge.ex`, add the schema list near the top of the module, +below the aliases: + +```elixir + # Every schema owning a contact_id that must follow the survivor. The six + # after `Reminder` are the ones the previous two-contact engine silently + # orphaned (Bug 1 in the design spec). + @owned_schemas [ + Kith.Contacts.Note, + Kith.Contacts.Address, + Kith.Contacts.ContactField, + Kith.Contacts.Document, + Kith.Contacts.Photo, + Kith.Activities.Call, + Kith.Activities.LifeEvent, + Kith.Reminders.Reminder, + Kith.Reminders.ReminderInstance, + Kith.Contacts.Debt, + Kith.Contacts.Gift, + Kith.Contacts.Pet, + Kith.Tasks.Task, + Kith.Conversations.Conversation, + Kith.Contacts.ImmichCandidate + ] +``` + +Then insert a remap step into the `Multi` in `run/4`, between `:survivor` and +`:soft_delete_losers`: + +```elixir + |> Multi.run(:remap_owned, fn repo, _changes -> + loser_ids = Enum.map(losers, & &1.id) + + Enum.each(@owned_schemas, fn schema -> + repo.update_all( + from(r in schema, where: r.contact_id in ^loser_ids), + set: [contact_id: survivor.id] + ) + end) + + {:ok, length(@owned_schemas)} + end) + |> Multi.run(:remap_activity_contacts, fn repo, _changes -> + loser_ids = Enum.map(losers, & &1.id) + + # Drop join rows that would collide with one the survivor already has, + # then move the rest. + repo.query!( + """ + DELETE FROM activity_contacts + WHERE contact_id = ANY($1) + AND activity_id IN (SELECT activity_id FROM activity_contacts WHERE contact_id = $2) + """, + [loser_ids, survivor.id] + ) + + repo.update_all( + from(ac in "activity_contacts", where: ac.contact_id in ^loser_ids), + set: [contact_id: survivor.id] + ) + + {:ok, :done} + end) + |> Multi.run(:remap_inbound_first_met, fn repo, _changes -> + loser_ids = Enum.map(losers, & &1.id) + + {count, _} = + repo.update_all( + from(c in Contact, where: c.first_met_through_id in ^loser_ids), + set: [first_met_through_id: survivor.id] + ) + + {:ok, count} + end) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/contacts_merge_test.exs` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/contacts/merge.ex test/kith/contacts_merge_test.exs +git commit -m "fix(contacts): remap debts, gifts, pets, tasks, conversations and immich candidates on merge" +``` + +--- + +### Task 6: Deduplication and relationships + +**Files:** +- Modify: `lib/kith/contacts/merge.ex` +- Test: `test/kith/contacts_merge_test.exs` + +**Interfaces:** +- Consumes: `Merge.run/4` from Task 5. +- Produces: no new public function; duplicate contact fields, tags, photos and relationships collapse after remapping. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith/contacts_merge_test.exs`: + +```elixir + describe "merge_cluster/4 deduplication" do + setup ctx do + scope = Kith.Accounts.Scope.for_user(ctx.user) + email_type = Repo.one!(from(t in Kith.Contacts.ContactFieldType, where: like(t.protocol, "mailto%"), limit: 1)) + %{scope: scope, email_type: email_type} + end + + test "identical contact fields collapse to one", ctx do + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_a, ctx.email_type.id, %{ + value: "sarah@example.com" + }) + + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_b, ctx.email_type.id, %{ + value: "sarah@example.com" + }) + + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_b, ctx.email_type.id, %{ + value: "other@example.com" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + values = + from(f in Kith.Contacts.ContactField, + where: f.contact_id == ^survivor.id, + select: f.value + ) + |> Repo.all() + |> Enum.sort() + + assert values == ["other@example.com", "sarah@example.com"] + end + + test "a tag on both members is kept once", ctx do + {:ok, tag} = Contacts.create_tag(ctx.account_id, %{name: "Design"}) + {:ok, _} = Contacts.add_tag_to_contact(ctx.contact_a, tag) + {:ok, _} = Contacts.add_tag_to_contact(ctx.contact_b, tag) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + count = + from(ct in "contact_tags", where: ct.contact_id == ^survivor.id, select: count()) + |> Repo.one() + + assert count == 1 + end + + test "a relationship between two merged members is removed, not self-referential", ctx do + type = Repo.one!(from(t in Kith.Contacts.RelationshipType, limit: 1)) + Kith.ContactsFixtures.relationship_fixture(ctx.contact_a, ctx.contact_b, type.id) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.aggregate( + from(r in Kith.Contacts.Relationship, + where: r.contact_id == ^survivor.id and r.related_contact_id == ^survivor.id + ), + :count + ) == 0 + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts_merge_test.exs -k "deduplication"` +Expected: FAIL — the contact-fields test finds three values, and the relationship test finds one self-referential row. + +- [ ] **Step 3: Write minimal implementation** + +In `lib/kith/contacts/merge.ex`, add a dedupe step to the `Multi` immediately +after `:remap_inbound_first_met`: + +```elixir + |> Multi.run(:dedupe_owned, fn repo, _changes -> + # Contact fields: same type and value collapse to the lowest id. + repo.query!( + """ + DELETE FROM contact_fields + WHERE id IN ( + SELECT cf.id FROM contact_fields cf + WHERE cf.contact_id = $1 + AND EXISTS ( + SELECT 1 FROM contact_fields other + WHERE other.contact_id = $1 + AND other.contact_field_type_id = cf.contact_field_type_id + AND other.value = cf.value + AND other.id < cf.id + ) + ) + """, + [survivor.id] + ) + + # Tags: the join table has no id, so dedupe on (contact_id, tag_id). + repo.query!( + """ + DELETE FROM contact_tags a + USING contact_tags b + WHERE a.contact_id = $1 + AND b.contact_id = $1 + AND a.tag_id = b.tag_id + AND a.ctid > b.ctid + """, + [survivor.id] + ) + + # Photos: identical content collapses. + repo.query!( + """ + DELETE FROM photos + WHERE id IN ( + SELECT p.id FROM photos p + WHERE p.contact_id = $1 + AND p.content_hash IS NOT NULL + AND EXISTS ( + SELECT 1 FROM photos other + WHERE other.contact_id = $1 + AND other.content_hash = p.content_hash + AND other.id < p.id + ) + ) + """, + [survivor.id] + ) + + {:ok, :done} + end) + |> Multi.run(:remap_relationships, fn repo, _changes -> + loser_ids = Enum.map(losers, & &1.id) + remap_relationships(repo, survivor.id, loser_ids) + end) +``` + +Add this private function at the bottom of the module: + +```elixir + # Relationships are directional and unique per (pair, type), so both + # directions need collision handling before the move, and any relationship + # that ends up pointing the survivor at itself is dropped. + defp remap_relationships(repo, survivor_id, loser_ids) do + repo.query!( + """ + DELETE FROM relationships + WHERE contact_id = ANY($1) + AND ( + related_contact_id = $2 + OR (related_contact_id, relationship_type_id) IN ( + SELECT related_contact_id, relationship_type_id + FROM relationships WHERE contact_id = $2 + ) + ) + """, + [loser_ids, survivor_id] + ) + + repo.update_all( + from(r in Kith.Contacts.Relationship, where: r.contact_id in ^loser_ids), + set: [contact_id: survivor_id] + ) + + repo.query!( + """ + DELETE FROM relationships + WHERE related_contact_id = ANY($1) + AND ( + contact_id = $2 + OR (contact_id, relationship_type_id) IN ( + SELECT contact_id, relationship_type_id + FROM relationships WHERE related_contact_id = $2 + ) + ) + """, + [loser_ids, survivor_id] + ) + + repo.update_all( + from(r in Kith.Contacts.Relationship, where: r.related_contact_id in ^loser_ids), + set: [related_contact_id: survivor_id] + ) + + repo.delete_all( + from(r in Kith.Contacts.Relationship, + where: r.contact_id == ^survivor_id and r.related_contact_id == ^survivor_id + ) + ) + + {:ok, :done} + end +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/contacts_merge_test.exs` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/contacts/merge.ex test/kith/contacts_merge_test.exs +git commit -m "feat(contacts): deduplicate fields, tags, photos and relationships after merge" +``` + +--- + +### Task 7: Drop list, last_talked_to, job cancellation, display name, audit + +**Files:** +- Modify: `lib/kith/contacts/merge.ex` +- Test: `test/kith/contacts_merge_test.exs` + +**Interfaces:** +- Consumes: `Merge.run/4` from Task 6. +- Produces: `run/4` now honours `resolution.drop`, sets `last_talked_to`, cancels losers' Oban jobs, enqueues `DisplayNameRecomputeWorker` for the survivor, and writes one `:contact_merged` audit entry. `run/4` gains an `:actor` requirement: `scope.user` is used as the audit actor. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith/contacts_merge_test.exs`: + +```elixir + describe "merge_cluster/4 side effects" do + setup ctx do + scope = Kith.Accounts.Scope.for_user(ctx.user) + email_type = Repo.one!(from(t in Kith.Contacts.ContactFieldType, where: like(t.protocol, "mailto%"), limit: 1)) + %{scope: scope, email_type: email_type} + end + + test "dropped records are removed and recorded in the audit entry", ctx do + keep = + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_a, ctx.email_type.id, %{ + value: "keep@example.com" + }) + + drop = + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_b, ctx.email_type.id, %{ + value: "drop@example.com" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{contact_fields: [drop.id]} + }) + + values = + from(f in Kith.Contacts.ContactField, + where: f.contact_id == ^survivor.id, + select: f.value + ) + |> Repo.all() + + assert values == ["keep@example.com"] + assert Repo.get(Kith.Contacts.ContactField, keep.id) + refute Repo.get(Kith.Contacts.ContactField, drop.id) + + log = + Repo.one!( + from(l in Kith.AuditLogs.AuditLog, + where: l.event == "contact_merged", + order_by: [desc: l.id], + limit: 1 + ) + ) + + assert [dropped] = log.metadata["dropped"] + assert dropped["type"] == "contact_fields" + assert dropped["value"] == "drop@example.com" + assert dropped["owner_id"] == ctx.contact_b.id + end + + test "rejects a dropped id belonging to no member", ctx do + stranger = + Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Zed"}) + + field = + Kith.ContactsFixtures.contact_field_fixture(stranger, ctx.email_type.id, %{ + value: "zed@example.com" + }) + + assert {:error, {:unknown_drop, :contact_fields}} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{contact_fields: [field.id]} + }) + end + + test "last_talked_to takes the maximum across members", ctx do + older = ~U[2025-01-01 00:00:00Z] + newer = ~U[2026-06-01 00:00:00Z] + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [last_talked_to: older] + ) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [last_talked_to: newer] + ) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert DateTime.compare(survivor.last_talked_to, newer) == :eq + end + + test "enqueues a display name recompute for the survivor", ctx do + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert_enqueued( + worker: Kith.Workers.DisplayNameRecomputeWorker, + args: %{contact_id: survivor.id} + ) + end + + test "a rejected merge changes nothing at all", ctx do + Kith.ContactsFixtures.note_fixture(ctx.contact_b, ctx.user.id) + + assert {:error, {:unknown_value, :company}} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{company: "Never Corp"}, + drop: %{} + }) + + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id).deleted_at == nil + + assert Repo.aggregate( + from(n in Kith.Contacts.Note, where: n.contact_id == ^ctx.contact_b.id), + :count + ) == 1 + + assert Repo.aggregate( + from(l in Kith.AuditLogs.AuditLog, where: l.event == "contact_merged"), + :count + ) == 0 + end + + test "writes one audit entry naming every loser", ctx do + c = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Alice"}) + + {:ok, _survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id, c.id], %{ + fields: %{}, + drop: %{} + }) + + logs = + from(l in Kith.AuditLogs.AuditLog, where: l.event == "contact_merged") |> Repo.all() + + assert length(logs) == 1 + assert Enum.sort(hd(logs).metadata["loser_ids"]) == Enum.sort([ctx.contact_b.id, c.id]) + end + end +``` + +Add `use Oban.Testing, repo: Kith.Repo` directly below `use Kith.DataCase` at +the top of `test/kith/contacts_merge_test.exs`. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts_merge_test.exs -k "side effects"` +Expected: FAIL — the dropped field is still present and no audit log row exists. + +- [ ] **Step 3: Write minimal implementation** + +In `lib/kith/contacts/merge.ex`, add these aliases at the top: + +```elixir + alias Kith.AuditLogs + alias Kith.Contacts.{Address, Contact, ContactField, MergeFields} + alias Kith.Workers.DisplayNameRecomputeWorker +``` + +Add the drop-target map below `@owned_schemas`: + +```elixir + # Which schema backs each key of `resolution.drop`. Tags are the exception: + # `contact_tags` is a bare join table, so its ids are tag ids. + @drop_schemas %{ + contact_fields: ContactField, + addresses: Address + } +``` + +Extend `run/4`: add `validate_drop/2` to the `with` chain, and append these +steps to the `Multi` after `:remap_relationships` and before +`:soft_delete_losers`: + +```elixir + |> Multi.run(:collect_dropped, fn repo, _changes -> + {:ok, describe_dropped(repo, resolution)} + end) + |> Multi.run(:apply_drop, fn repo, _changes -> + Enum.each(resolution.drop || %{}, fn + {_key, []} -> + :ok + + {:tags, tag_ids} -> + repo.delete_all( + from(ct in "contact_tags", + where: ct.contact_id == ^survivor.id and ct.tag_id in ^tag_ids + ) + ) + + {key, ids} -> + schema = Map.fetch!(@drop_schemas, key) + repo.delete_all(from(r in schema, where: r.id in ^ids)) + end) + + {:ok, :done} + end) + |> Multi.run(:last_talked_to, fn repo, %{survivor: survivor} -> + latest = + members + |> Enum.map(& &1.last_talked_to) + |> Enum.reject(&is_nil/1) + |> case do + [] -> nil + dates -> Enum.max(dates, DateTime) + end + + if latest && latest != survivor.last_talked_to do + survivor |> Ecto.Changeset.change(%{last_talked_to: latest}) |> repo.update() + else + {:ok, survivor} + end + end) + |> Multi.run(:cancel_jobs, fn _repo, _changes -> + Enum.each(losers, &Kith.Reminders.cancel_all_for_contact(&1.id, scope.account.id)) + {:ok, :done} + end) +``` + +And append these two steps after `:soft_delete_losers`: + +```elixir + |> Multi.run(:recompute_display_name, fn _repo, _changes -> + %{contact_id: survivor.id} + |> DisplayNameRecomputeWorker.new() + |> Oban.insert() + end) + |> Multi.run(:audit, fn _repo, changes -> + AuditLogs.log_event(scope.account.id, scope.user, :contact_merged, + contact_id: survivor.id, + contact_name: survivor.display_name, + metadata: %{ + survivor_id: survivor.id, + loser_ids: Enum.map(losers, & &1.id), + fields: inspect_fields(resolution), + dropped: changes.collect_dropped + } + ) + end) +``` + +Add these private functions: + +```elixir + defp validate_drop(members, %{drop: drop}) when is_map(drop) do + member_ids = Enum.map(members, & &1.id) + + Enum.reduce_while(drop, :ok, fn + {_key, []}, :ok -> + {:cont, :ok} + + {:tags, _ids}, :ok -> + {:cont, :ok} + + {key, ids}, :ok -> + schema = Map.fetch!(@drop_schemas, key) + + owned = + from(r in schema, where: r.id in ^ids and r.contact_id in ^member_ids, select: r.id) + |> Repo.all() + + if Enum.sort(owned) == Enum.sort(ids) do + {:cont, :ok} + else + {:halt, {:error, {:unknown_drop, key}}} + end + end) + end + + defp validate_drop(_members, _resolution), do: :ok + + # Captured before deletion so the audit trail survives the rows it describes. + defp describe_dropped(repo, %{drop: drop}) when is_map(drop) do + Enum.flat_map(drop, fn + {_key, []} -> + [] + + {:tags, ids} -> + Enum.map(ids, &%{type: "tags", value: to_string(&1), owner_id: nil}) + + {key, ids} -> + schema = Map.fetch!(@drop_schemas, key) + + from(r in schema, where: r.id in ^ids, select: {r.value, r.contact_id}) + |> repo.all() + |> Enum.map(fn {value, owner_id} -> + %{type: to_string(key), value: value, owner_id: owner_id} + end) + end) + end + + defp describe_dropped(_repo, _resolution), do: [] + + defp inspect_fields(%{fields: fields}) do + Map.new(fields, fn {field, value} -> {to_string(field), inspect(value)} end) + end +``` + +Update the `with` in `run/4` to include the drop validation: + +```elixir + with {:ok, members} <- lock_and_load(scope, survivor_id, loser_ids), + survivor = Enum.find(members, &(&1.id == survivor_id)), + :ok <- validate_fields(members, resolution), + :ok <- validate_drop(members, resolution) do +``` + +**Change what `run/4` returns.** The `:last_talked_to` step produces a newer +survivor struct than the `:survivor` step, so returning the latter would hand +back a contact whose `last_talked_to` is stale. Replace the final `case` clause: + +```elixir + |> case do + {:ok, %{last_talked_to: survivor}} -> {:ok, survivor} + {:error, _step, reason, _changes} -> {:error, reason} + end +``` + +For the same reason, the `:audit` step must read the survivor from `changes` +rather than closing over the stale outer binding — its function head becomes +`fn _repo, changes ->` and it uses `changes.last_talked_to` wherever the audit +metadata references the survivor's `display_name`. + +Note `describe_dropped/2` selects `r.value`, which exists on `ContactField` but +not on `Address`. Give `Address` its own clause by matching on the key before +the generic one: + +```elixir + {:addresses, ids} -> + from(a in Address, where: a.id in ^ids, select: {a.line1, a.contact_id}) + |> repo.all() + |> Enum.map(fn {line1, owner_id} -> + %{type: "addresses", value: line1, owner_id: owner_id} + end) +``` + +Place that clause immediately after the `{:tags, ids}` clause. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/contacts_merge_test.exs` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/contacts/merge.ex test/kith/contacts_merge_test.exs +git commit -m "feat(contacts): apply drop list, last talked to, job cancellation and audit on merge" +``` + +--- + +### Task 8: Candidate pair resolution and repointing + +**Files:** +- Modify: `lib/kith/duplicate_detection.ex` +- Modify: `lib/kith/contacts/merge.ex` +- Test: `test/kith/duplicate_detection_test.exs` + +**Interfaces:** +- Consumes: `Merge.run/4` from Task 7. +- Produces: `DuplicateDetection.resolve_after_merge(account_id, survivor_id, loser_ids, unchecked_ids)` returning `:ok`. Called from the engine's `Multi`. Sets pairs wholly inside the merged set to `merged`, pairs from a merged member to an unchecked member to `dismissed`, repoints every remaining pair referencing a loser onto the survivor, and leaves unchecked-to-unchecked pairs untouched. +- `merge_cluster/4`'s `resolution` map gains an optional `:unchecked_ids` key (default `[]`). + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith/duplicate_detection_test.exs`: + +```elixir + describe "resolve_after_merge/4" do + setup do + Kith.ContactsFixtures.seed_reference_data!() + user = Kith.AccountsFixtures.user_fixture() + account_id = user.account_id + + names = [a: "Ann", b: "Bea", c: "Cal", d: "Dee", e: "Eve"] + + contacts = + Map.new(names, fn {key, name} -> + {key, Kith.ContactsFixtures.contact_fixture(account_id, %{first_name: name})} + end) + + %{user: user, account_id: account_id, contacts: contacts} + end + + defp pair!(account_id, one, two, status \\ "pending") do + {low, high} = if one.id < two.id, do: {one, two}, else: {two, one} + + Repo.insert!(%Kith.Contacts.DuplicateCandidate{ + account_id: account_id, + contact_id: low.id, + duplicate_contact_id: high.id, + score: 0.9, + status: status, + detected_at: DateTime.utc_now(:second) + }) + end + + defp status_of(account_id, one, two) do + {low, high} = if one.id < two.id, do: {one, two}, else: {two, one} + + Repo.one( + from(d in Kith.Contacts.DuplicateCandidate, + where: + d.account_id == ^account_id and d.contact_id == ^low.id and + d.duplicate_contact_id == ^high.id, + select: d.status + ) + ) + end + + test "pairs inside the merged set become merged", ctx do + %{a: a, b: b} = ctx.contacts + pair!(ctx.account_id, a, b) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], []) + + assert status_of(ctx.account_id, a, b) == "merged" + end + + test "pairs from a merged member to an unchecked member become dismissed", ctx do + %{a: a, b: b, d: d} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, a, d) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], [d.id]) + + assert status_of(ctx.account_id, a, d) == "dismissed" + end + + test "pairs between two unchecked members are untouched", ctx do + %{a: a, b: b, d: d, e: e} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, d, e) + + :ok = + Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], [d.id, e.id]) + + assert status_of(ctx.account_id, d, e) == "pending" + end + + test "a loser's dismissal is repointed onto the survivor", ctx do + %{a: a, b: b, d: d} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, b, d, "dismissed") + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], [d.id]) + + assert status_of(ctx.account_id, a, d) == "dismissed" + assert status_of(ctx.account_id, b, d) == nil + end + + test "repointing keeps the strongest status on collision", ctx do + %{a: a, b: b, d: d} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, a, d, "dismissed") + pair!(ctx.account_id, b, d) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], [d.id]) + + assert status_of(ctx.account_id, a, d) == "dismissed" + end + + test "pairs in an unrelated cluster are untouched", ctx do + %{a: a, b: b, d: d, e: e} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, d, e) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], []) + + assert status_of(ctx.account_id, d, e) == "pending" + end + + test "every merged row has at least one trashed endpoint", ctx do + %{a: a, b: b} = ctx.contacts + pair!(ctx.account_id, a, b) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^b.id), + set: [deleted_at: DateTime.utc_now(:second)] + ) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], []) + + merged_rows = + from(d in Kith.Contacts.DuplicateCandidate, + where: d.status == "merged", + join: c1 in Kith.Contacts.Contact, + on: c1.id == d.contact_id, + join: c2 in Kith.Contacts.Contact, + on: c2.id == d.duplicate_contact_id, + select: {c1.deleted_at, c2.deleted_at} + ) + |> Repo.all() + + assert merged_rows != [] + assert Enum.all?(merged_rows, fn {one, two} -> one != nil or two != nil end) + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/duplicate_detection_test.exs -k "resolve_after_merge"` +Expected: FAIL with `function Kith.DuplicateDetection.resolve_after_merge/4 is undefined`. + +- [ ] **Step 3: Write minimal implementation** + +Append to `lib/kith/duplicate_detection.ex`: + +```elixir + @status_rank %{"pending" => 0, "dismissed" => 1, "merged" => 2} + + @doc """ + Settles candidate pairs after a merge. + + Three rules, from the design spec §2: + + * both endpoints merged → `merged` + * one merged, one unchecked → `dismissed` (the user reviewed and rejected it) + * both unchecked → untouched + + Then every remaining pair referencing a loser is repointed onto the survivor, + because the survivor now *is* that contact. Without repointing, a dismissal + recorded against a loser evaporates when the loser is trashed and the rejected + match returns on the next scan. + """ + def resolve_after_merge(account_id, survivor_id, loser_ids, unchecked_ids) do + merged_ids = [survivor_id | loser_ids] + now = DateTime.utc_now() |> DateTime.truncate(:second) + + Repo.transaction(fn -> + set_status(account_id, merged_ids, merged_ids, "merged", now) + set_status(account_id, merged_ids, unchecked_ids, "dismissed", now) + repoint(account_id, survivor_id, loser_ids) + end) + + :ok + end + + defp set_status(_account_id, _left, [], _status, _now), do: :ok + + defp set_status(account_id, left, right, status, now) do + from(d in DuplicateCandidate, + where: d.account_id == ^account_id, + where: + (d.contact_id in ^left and d.duplicate_contact_id in ^right) or + (d.contact_id in ^right and d.duplicate_contact_id in ^left) + ) + |> Repo.update_all(set: [status: status, resolved_at: now]) + end + + # Delete-then-insert rather than update_all: the unique index on + # (account_id, contact_id, duplicate_contact_id) and the contact_id ordering + # check constraint both reject an in-place rewrite. + defp repoint(account_id, survivor_id, []) when is_integer(survivor_id), do: :ok + + defp repoint(account_id, survivor_id, loser_ids) do + rows = + from(d in DuplicateCandidate, + where: d.account_id == ^account_id, + where: d.contact_id in ^loser_ids or d.duplicate_contact_id in ^loser_ids + ) + |> Repo.all() + + Repo.delete_all( + from(d in DuplicateCandidate, + where: d.account_id == ^account_id, + where: d.contact_id in ^loser_ids or d.duplicate_contact_id in ^loser_ids + ) + ) + + rows + |> Enum.map(&repoint_row(&1, survivor_id, loser_ids)) + |> Enum.reject(&is_nil/1) + |> Enum.group_by(fn row -> {row.contact_id, row.duplicate_contact_id} end) + |> Enum.each(fn {{low, high}, candidates} -> + strongest = Enum.max_by(candidates, &Map.fetch!(@status_rank, &1.status)) + merge_or_insert(account_id, low, high, strongest) + end) + end + + defp repoint_row(row, survivor_id, loser_ids) do + one = if row.contact_id in loser_ids, do: survivor_id, else: row.contact_id + two = if row.duplicate_contact_id in loser_ids, do: survivor_id, else: row.duplicate_contact_id + + if one == two do + nil + else + {low, high} = if one < two, do: {one, two}, else: {two, one} + %{row | contact_id: low, duplicate_contact_id: high} + end + end + + defp merge_or_insert(account_id, low, high, candidate) do + existing = + Repo.one( + from(d in DuplicateCandidate, + where: + d.account_id == ^account_id and d.contact_id == ^low and + d.duplicate_contact_id == ^high + ) + ) + + winner = + if existing && + Map.fetch!(@status_rank, existing.status) >= Map.fetch!(@status_rank, candidate.status) do + existing.status + else + candidate.status + end + + attrs = %{ + contact_id: low, + duplicate_contact_id: high, + score: candidate.score, + reasons: candidate.reasons, + status: winner, + detected_at: candidate.detected_at, + resolved_at: candidate.resolved_at + } + + if existing do + existing |> DuplicateCandidate.changeset(attrs) |> Repo.update!() + else + %DuplicateCandidate{account_id: account_id} + |> DuplicateCandidate.changeset(attrs) + |> Repo.insert!() + end + end +``` + +Then wire it into the engine. In `lib/kith/contacts/merge.ex`, add this step to +the `Multi` immediately after `:soft_delete_losers`: + +```elixir + |> Multi.run(:resolve_pairs, fn _repo, _changes -> + Kith.DuplicateDetection.resolve_after_merge( + scope.account.id, + survivor.id, + Enum.map(losers, & &1.id), + Map.get(resolution, :unchecked_ids, []) + ) + + {:ok, :done} + end) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/duplicate_detection_test.exs` +Expected: PASS, 6 new tests. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/duplicate_detection.ex lib/kith/contacts/merge.ex test/kith/duplicate_detection_test.exs +git commit -m "feat(duplicates): resolve and repoint candidate pairs after a merge" +``` + +--- + +### Task 9: Adapt the API and shim the old wizard + +**Files:** +- Modify: `lib/kith/contacts.ex:1699` (replace `merge_contacts/3` body with a shim) +- Modify: `lib/kith_web/controllers/api/contact_controller.ex:218` +- Modify: `lib/kith_web/live/contact_live/merge.ex:174` (remove the blanket dismissal) +- Test: `test/kith/contacts_merge_test.exs`, `test/kith_web/controllers/api/contact_controller_test.exs` + +**Interfaces:** +- Consumes: `MergeResolution.resolve/2`, `Contacts.merge_cluster/4`. +- Produces: `Contacts.merge_contacts(survivor_id, non_survivor_id, field_choices \\ %{})` keeps its arity and return shape `{:ok, %Contact{}}`, now implemented over the new engine. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith/contacts_merge_test.exs`: + +```elixir + describe "merge_contacts/3 shim" do + test "still honours non_survivor field choices", ctx do + {:ok, survivor} = + Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id, %{ + "company" => "non_survivor" + }) + + assert survivor.company == "New Corp" + end + + test "fills a gap on the survivor from the loser", ctx do + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [middle_name: nil] + ) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [middle_name: "Jo"] + ) + + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert survivor.middle_name == "Jo" + end + + test "never overrides an existing survivor value by default", ctx do + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert survivor.company == "Old Corp" + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts_merge_test.exs -k "shim"` +Expected: FAIL on the gap-filling test — the old engine leaves `middle_name` nil. + +- [ ] **Step 3: Write minimal implementation** + +In `lib/kith/contacts.ex`, replace the whole `merge_contacts/3` function +(currently lines 1699–1866) and its now-unused private helpers +`update_survivor_fields/3`, `merge_last_talked_to/3`, `remap_relationships/3`, +`fetch_active_contact/1`, `validate_merge/2` and `most_recent_date/2` with: + +```elixir + @doc """ + Merges `non_survivor_id` into `survivor_id`. + + Retained for the existing wizard and REST API. `field_choices` uses the old + `%{"field" => "survivor" | "non_survivor"}` shape; everything not named is + resolved by `Kith.Contacts.MergeResolution`, which fills gaps on the survivor + without overriding values it already has. + """ + def merge_contacts(survivor_id, non_survivor_id, field_choices \\ %{}) do + with {:ok, survivor} <- fetch_mergeable(survivor_id), + {:ok, loser} <- fetch_mergeable(non_survivor_id) do + scope = Kith.Accounts.Scope.for_account_id(survivor.account_id) + resolution = MergeResolution.resolve([survivor, loser], survivor.id) + fields = apply_legacy_choices(resolution.fields, survivor, loser, field_choices) + + merge_cluster(scope, survivor.id, [loser.id], %{fields: fields, drop: %{}}) + end + end + + defp fetch_mergeable(id) do + case Repo.get(Contact, id) do + nil -> {:error, :not_found} + %Contact{deleted_at: nil} = contact -> {:ok, contact} + %Contact{} -> {:error, :trashed} + end + end + + defp apply_legacy_choices(fields, survivor, loser, choices) do + Enum.reduce(choices, fields, fn {field_string, source}, acc -> + field = String.to_existing_atom(field_string) + + if field in MergeFields.choice_fields() do + contact = if source == "non_survivor", do: loser, else: survivor + Map.put(acc, field, Map.fetch!(contact, field) || :clear) + else + acc + end + end) + end +``` + +Add `MergeFields` and `MergeResolution` to the `Kith.Contacts.{...}` alias block. + +`Kith.Accounts.Scope.for_account_id/1` does not exist yet. Add it to +`lib/kith/accounts/scope.ex`: + +```elixir + @doc """ + Builds a scope for background and adapter callers that have an account id but + no user. `user` is nil, so callers needing an audit actor must supply one. + """ + def for_account_id(account_id) do + %__MODULE__{user: nil, account: Kith.Repo.get!(Kith.Accounts.Account, account_id)} + end +``` + +Because `scope.user` may now be nil, guard the audit step in +`lib/kith/contacts/merge.ex` — replace the `:audit` step body's first line with: + +```elixir + if scope.user do + AuditLogs.log_event(scope.account.id, scope.user, :contact_merged, +``` + +and close the `if` with `else {:ok, :skipped} end` around the existing call. + +In `lib/kith_web/controllers/api/contact_controller.ex`, replace the body of +`merge/2` (line 218) so it uses the scope it already has: + +```elixir + def merge(conn, %{"survivor_id" => survivor_id, "non_survivor_id" => non_survivor_id}) do + scope = conn.assigns.current_scope + user = scope.user + account_id = scope.account.id + + with true <- Policy.can?(user, :update, :contact), + :ok <- validate_merge_ids(survivor_id, non_survivor_id), + survivor when not is_nil(survivor) <- Contacts.get_contact(account_id, survivor_id), + loser when not is_nil(loser) <- Contacts.get_contact(account_id, non_survivor_id), + resolution = Kith.Contacts.MergeResolution.resolve([survivor, loser], survivor.id), + {:ok, merged} <- + Contacts.merge_cluster(scope, survivor.id, [loser.id], %{ + fields: resolution.fields, + drop: %{} + }) do + json(conn, %{data: ContactJSON.data(merged)}) + else + false -> {:error, :forbidden} + nil -> {:error, :not_found} + {:error, :same_ids} -> {:error, :bad_request, "Cannot merge a contact with itself."} + {:error, reason} -> {:error, :bad_request, "Merge failed: #{inspect(reason)}"} + end + end +``` + +In `lib/kith_web/live/contact_live/merge.ex`, delete these two lines (currently +174–175) — pair resolution is now the engine's job, and the blanket dismissal is +Bug 2: + +```elixir + DuplicateDetection.dismiss_candidates_for_contact(account_id, contact_a.id) + DuplicateDetection.dismiss_candidates_for_contact(account_id, contact_b.id) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test` +Expected: PASS across the whole suite. If `test/kith_web/live/contact_live/` has +a merge test asserting the old dismissal behaviour, update it to assert the new +rules from Task 8 instead. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix credo --strict +mix test +git add lib/kith/contacts.ex lib/kith/accounts/scope.ex lib/kith/contacts/merge.ex lib/kith_web/controllers/api/contact_controller.ex lib/kith_web/live/contact_live/merge.ex test/ +git commit -m "refactor(contacts): route legacy merge paths through the new engine" +``` + +--- + +## Slice 1 completion checklist + +- [ ] `mix precommit` passes. +- [ ] Spec scenarios covered by this slice: D1–D10, F4, G1, G3, Bug 1, Bug 2. +- [ ] D9 (concurrent merges) has its locking implemented in Task 4 but no + automated test — `FOR UPDATE` contention needs two connections, which the + Ecto sandbox does not give us. Verify by hand: open two `iex -S mix` + sessions against the dev database and call `merge_cluster/4` on + overlapping clusters; the second must block, then fail validation on the + now-trashed member rather than merging. +- [ ] No UI change is visible: the duplicates page and the merge wizard behave as + before, except that merging no longer dismisses pairs the user never saw. + +## Not in this slice + +Slice 2 (clusters and the new screen) and slice 3 (manual merge and cleanup) are +separate plans. `list_clusters/2`, the negative-edge builder, the cluster +LiveView, contacts-index multi-select and the deletion of `ContactLive.Merge` +and `ContactLive.Duplicates` all belong there. diff --git a/docs/superpowers/plans/2026-08-17-merge-revamp-slice-2-clusters-screen.md b/docs/superpowers/plans/2026-08-17-merge-revamp-slice-2-clusters-screen.md new file mode 100644 index 00000000..1a6c57b8 --- /dev/null +++ b/docs/superpowers/plans/2026-08-17-merge-revamp-slice-2-clusters-screen.md @@ -0,0 +1,2387 @@ +# Merge Revamp — Slice 2: Clusters and the Merge Screen — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace the pairwise duplicates queue and the three-step merge wizard with a cluster list and a single-screen cluster merge, built on slice 1's resolver and engine. + +**Architecture:** Clusters are derived, not stored — `list_clusters/2` runs a greedy union over pending candidate pairs where dismissed pairs act as negative edges that block a union. A summary module gathers the multi-valued records and history counts the screen shows. The `ClusterMerge` LiveView holds member selection and field overrides in assigns, recomputes resolution on every selection change, and submits a complete resolution to `merge_cluster/4`. + +**Tech Stack:** Elixir, Ecto, Phoenix LiveView, ExUnit. + +**Spec:** `docs/superpowers/specs/2026-08-17-duplicate-merge-revamp-design.md` + +**Depends on:** `2026-08-17-merge-revamp-slice-1-engine.md` must be complete and merged. This plan calls `Kith.Contacts.MergeResolution.resolve/2`, `Kith.Contacts.merge_cluster/4` and `Kith.DuplicateDetection.resolve_after_merge/4`. + +## Global Constraints + +- Account-scoped multitenancy: every query goes through `scope.account.id`. +- Never add `Co-Authored-By` lines to commits. +- Run `mix test` before every commit and ensure 0 failures. +- `mix format` after each task; `.heex` uses the `Phoenix.LiveView.HTMLFormatter` plugin. +- LiveView routes live in the `:require_authenticated_user` live session in `lib/kith_web/router.ex`. +- Authorization uses `Kith.Policy.can?(user, :update, :contact)`. A `viewer` must be refused. +- One deviation from the spec, deliberate and better: §1 states the negative-edge check costs O(|C₁|×|C₂|) per edge. The implementation below checks each dismissed pair against the two groups instead, which is O(|negatives|) per edge. + +### Carried over from slice 1 — read before building the drop list + +Slice 1's engine landed with a caller contract that only documentation enforces, and this slice is +the first caller that can violate it. Both items were found in slice 1's reviews and deliberately +left for here. + +- **A drop list must name EVERY row backing a dropped value.** The engine's `:dedupe_owned` step + runs before `:apply_drop`. If the survivor and a loser both hold the same contact-field value and + the user unchecks it, dedupe keeps whichever row has the lower id: if that is the survivor's, the + drop deletes it and the value is gone (correct); if it is the loser's, the drop's id matches + nothing and **the excluded value silently comes back**, roughly half the time, depending on row + ids. See the `## Contract` section of `Kith.Contacts.Merge`'s moduledoc. Reordering the Multi + steps makes it worse, not better. Two ways out — decide before Task 8: (a) the LiveView expands + each unchecked value to every member row backing it, or (b) the engine grows value-based dropping + for the dedupe keys, which is the sturdier fix since it cannot be forgotten by a future caller. + Option (b) is a slice-1 file change and needs its own test. + +- **Never let the user pick a raw Immich field value.** Slice 1 exempted the four Immich columns + from the engine's held-by-member validation, because the group is resolved as a unit and can + legitimately synthesise `immich_status: "unlinked"` when no member is linked. That means the + engine no longer rejects an Immich value no member holds: a bogus `immich_status` would sail past + validation and hit the bare DB CHECK `contacts_immich_status_values`, raising a `Postgrex.Error` + that escapes the error contract entirely (there is no `validate_inclusion` for it on + `Contact.update_changeset/2`). The §4 Immich row must therefore submit a chosen MEMBER, with the + LiveView copying that member's whole four-field group — never free-form field values. + +- **Do not offer a member's id as a `first_met_through_id` value.** The engine now enforces spec D4 + by silently coercing a `first_met_through_id` that names any merged member to `:clear`. If the + Identity section's segmented control lists the loser as a candidate value, the user can click it, + the merge will succeed, and the field will come back empty with no explanation. Filter member ids + out of that row's options (and out of its attribution), or render the row as "cleared — a contact + cannot be met through a record it just absorbed". + +- **`Kith.Accounts.Scope.system_for_account_id/1` is not an authorization scope.** It fabricates a + full-privilege scope with `user: nil` for the legacy two-contact shim. Do not use it in this + slice: the cluster screen has a real `current_scope`, and passing a userless scope would also + silently skip the engine's audit entry. + +## File Structure + +| File | Responsibility | +|---|---| +| `lib/kith/duplicate_detection/cluster.ex` (create) | The derived cluster struct. Data only. | +| `lib/kith/duplicate_detection.ex` (modify) | `list_clusters/2`, `get_cluster/2`, `cluster_count/1`, `default_primary/1`, `dismiss_selection/3`. | +| `lib/kith/contacts/merge_summary.ex` (create) | Multi-valued records and history counts for a member set. Read-only. | +| `lib/kith/contacts/merge_resolution.ex` (modify) | Adds `candidates_for/2` so the UI can open any field as a choice. | +| `lib/kith_web/live/contact_live/cluster_merge.ex` (create) | The merge screen. | +| `lib/kith_web/live/contact_live/index.ex` (modify) | `:duplicates` action lists clusters. | +| `lib/kith_web/live/contact_live/index.html.heex:378-478` (modify) | Cluster rows replace pair cards. | +| `lib/kith_web/router.ex` (modify) | Adds the cluster route. | + +--- + +### Task 1: Cluster derivation + +**Files:** +- Create: `lib/kith/duplicate_detection/cluster.ex` +- Modify: `lib/kith/duplicate_detection.ex` +- Test: `test/kith/duplicate_detection_test.exs` + +**Interfaces:** +- Produces: `%Kith.DuplicateDetection.Cluster{key: integer, contacts: [Contact.t()], pairs: [DuplicateCandidate.t()], max_score: float, reasons: [String.t()]}` and `DuplicateDetection.list_clusters(account_id, opts)` returning a list of clusters ordered by `max_score` descending, then `key` ascending. `opts` accepts `:limit` (default 20) and `:offset` (default 0). + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith/duplicate_detection_test.exs`: + +```elixir + describe "list_clusters/2" do + setup do + Kith.ContactsFixtures.seed_reference_data!() + user = Kith.AccountsFixtures.user_fixture() + account_id = user.account_id + + contacts = + Map.new([a: "Ann", b: "Bea", c: "Cal", d: "Dee", e: "Eve"], fn {key, name} -> + {key, Kith.ContactsFixtures.contact_fixture(account_id, %{first_name: name})} + end) + + %{user: user, account_id: account_id, contacts: contacts} + end + + defp candidate!(account_id, one, two, opts \\ []) do + {low, high} = if one.id < two.id, do: {one, two}, else: {two, one} + + Repo.insert!(%Kith.Contacts.DuplicateCandidate{ + account_id: account_id, + contact_id: low.id, + duplicate_contact_id: high.id, + score: Keyword.get(opts, :score, 0.9), + reasons: Keyword.get(opts, :reasons, ["email_match"]), + status: Keyword.get(opts, :status, "pending"), + detected_at: DateTime.utc_now(:second) + }) + end + + defp member_ids(cluster), do: cluster.contacts |> Enum.map(& &1.id) |> Enum.sort() + + test "transitive pairs collapse into one cluster", ctx do + %{a: a, b: b, c: c} = ctx.contacts + candidate!(ctx.account_id, a, b) + candidate!(ctx.account_id, b, c) + + assert [cluster] = Kith.DuplicateDetection.list_clusters(ctx.account_id) + assert member_ids(cluster) == Enum.sort([a.id, b.id, c.id]) + end + + test "disjoint pairs stay separate", ctx do + %{a: a, b: b, d: d, e: e} = ctx.contacts + candidate!(ctx.account_id, a, b) + candidate!(ctx.account_id, d, e) + + clusters = Kith.DuplicateDetection.list_clusters(ctx.account_id) + + assert length(clusters) == 2 + end + + test "clusters are ordered by their highest score", ctx do + %{a: a, b: b, d: d, e: e} = ctx.contacts + candidate!(ctx.account_id, a, b, score: 0.6) + candidate!(ctx.account_id, d, e, score: 0.95) + + [first, second] = Kith.DuplicateDetection.list_clusters(ctx.account_id) + + assert first.max_score == 0.95 + assert second.max_score == 0.6 + end + + test "a cluster carries the union of its pair reasons", ctx do + %{a: a, b: b, c: c} = ctx.contacts + candidate!(ctx.account_id, a, b, reasons: ["email_match"]) + candidate!(ctx.account_id, b, c, reasons: ["phone_match", "email_match"]) + + assert [cluster] = Kith.DuplicateDetection.list_clusters(ctx.account_id) + assert Enum.sort(cluster.reasons) == ["email_match", "phone_match"] + end + + test "the key is the lowest member id", ctx do + %{a: a, b: b} = ctx.contacts + candidate!(ctx.account_id, a, b) + + assert [cluster] = Kith.DuplicateDetection.list_clusters(ctx.account_id) + assert cluster.key == Enum.min([a.id, b.id]) + end + + test "a dismissed pair blocks a transitive union", ctx do + %{a: a, b: b, c: c} = ctx.contacts + # A and C were reviewed and rejected; B links to both. + candidate!(ctx.account_id, a, c, status: "dismissed") + candidate!(ctx.account_id, a, b, score: 0.9) + candidate!(ctx.account_id, b, c, score: 0.6) + + clusters = Kith.DuplicateDetection.list_clusters(ctx.account_id) + + assert [cluster] = clusters + assert member_ids(cluster) == Enum.sort([a.id, b.id]) + refute c.id in member_ids(cluster) + end + + test "the stronger edge wins when a dismissal blocks the weaker", ctx do + %{a: a, b: b, c: c} = ctx.contacts + candidate!(ctx.account_id, a, c, status: "dismissed") + candidate!(ctx.account_id, a, b, score: 0.6) + candidate!(ctx.account_id, b, c, score: 0.9) + + assert [cluster] = Kith.DuplicateDetection.list_clusters(ctx.account_id) + assert member_ids(cluster) == Enum.sort([b.id, c.id]) + end + + test "tied scores cluster deterministically", ctx do + %{a: a, b: b, c: c} = ctx.contacts + candidate!(ctx.account_id, a, b, score: 0.85) + candidate!(ctx.account_id, b, c, score: 0.85) + + first = Kith.DuplicateDetection.list_clusters(ctx.account_id) |> Enum.map(&member_ids/1) + + for _ <- 1..5 do + assert Enum.map(Kith.DuplicateDetection.list_clusters(ctx.account_id), &member_ids/1) == + first + end + end + + test "paginates over clusters, not pairs", ctx do + %{a: a, b: b, d: d, e: e} = ctx.contacts + candidate!(ctx.account_id, a, b, score: 0.95) + candidate!(ctx.account_id, d, e, score: 0.6) + + assert [one] = Kith.DuplicateDetection.list_clusters(ctx.account_id, limit: 1) + assert one.max_score == 0.95 + + assert [two] = Kith.DuplicateDetection.list_clusters(ctx.account_id, limit: 1, offset: 1) + assert two.max_score == 0.6 + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/duplicate_detection_test.exs -k "list_clusters"` +Expected: FAIL with `function Kith.DuplicateDetection.list_clusters/1 is undefined`. + +- [ ] **Step 3: Write minimal implementation** + +Create `lib/kith/duplicate_detection/cluster.ex`: + +```elixir +defmodule Kith.DuplicateDetection.Cluster do + @moduledoc """ + A derived group of contacts that detection believes are the same person. + + Clusters have no database row — they are computed from `duplicate_candidates` + on every read, so they cannot drift from the pairs that justify them. `key` is + the lowest member contact id and is used for routing. + """ + + defstruct [:key, :contacts, :pairs, :max_score, :reasons] + + @type t :: %__MODULE__{ + key: integer(), + contacts: [Kith.Contacts.Contact.t()], + pairs: [Kith.Contacts.DuplicateCandidate.t()], + max_score: float(), + reasons: [String.t()] + } +end +``` + +Append to `lib/kith/duplicate_detection.ex` (and add +`alias Kith.DuplicateDetection.Cluster` plus `alias Kith.Contacts.Contact` at the +top): + +```elixir + @doc """ + Lists derived duplicate clusters, highest confidence first. + + Options: `:limit` (default #{@default_page_size}), `:offset` (default 0). + """ + def list_clusters(account_id, opts \\ []) do + limit = Keyword.get(opts, :limit, @default_page_size) + offset = Keyword.get(opts, :offset, 0) + + account_id + |> build_clusters() + |> Enum.drop(offset) + |> Enum.take(limit) + end + + @doc "How many clusters are pending review." + def cluster_count(account_id), do: account_id |> build_clusters() |> length() + + defp build_clusters(account_id) do + candidates = + DuplicateCandidate + |> scope_to_account(account_id) + |> where([d], d.status in ["pending", "dismissed"]) + |> Repo.all() + + {pending, dismissed} = Enum.split_with(candidates, &(&1.status == "pending")) + + negatives = + MapSet.new(dismissed, fn d -> {d.contact_id, d.duplicate_contact_id} end) + + pending + # Fully deterministic: scores come from a small fixed set, so ties are the + # common case. Without the id tiebreak, which edge wins would depend on + # arbitrary row order and clustering would vary between identical runs. + |> Enum.sort_by(&{-&1.score, &1.contact_id, &1.duplicate_contact_id}) + |> Enum.reduce(%{groups: %{}, of: %{}, next: 0}, &union_pair(&2, &1, negatives)) + |> to_clusters(pending) + end + + defp union_pair(acc, pair, negatives) do + {acc, group_a} = ensure_group(acc, pair.contact_id) + {acc, group_b} = ensure_group(acc, pair.duplicate_contact_id) + + cond do + group_a == group_b -> acc + blocked?(acc, group_a, group_b, negatives) -> acc + true -> merge_groups(acc, group_a, group_b) + end + end + + defp ensure_group(acc, contact_id) do + case Map.fetch(acc.of, contact_id) do + {:ok, group_id} -> + {acc, group_id} + + :error -> + group_id = acc.next + + acc = %{ + acc + | groups: Map.put(acc.groups, group_id, MapSet.new([contact_id])), + of: Map.put(acc.of, contact_id, group_id), + next: group_id + 1 + } + + {acc, group_id} + end + end + + # A dismissed pair is a negative edge: the user already reviewed those two + # contacts and rejected the match, so no third contact may reunite them by + # transitivity. + defp blocked?(acc, group_a, group_b, negatives) do + members_a = Map.fetch!(acc.groups, group_a) + members_b = Map.fetch!(acc.groups, group_b) + + Enum.any?(negatives, fn {one, two} -> + (MapSet.member?(members_a, one) and MapSet.member?(members_b, two)) or + (MapSet.member?(members_a, two) and MapSet.member?(members_b, one)) + end) + end + + defp merge_groups(acc, group_a, group_b) do + members_b = Map.fetch!(acc.groups, group_b) + members = MapSet.union(Map.fetch!(acc.groups, group_a), members_b) + + of = Enum.reduce(members_b, acc.of, &Map.put(&2, &1, group_a)) + + %{acc | groups: acc.groups |> Map.put(group_a, members) |> Map.delete(group_b), of: of} + end + + defp to_clusters(acc, pending) do + contacts = + acc.of + |> Map.keys() + |> then(fn ids -> from(c in Contact, where: c.id in ^ids) |> Repo.all() end) + |> Map.new(&{&1.id, &1}) + + acc.groups + |> Enum.filter(fn {_id, members} -> MapSet.size(members) >= 2 end) + |> Enum.map(fn {_id, members} -> build_cluster(members, pending, contacts) end) + |> Enum.sort_by(&{-&1.max_score, &1.key}) + end + + defp build_cluster(members, pending, contacts) do + pairs = + Enum.filter(pending, fn p -> + MapSet.member?(members, p.contact_id) and + MapSet.member?(members, p.duplicate_contact_id) + end) + + member_contacts = + members + |> Enum.map(&Map.get(contacts, &1)) + |> Enum.reject(&is_nil/1) + |> Enum.sort_by(& &1.id) + + %Cluster{ + key: members |> Enum.min(), + contacts: member_contacts, + pairs: pairs, + max_score: pairs |> Enum.map(& &1.score) |> Enum.max(fn -> 0.0 end), + reasons: pairs |> Enum.flat_map(& &1.reasons) |> Enum.uniq() + } + end +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/duplicate_detection_test.exs` +Expected: PASS, 9 new tests. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/duplicate_detection/cluster.ex lib/kith/duplicate_detection.ex test/kith/duplicate_detection_test.exs +git commit -m "feat(duplicates): derive clusters with dismissed pairs as negative edges" +``` + +--- + +### Task 2: Cluster lookup, trashed filtering, primary and selection dismissal + +**Files:** +- Modify: `lib/kith/duplicate_detection.ex` +- Test: `test/kith/duplicate_detection_test.exs` + +**Interfaces:** +- Produces: + - `get_cluster(account_id, contact_id)` → `%Cluster{}` or `nil`. Accepts **any** member id, not only the key. + - `default_primary(contacts)` → `%Contact{}`, the member with the most attached records, tie-broken by earliest `inserted_at`. + - `dismiss_selection(account_id, selected_ids, unchecked_ids)` → `:ok`. Writes the full clique over `selected_ids` as `dismissed`, plus every selected↔unchecked pair; leaves unchecked↔unchecked alone. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith/duplicate_detection_test.exs`: + +```elixir + describe "get_cluster/2 and trashed members" do + setup do + Kith.ContactsFixtures.seed_reference_data!() + user = Kith.AccountsFixtures.user_fixture() + + contacts = + Map.new([a: "Ann", b: "Bea", c: "Cal"], fn {key, name} -> + {key, Kith.ContactsFixtures.contact_fixture(user.account_id, %{first_name: name})} + end) + + %{user: user, account_id: user.account_id, contacts: contacts} + end + + test "any member id resolves to the cluster", ctx do + %{a: a, b: b, c: c} = ctx.contacts + candidate!(ctx.account_id, a, b) + candidate!(ctx.account_id, b, c) + + for member <- [a, b, c] do + cluster = Kith.DuplicateDetection.get_cluster(ctx.account_id, member.id) + assert cluster + assert member.id in Enum.map(cluster.contacts, & &1.id) + end + end + + test "a contact in no cluster returns nil", ctx do + assert Kith.DuplicateDetection.get_cluster(ctx.account_id, ctx.contacts.a.id) == nil + end + + test "a trashed member is excluded and a one-member cluster disappears", ctx do + %{a: a, b: b} = ctx.contacts + candidate!(ctx.account_id, a, b) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^b.id), + set: [deleted_at: DateTime.utc_now(:second)] + ) + + assert Kith.DuplicateDetection.list_clusters(ctx.account_id) == [] + assert Kith.DuplicateDetection.get_cluster(ctx.account_id, a.id) == nil + end + + test "a trashed member is dropped but a three-member cluster survives", ctx do + %{a: a, b: b, c: c} = ctx.contacts + candidate!(ctx.account_id, a, b) + candidate!(ctx.account_id, b, c) + + Repo.update_all(from(x in Kith.Contacts.Contact, where: x.id == ^c.id), + set: [deleted_at: DateTime.utc_now(:second)] + ) + + assert [cluster] = Kith.DuplicateDetection.list_clusters(ctx.account_id) + assert Enum.map(cluster.contacts, & &1.id) |> Enum.sort() == Enum.sort([a.id, b.id]) + end + end + + describe "default_primary/1" do + setup do + Kith.ContactsFixtures.seed_reference_data!() + user = Kith.AccountsFixtures.user_fixture() + %{user: user, account_id: user.account_id} + end + + test "picks the member with the most attached records", ctx do + thin = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Thin"}) + rich = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Rich"}) + + Kith.ContactsFixtures.note_fixture(rich, ctx.user.id) + Kith.ContactsFixtures.note_fixture(rich, ctx.user.id) + Kith.ContactsFixtures.address_fixture(rich) + Kith.ContactsFixtures.note_fixture(thin, ctx.user.id) + + assert Kith.DuplicateDetection.default_primary([thin, rich]).id == rich.id + end + + test "breaks ties toward the earliest created", ctx do + older = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Older"}) + newer = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Newer"}) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^older.id), + set: [inserted_at: ~U[2020-01-01 00:00:00Z]] + ) + + older = Repo.get!(Kith.Contacts.Contact, older.id) + + assert Kith.DuplicateDetection.default_primary([newer, older]).id == older.id + end + end + + describe "dismiss_selection/3" do + setup do + Kith.ContactsFixtures.seed_reference_data!() + user = Kith.AccountsFixtures.user_fixture() + + contacts = + Map.new([a: "Ann", b: "Bea", c: "Cal", d: "Dee", e: "Eve"], fn {key, name} -> + {key, Kith.ContactsFixtures.contact_fixture(user.account_id, %{first_name: name})} + end) + + %{account_id: user.account_id, contacts: contacts} + end + + test "writes the full clique over the selected members", ctx do + %{a: a, b: b, c: c} = ctx.contacts + # A chain: only two pairs exist, but all three combinations must end up + # dismissed or a later contact could reunite A and C by transitivity. + candidate!(ctx.account_id, a, b) + candidate!(ctx.account_id, b, c) + + :ok = Kith.DuplicateDetection.dismiss_selection(ctx.account_id, [a.id, b.id, c.id], []) + + assert status_of(ctx.account_id, a, b) == "dismissed" + assert status_of(ctx.account_id, b, c) == "dismissed" + assert status_of(ctx.account_id, a, c) == "dismissed" + end + + test "dismisses selected-to-unchecked but never unchecked-to-unchecked", ctx do + %{a: a, b: b, d: d, e: e} = ctx.contacts + candidate!(ctx.account_id, a, b) + candidate!(ctx.account_id, a, d) + candidate!(ctx.account_id, d, e) + + :ok = + Kith.DuplicateDetection.dismiss_selection(ctx.account_id, [a.id, b.id], [d.id, e.id]) + + assert status_of(ctx.account_id, a, d) == "dismissed" + assert status_of(ctx.account_id, d, e) == "pending" + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/duplicate_detection_test.exs -k "get_cluster"` +Expected: FAIL with `function Kith.DuplicateDetection.get_cluster/2 is undefined`. + +- [ ] **Step 3: Write minimal implementation** + +In `lib/kith/duplicate_detection.ex`, filter trashed members inside +`build_clusters/1`. Replace the `candidates` binding with: + +```elixir + candidates = + DuplicateCandidate + |> scope_to_account(account_id) + |> where([d], d.status in ["pending", "dismissed"]) + |> join(:inner, [d], one in Contact, on: one.id == d.contact_id) + |> join(:inner, [d, _one], two in Contact, on: two.id == d.duplicate_contact_id) + |> where([d, one, two], is_nil(one.deleted_at) and is_nil(two.deleted_at)) + |> select([d], d) + |> Repo.all() +``` + +Then add these functions: + +```elixir + @doc """ + Finds the cluster containing `contact_id`, or `nil`. + + Takes any member id rather than only the cluster key, so a bookmark survives + the key shifting when a lower-id member joins. + """ + def get_cluster(account_id, contact_id) do + account_id + |> build_clusters() + |> Enum.find(fn cluster -> + Enum.any?(cluster.contacts, &(&1.id == contact_id)) + end) + end + + @count_schemas [ + Kith.Contacts.Note, + Kith.Contacts.Address, + Kith.Contacts.ContactField, + Kith.Contacts.Document, + Kith.Contacts.Photo, + Kith.Activities.Call, + Kith.Activities.LifeEvent + ] + + @doc """ + The member that should survive by default: the one holding the most attached + records, tie-broken by earliest creation. + + Moving the fewest rows is the cheap part; the real reason is that the richest, + oldest record is the id external clients (CardDAV, Immich) are already pinned + to. + """ + def default_primary(contacts) do + ids = Enum.map(contacts, & &1.id) + counts = attached_counts(ids) + + Enum.max_by(contacts, fn contact -> + {Map.get(counts, contact.id, 0), -DateTime.to_unix(contact.inserted_at)} + end) + end + + defp attached_counts(ids) do + schema_counts = + Enum.reduce(@count_schemas, %{}, fn schema, acc -> + from(r in schema, + where: r.contact_id in ^ids, + group_by: r.contact_id, + select: {r.contact_id, count(r.id)} + ) + |> Repo.all() + |> Enum.reduce(acc, fn {id, n}, acc -> Map.update(acc, id, n, &(&1 + n)) end) + end) + + from(ac in "activity_contacts", + where: ac.contact_id in ^ids, + group_by: ac.contact_id, + select: {ac.contact_id, count()} + ) + |> Repo.all() + |> Enum.reduce(schema_counts, fn {id, n}, acc -> Map.update(acc, id, n, &(&1 + n)) end) + end + + @doc """ + Records that the selected members are not duplicates of each other. + + Writes the full clique over `selected_ids` rather than only the pairs + detection happened to produce: a cluster is often a chain (A–B, B–C), and + dismissing only those leaves no negative edge between A and C. Pairs between + two unchecked members are never touched — the user excluded them, and + excluding is not a statement about them. + """ + def dismiss_selection(account_id, selected_ids, unchecked_ids) do + now = DateTime.utc_now() |> DateTime.truncate(:second) + + cliques = + for one <- selected_ids, two <- selected_ids, one < two, do: {one, two} + + crosses = + for one <- selected_ids, two <- unchecked_ids, one != two do + if one < two, do: {one, two}, else: {two, one} + end + + (cliques ++ crosses) + |> Enum.uniq() + |> Enum.each(fn {low, high} -> upsert_dismissed(account_id, low, high, now) end) + + :ok + end + + defp upsert_dismissed(account_id, low, high, now) do + case Repo.one( + from(d in DuplicateCandidate, + where: + d.account_id == ^account_id and d.contact_id == ^low and + d.duplicate_contact_id == ^high + ) + ) do + nil -> + %DuplicateCandidate{account_id: account_id} + |> DuplicateCandidate.changeset(%{ + contact_id: low, + duplicate_contact_id: high, + score: 0.0, + reasons: ["user_rejected"], + status: "dismissed", + detected_at: now, + resolved_at: now + }) + |> Repo.insert!() + + %DuplicateCandidate{status: "merged"} = existing -> + existing + + existing -> + existing |> DuplicateCandidate.dismiss_changeset() |> Repo.update!() + end + end +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/duplicate_detection_test.exs` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/duplicate_detection.ex test/kith/duplicate_detection_test.exs +git commit -m "feat(duplicates): add cluster lookup, primary selection and clique dismissal" +``` + +--- + +### Task 3: Merge summary — multi-valued records and history counts + +**Files:** +- Create: `lib/kith/contacts/merge_summary.ex` +- Test: `test/kith/contacts/merge_summary_test.exs` + +**Interfaces:** +- Produces: `MergeSummary.build(members)` returning + `%{contact_fields: [entry], addresses: [entry], tags: [entry], aliases: [entry], history: %{atom => integer}}` + where `entry` is `%{id: term, label: String.t(), value: String.t(), owner_id: integer, duplicate?: boolean}`. + For tags, `id` is the tag id. For aliases, `id` is the string itself. + +- [ ] **Step 1: Write the failing test** + +Create `test/kith/contacts/merge_summary_test.exs`: + +```elixir +defmodule Kith.Contacts.MergeSummaryTest do + use Kith.DataCase, async: false + + import Ecto.Query + + alias Kith.Contacts.MergeSummary + alias Kith.{AccountsFixtures, ContactsFixtures} + + setup do + ContactsFixtures.seed_reference_data!() + user = AccountsFixtures.user_fixture() + + email_type = + Repo.one!( + from(t in Kith.Contacts.ContactFieldType, where: like(t.protocol, "mailto%"), limit: 1) + ) + + a = ContactsFixtures.contact_fixture(user.account_id, %{first_name: "Sarah"}) + b = ContactsFixtures.contact_fixture(user.account_id, %{first_name: "Sarah"}) + + %{user: user, account_id: user.account_id, email_type: email_type, a: a, b: b} + end + + test "combines contact fields and marks the repeat as a duplicate", ctx do + ContactsFixtures.contact_field_fixture(ctx.a, ctx.email_type.id, %{value: "s@example.com"}) + ContactsFixtures.contact_field_fixture(ctx.b, ctx.email_type.id, %{value: "s@example.com"}) + ContactsFixtures.contact_field_fixture(ctx.b, ctx.email_type.id, %{value: "other@example.com"}) + + summary = MergeSummary.build([ctx.a, ctx.b]) + + assert length(summary.contact_fields) == 3 + assert Enum.count(summary.contact_fields, & &1.duplicate?) == 1 + + kept = Enum.reject(summary.contact_fields, & &1.duplicate?) + assert Enum.sort(Enum.map(kept, & &1.value)) == ["other@example.com", "s@example.com"] + end + + test "the first occurrence is kept and the later one marked duplicate", ctx do + first = ContactsFixtures.contact_field_fixture(ctx.a, ctx.email_type.id, %{value: "s@example.com"}) + second = ContactsFixtures.contact_field_fixture(ctx.b, ctx.email_type.id, %{value: "s@example.com"}) + + summary = MergeSummary.build([ctx.a, ctx.b]) + + assert Enum.find(summary.contact_fields, &(&1.id == first.id)).duplicate? == false + assert Enum.find(summary.contact_fields, &(&1.id == second.id)).duplicate? == true + end + + test "addresses dedupe on line1 and postal code", ctx do + ContactsFixtures.address_fixture(ctx.a, %{line1: "1 Main St", postal_code: "94110"}) + ContactsFixtures.address_fixture(ctx.b, %{line1: " 1 main st ", postal_code: "94110"}) + + summary = MergeSummary.build([ctx.a, ctx.b]) + + assert Enum.count(summary.addresses, & &1.duplicate?) == 1 + end + + test "aliases are unioned across members", ctx do + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.a.id), + set: [aliases: ["Sarah K."]] + ) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.b.id), + set: [aliases: ["Sarah K.", "SK"]] + ) + + a = Repo.get!(Kith.Contacts.Contact, ctx.a.id) + b = Repo.get!(Kith.Contacts.Contact, ctx.b.id) + + summary = MergeSummary.build([a, b]) + + assert Enum.map(summary.aliases, & &1.value) |> Enum.sort() == ["SK", "Sarah K."] + end + + test "history counts every type across every member", ctx do + ContactsFixtures.note_fixture(ctx.a, ctx.user.id) + ContactsFixtures.note_fixture(ctx.b, ctx.user.id) + ContactsFixtures.document_fixture(ctx.b) + + summary = MergeSummary.build([ctx.a, ctx.b]) + + assert summary.history.notes == 2 + assert summary.history.documents == 1 + assert summary.history.calls == 0 + end +end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts/merge_summary_test.exs` +Expected: FAIL with `module Kith.Contacts.MergeSummary is not available`. + +- [ ] **Step 3: Write minimal implementation** + +Create `lib/kith/contacts/merge_summary.ex`: + +```elixir +defmodule Kith.Contacts.MergeSummary do + @moduledoc """ + Gathers the multi-valued records and history counts a merge screen shows. + + Read-only. Duplicates are marked rather than removed so the screen can show + what the merge will collapse instead of silently dropping it. + """ + + import Ecto.Query, warn: false + + alias Kith.Contacts.{Address, ContactField, Contact, Document, Note, Photo} + alias Kith.Repo + + @history [ + notes: Note, + documents: Document, + photos: Photo, + calls: Kith.Activities.Call, + life_events: Kith.Activities.LifeEvent, + reminders: Kith.Reminders.Reminder, + gifts: Kith.Contacts.Gift, + debts: Kith.Contacts.Debt, + pets: Kith.Contacts.Pet, + tasks: Kith.Tasks.Task, + conversations: Kith.Conversations.Conversation, + relationships: Kith.Contacts.Relationship + ] + + @doc "Builds the summary for `members`." + def build(members) do + ids = Enum.map(members, & &1.id) + + %{ + contact_fields: contact_fields(ids), + addresses: addresses(ids), + tags: tags(ids), + aliases: aliases(members), + history: history(ids) + } + end + + defp contact_fields(ids) do + from(f in ContactField, + where: f.contact_id in ^ids, + join: t in assoc(f, :contact_field_type), + order_by: [asc: f.id], + select: {f, t.label} + ) + |> Repo.all() + |> Enum.map(fn {field, label} -> + %{ + id: field.id, + label: label, + value: field.value, + owner_id: field.contact_id, + key: {field.contact_field_type_id, normalize(field.value)} + } + end) + |> mark_duplicates() + end + + defp addresses(ids) do + from(a in Address, where: a.contact_id in ^ids, order_by: [asc: a.id]) + |> Repo.all() + |> Enum.map(fn address -> + %{ + id: address.id, + label: address.label || "Address", + value: Enum.join(Enum.reject([address.line1, address.city], &is_nil/1), ", "), + owner_id: address.contact_id, + key: {normalize(address.line1), normalize(address.postal_code)} + } + end) + |> mark_duplicates() + end + + defp tags(ids) do + from(ct in "contact_tags", + join: t in Kith.Contacts.Tag, + on: t.id == ct.tag_id, + where: ct.contact_id in ^ids, + order_by: [asc: t.name], + select: {ct.contact_id, t.id, t.name} + ) + |> Repo.all() + |> Enum.map(fn {owner_id, tag_id, name} -> + %{id: tag_id, label: "Tag", value: name, owner_id: owner_id, key: tag_id} + end) + |> mark_duplicates() + end + + defp aliases(members) do + members + |> Enum.flat_map(fn member -> + Enum.map(member.aliases || [], &%{ + id: &1, + label: "Alias", + value: &1, + owner_id: member.id, + key: &1 + }) + end) + |> mark_duplicates() + end + + defp history(ids) do + counts = + Map.new(@history, fn {key, schema} -> + {key, Repo.aggregate(from(r in schema, where: r.contact_id in ^ids), :count)} + end) + + activities = + from(ac in "activity_contacts", where: ac.contact_id in ^ids, select: count()) + |> Repo.one() + + Map.put(counts, :activities, activities) + end + + # The first occurrence of a key wins; later ones are what the merge collapses. + defp mark_duplicates(entries) do + {marked, _seen} = + Enum.map_reduce(entries, MapSet.new(), fn entry, seen -> + duplicate? = MapSet.member?(seen, entry.key) + {entry |> Map.put(:duplicate?, duplicate?) |> Map.delete(:key), + MapSet.put(seen, entry.key)} + end) + + marked + end + + defp normalize(nil), do: nil + defp normalize(value) when is_binary(value), do: value |> String.trim() |> String.downcase() +end +``` + +If `Kith.Contacts.Address` has no `label` field, drop the `address.label ||` +fallback and use `"Address"` directly — check the schema before writing. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/contacts/merge_summary_test.exs` +Expected: PASS, 5 tests. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/contacts/merge_summary.ex test/kith/contacts/merge_summary_test.exs +git commit -m "feat(contacts): summarise multi-valued records and history for a merge" +``` + +--- + +### Task 4: Candidate values for any field + +**Files:** +- Modify: `lib/kith/contacts/merge_resolution.ex` +- Test: `test/kith/contacts/merge_resolution_test.exs` + +**Interfaces:** +- Produces: `MergeResolution.candidates_for(members, field)` returning + `[%{value: term, member_ids: [integer], count: integer}]` for **any** choice field, including one where every member agrees. This is what lets the screen open a resolved row as a choice. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith/contacts/merge_resolution_test.exs`: + +```elixir + describe "candidates_for/2" do + test "a resolved field still offers its single value", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah"}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + assert [%{value: "Sarah", count: 2}] = + MergeResolution.candidates_for([a, b], :first_name) + end + + test "an empty field offers nothing", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", occupation: nil}) + b = contact(ctx.account_id, %{first_name: "Sarah", occupation: nil}) + + assert MergeResolution.candidates_for([a, b], :occupation) == [] + end + + test "a contested field offers every value, most-held first", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Figma"}) + b = contact(ctx.account_id, %{first_name: "Sarah", company: "Stripe"}) + c = contact(ctx.account_id, %{first_name: "Sarah", company: "Stripe"}) + + assert [%{value: "Stripe", count: 2}, %{value: "Figma", count: 1}] = + MergeResolution.candidates_for([a, b, c], :company) + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith/contacts/merge_resolution_test.exs -k "candidates_for"` +Expected: FAIL with `function Kith.Contacts.MergeResolution.candidates_for/2 is undefined`. + +- [ ] **Step 3: Write minimal implementation** + +Add to `lib/kith/contacts/merge_resolution.ex`: + +```elixir + @doc """ + Every distinct value `members` hold for `field`, most-held first. + + Unlike `conflicts`, this is populated even when the members agree — the screen + uses it to open an already-resolved row as a choice. + """ + def candidates_for(members, field) do + members + |> Enum.map(fn member -> {member.id, normalize(Map.fetch!(member, field))} end) + |> Enum.reject(fn {_id, value} -> is_nil(value) end) + |> candidates() + |> Enum.sort_by(& &1.count, :desc) + end +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith/contacts/merge_resolution_test.exs` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith/contacts/merge_resolution.ex test/kith/contacts/merge_resolution_test.exs +git commit -m "feat(contacts): expose candidate values for any mergeable field" +``` + +--- + +### Task 5: Duplicates index lists clusters + +**Files:** +- Modify: `lib/kith_web/live/contact_live/index.ex:60-71,255-280` +- Modify: `lib/kith_web/live/contact_live/index.html.heex:378-478` +- Test: `test/kith_web/live/contact_live/duplicates_index_test.exs` (create) + +**Interfaces:** +- Consumes: `DuplicateDetection.list_clusters/2`, `cluster_count/1`. +- Produces: the `:duplicates` action assigns `:clusters` (list of `%Cluster{}`), `:clusters_total`, `:clusters_has_more`. The old `:candidates`, `:duplicates_total`, `:duplicates_has_more` assigns and the `dismiss` / `load_more_duplicates` handlers are removed. + +- [ ] **Step 1: Write the failing test** + +Create `test/kith_web/live/contact_live/duplicates_index_test.exs`: + +```elixir +defmodule KithWeb.ContactLive.DuplicatesIndexTest do + use KithWeb.ConnCase + + import Phoenix.LiveViewTest + import Ecto.Query + + alias Kith.{AccountsFixtures, ContactsFixtures} + + setup %{conn: conn} do + ContactsFixtures.seed_reference_data!() + user = AccountsFixtures.user_fixture() + %{conn: log_in_user(conn, user), user: user, account_id: user.account_id} + end + + defp candidate!(account_id, one, two, score \\ 0.9) do + {low, high} = if one.id < two.id, do: {one, two}, else: {two, one} + + Kith.Repo.insert!(%Kith.Contacts.DuplicateCandidate{ + account_id: account_id, + contact_id: low.id, + duplicate_contact_id: high.id, + score: score, + reasons: ["email_match"], + status: "pending", + detected_at: DateTime.utc_now(:second) + }) + end + + test "four duplicates render as one cluster entry", ctx do + [a, b, c, d] = + for name <- ~w(Ann Bea Cal Dee) do + ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: name}) + end + + candidate!(ctx.account_id, a, b) + candidate!(ctx.account_id, b, c) + candidate!(ctx.account_id, c, d) + + {:ok, _live, html} = live(ctx.conn, ~p"/contacts/duplicates") + + assert html =~ "1 possible duplicate" + assert html =~ "4 contacts" + + for name <- ~w(Ann Bea Cal Dee), do: assert(html =~ name) + end + + test "each cluster links to its cluster screen", ctx do + a = ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Ann"}) + b = ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Bea"}) + candidate!(ctx.account_id, a, b) + + {:ok, _live, html} = live(ctx.conn, ~p"/contacts/duplicates") + + assert html =~ "/contacts/duplicates/cluster/#{min(a.id, b.id)}" + end + + test "shows the empty state when nothing is pending", ctx do + {:ok, _live, html} = live(ctx.conn, ~p"/contacts/duplicates") + + assert html =~ "No duplicates found" + end +end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith_web/live/contact_live/duplicates_index_test.exs` +Expected: FAIL — the page renders pair cards, so `4 contacts` is absent. + +- [ ] **Step 3: Write minimal implementation** + +In `lib/kith_web/live/contact_live/index.ex`, replace the `apply_action` clause +for `:duplicates` (line 62) with: + +```elixir + defp apply_action(socket, :duplicates, _params) do + account_id = socket.assigns.current_scope.account.id + clusters = DuplicateDetection.list_clusters(account_id, limit: @duplicates_page_size) + total = DuplicateDetection.cluster_count(account_id) + + socket + |> assign(:page_title, "Duplicate Contacts") + |> assign(:clusters, clusters) + |> assign(:clusters_total, total) + |> assign(:clusters_has_more, length(clusters) >= @duplicates_page_size) + end +``` + +Replace the `mount/3` assigns `:duplicates_total` and `:duplicates_has_more` +(lines 36–37) with: + +```elixir + |> assign(:clusters, []) + |> assign(:clusters_total, 0) + |> assign(:clusters_has_more, false) +``` + +Delete the `dismiss` handler (around line 255) and replace the +`load_more_duplicates` handler (line 267) with: + +```elixir + def handle_event("load_more_duplicates", _params, socket) do + account_id = socket.assigns.current_scope.account.id + + more = + DuplicateDetection.list_clusters(account_id, + limit: @duplicates_page_size, + offset: length(socket.assigns.clusters) + ) + + {:noreply, + socket + |> assign(:clusters, socket.assigns.clusters ++ more) + |> assign(:clusters_has_more, length(more) >= @duplicates_page_size)} + end +``` + +In `lib/kith_web/live/contact_live/index.html.heex`, replace lines 378–478 (the +whole duplicates panel) with: + +```heex + <%!-- Duplicates panel --%> + <%= if @live_action == :duplicates do %> +
+

+ {@clusters_total} possible duplicate{if @clusters_total != 1, do: "s"} found +

+ +
+ + <%= if @clusters == [] do %> + + <% else %> +
+ <.link + :for={cluster <- @clusters} + navigate={~p"/contacts/duplicates/cluster/#{cluster.key}"} + class="block rounded-[var(--radius-lg)] border border-[var(--color-border)] bg-[var(--color-surface-elevated)] shadow-sm p-4 hover:border-[var(--color-border-focus)] transition-colors" + > +
+
+ + {round(cluster.max_score * 100)}% match + + + {Enum.join(cluster.reasons, ", ")} + +
+ + {length(cluster.contacts)} contacts + +
+ +
+ + + + {contact.display_name} + + +
+ + +
+ +
+
+ <% end %> + <% end %> +``` + +The nav badge in `user_auth.ex:233` keeps using `pending_count/1` (pairs). Leave +it — a badge counting pairs while the page counts clusters is confusing, so +change that call to `DuplicateDetection.cluster_count(account_id)` in the same +edit. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith_web/live/contact_live/duplicates_index_test.exs` +Expected: PASS, 3 tests. The route does not exist yet, so the link assertion +checks the string only — `~p` will raise if the route is missing, so add the +route now (Task 8 wires the LiveView; adding the route early is fine): + +In `lib/kith_web/router.ex`, inside the `:require_authenticated_user` live +session, below the `/contacts/duplicates` line: + +```elixir + live "/contacts/duplicates/cluster/:id", ContactLive.ClusterMerge, :show +``` + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith_web/live/contact_live/index.ex lib/kith_web/live/contact_live/index.html.heex lib/kith_web/router.ex lib/kith_web/user_auth.ex test/kith_web/live/contact_live/duplicates_index_test.exs +git commit -m "feat(duplicates): list clusters instead of pairs" +``` + +--- + +### Task 6: Cluster merge screen — rendering + +**Files:** +- Create: `lib/kith_web/live/contact_live/cluster_merge.ex` +- Test: `test/kith_web/live/contact_live/cluster_merge_test.exs` + +**Interfaces:** +- Consumes: `DuplicateDetection.get_cluster/2`, `default_primary/1`, `MergeResolution.resolve/2`, `candidates_for/2`, `MergeSummary.build/1`, `MergeFields`. +- Produces: `KithWeb.ContactLive.ClusterMerge` at `/contacts/duplicates/cluster/:id`. Assigns: `:cluster`, `:members`, `:selected_ids` (MapSet), `:primary_id`, `:overrides` (map), `:resolution`, `:summary`, `:dropped` (MapSet of `{type, id}`), `:error`. + +- [ ] **Step 1: Write the failing test** + +Create `test/kith_web/live/contact_live/cluster_merge_test.exs`: + +```elixir +defmodule KithWeb.ContactLive.ClusterMergeTest do + use KithWeb.ConnCase + + import Phoenix.LiveViewTest + import Ecto.Query + + alias Kith.{AccountsFixtures, ContactsFixtures} + + setup %{conn: conn} do + ContactsFixtures.seed_reference_data!() + user = AccountsFixtures.user_fixture() + account_id = user.account_id + + a = + ContactsFixtures.contact_fixture(account_id, %{ + first_name: "Sarah", + last_name: "Kim", + company: "Figma" + }) + + b = + ContactsFixtures.contact_fixture(account_id, %{ + first_name: "Sarah", + last_name: "Kim", + company: "Stripe", + middle_name: "Jiyoung" + }) + + candidate!(account_id, a, b) + + %{conn: log_in_user(conn, user), user: user, account_id: account_id, a: a, b: b} + end + + defp candidate!(account_id, one, two) do + {low, high} = if one.id < two.id, do: {one, two}, else: {two, one} + + Kith.Repo.insert!(%Kith.Contacts.DuplicateCandidate{ + account_id: account_id, + contact_id: low.id, + duplicate_contact_id: high.id, + score: 0.9, + reasons: ["email_match"], + status: "pending", + detected_at: DateTime.utc_now(:second) + }) + end + + defp cluster_path(a, b), do: "/contacts/duplicates/cluster/#{min(a.id, b.id)}" + + test "renders every member with a checkbox", ctx do + {:ok, _live, html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + assert html =~ "Merge 2 contacts" + assert html =~ ~s(phx-value-id="#{ctx.a.id}") + assert html =~ ~s(phx-value-id="#{ctx.b.id}") + end + + test "renders an agreed field with its attribution and no choice", ctx do + {:ok, _live, html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + assert html =~ "all 2 agree" + assert html =~ "Sarah" + end + + test "renders a gap-filled field attributed to its only source", ctx do + {:ok, _live, html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + assert html =~ "Jiyoung" + assert html =~ "only" + end + + test "renders a contested field as a choice and flags the section", ctx do + {:ok, _live, html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + assert html =~ "Figma" + assert html =~ "Stripe" + assert html =~ "1 needs a decision" + end + + test "a section with a conflict is open, one without is folded", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + assert has_element?(live, "details#section-identity[open]") + refute has_element?(live, "details#section-contact-details[open]") + end + + test "required fields offer no Leave empty option", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + refute has_element?(live, "button[phx-value-field='first_name'][phx-value-index='clear']") + end + + test "an unknown contact id renders not found", ctx do + assert {:error, {:live_redirect, %{to: "/contacts"}}} = + live(ctx.conn, "/contacts/duplicates/cluster/0") + end +end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs` +Expected: FAIL — `KithWeb.ContactLive.ClusterMerge` is undefined. + +- [ ] **Step 3: Write minimal implementation** + +Create `lib/kith_web/live/contact_live/cluster_merge.ex`: + +```elixir +defmodule KithWeb.ContactLive.ClusterMerge do + @moduledoc """ + Reviews and merges a cluster of duplicate contacts on one screen. + + All state is derived from the member selection: unchecking a member re-runs + `MergeResolution` over the remaining members and discards any explicit field + choices, so what is on screen is always a resolution of exactly the checked + set. + """ + + use KithWeb, :live_view + + alias Kith.Contacts + alias Kith.Contacts.{MergeFields, MergeResolution, MergeSummary} + alias Kith.DuplicateDetection + alias Kith.Policy + + @impl true + def mount(_params, _session, socket), do: {:ok, assign(socket, :error, nil)} + + @impl true + def handle_params(%{"id" => id}, _uri, socket) do + scope = socket.assigns.current_scope + + cond do + not Policy.can?(scope.user, :update, :contact) -> + {:noreply, + socket + |> put_flash(:error, "You don't have permission to merge contacts") + |> push_navigate(to: ~p"/contacts")} + + cluster = DuplicateDetection.get_cluster(scope.account.id, String.to_integer(id)) -> + {:noreply, load_cluster(socket, cluster)} + + true -> + {:noreply, + socket + |> put_flash(:error, "No duplicate cluster found for that contact") + |> push_navigate(to: ~p"/contacts")} + end + end + + defp load_cluster(socket, cluster) do + members = cluster.contacts + primary = DuplicateDetection.default_primary(members) + + socket + |> assign(:page_title, "Merge duplicates") + |> assign(:cluster, cluster) + |> assign(:members, members) + |> assign(:selected_ids, MapSet.new(members, & &1.id)) + |> assign(:primary_id, primary.id) + |> assign(:overrides, %{}) + |> assign(:dropped, MapSet.new()) + |> assign(:error, nil) + |> recompute() + end + + # Everything downstream of the selection is derived, never patched in place. + defp recompute(socket) do + selected = selected_members(socket) + primary_id = ensure_primary(socket, selected) + + socket + |> assign(:primary_id, primary_id) + |> assign(:resolution, MergeResolution.resolve(selected, primary_id)) + |> assign(:summary, MergeSummary.build(selected)) + end + + defp selected_members(socket) do + Enum.filter(socket.assigns.members, &MapSet.member?(socket.assigns.selected_ids, &1.id)) + end + + defp ensure_primary(socket, selected) do + if Enum.any?(selected, &(&1.id == socket.assigns.primary_id)) do + socket.assigns.primary_id + else + case selected do + [] -> nil + members -> DuplicateDetection.default_primary(members).id + end + end + end + + # ── Rendering helpers ────────────────────────────────────────────────── + + defp effective(assigns, field) do + Map.get(assigns.overrides, field, Map.get(assigns.resolution.fields, field)) + end + + defp conflict?(assigns, field), do: Map.has_key?(assigns.resolution.conflicts, field) + + defp conflict_count(assigns) do + Enum.count(MergeFields.choice_fields(), &conflict?(assigns, &1)) + end + + defp candidates(assigns, field) do + MergeResolution.candidates_for(selected_from_assigns(assigns), field) + end + + defp selected_from_assigns(assigns) do + Enum.filter(assigns.members, &MapSet.member?(assigns.selected_ids, &1.id)) + end + + defp attribution_text(assigns, field) do + total = MapSet.size(assigns.selected_ids) + + case Map.get(assigns.resolution.attributions, field) do + :all_agree -> "all #{total} agree" + {:only, id} -> "only #{member_name(assigns, id)} has this" + {:some, n} -> "#{n} of #{total}" + _ -> nil + end + end + + defp member_name(assigns, id) do + case Enum.find(assigns.members, &(&1.id == id)) do + nil -> "unknown" + contact -> contact.display_name + end + end + + defp humanize(field) do + field |> Atom.to_string() |> String.replace("_id", "") |> String.replace("_", " ") + end + + defp display(nil), do: nil + defp display(:clear), do: nil + defp display(%Date{} = date), do: Date.to_iso8601(date) + defp display(%DateTime{} = at), do: DateTime.to_iso8601(at) + defp display(true), do: "Yes" + defp display(false), do: "No" + defp display(list) when is_list(list), do: Enum.join(list, ", ") + defp display(value), do: to_string(value) + + @impl true + def render(assigns) do + ~H""" + +
+
+ {@error} + <.link navigate={~p"/contacts/duplicates/cluster/#{@cluster.key}"} class="underline ms-2"> + Reload + +
+ + +
+
+

+ {length(@members)} possible duplicates · {MapSet.size(@selected_ids)} selected +

+

+ Matched on {Enum.join(@cluster.reasons, ", ")}. Uncheck anyone who isn't this person. +

+
+
+ +
+ +
+ + +
0} class="rounded-[var(--radius-lg)] border border-[var(--color-border)] bg-[var(--color-surface-elevated)]"> + + + Identity + + {length(MergeFields.choice_fields())} fields · click any value to change it + + 0} + class="rounded-full bg-[var(--color-danger-subtle)] text-[var(--color-danger)] px-2 py-0.5 text-xs font-medium" + > + {conflict_count(assigns)} need{if conflict_count(assigns) == 1, do: "s"} a decision + + + + +
+
+ {humanize(field)} +
+ +
+ +
+ +
+ + {display(effective(assigns, field))} + + + Not set on any contact + + + {attribution_text(assigns, field)} + +
+
+
+ +
+ + + Contact details + + {length(@summary.contact_fields)} fields, {length(@summary.addresses)} addresses, + {length(@summary.tags)} tags combined + + + + +
+

{label}

+ +

None

+
+
+ +
+ + + Carried over as-is + + {@summary.history |> Map.values() |> Enum.sum()} records move to the primary + + + +
+ 0} + class="rounded-[var(--radius-sm)] border border-[var(--color-border-subtle)] bg-[var(--color-surface-sunken)] px-2.5 py-1 text-xs" + > + {count} {String.replace(to_string(key), "_", " ")} + +
+
+ + +
+

+ {MapSet.size(@selected_ids) - 1} contact(s) move to trash and stay recoverable for 30 days. + + Unchecked contacts will be marked as not duplicates and won't be suggested again. + +

+
+ Not duplicates + + Merge {MapSet.size(@selected_ids)} contacts + +
+
+ +
+ + """ + end +end +``` + +Remove the stray `` before `` in the member strip if the compiler +flags it — the label wraps checkbox, avatar, name and the make-primary button. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs` +Expected: FAIL on the event tests only (Task 7 adds handlers); the rendering +tests pass. Add `@impl true def handle_event(_, _, socket), do: {:noreply, socket}` +temporarily if LiveView raises on the missing callback, and delete it in Task 7. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test test/kith_web/live/contact_live/cluster_merge_test.exs +git add lib/kith_web/live/contact_live/cluster_merge.ex test/kith_web/live/contact_live/cluster_merge_test.exs +git commit -m "feat(duplicates): render the cluster merge screen" +``` + +--- + +### Task 7: Cluster merge screen — interactions + +**Files:** +- Modify: `lib/kith_web/live/contact_live/cluster_merge.ex` +- Test: `test/kith_web/live/contact_live/cluster_merge_test.exs` + +**Interfaces:** +- Produces: `handle_event/3` clauses for `"toggle-member"`, `"set-primary"`, `"choose-field"`, `"toggle-value"`. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith_web/live/contact_live/cluster_merge_test.exs`: + +```elixir + describe "interactions" do + test "unchecking a member recomputes the result", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + html = + live + |> element("input[phx-click='toggle-member'][phx-value-id='#{ctx.b.id}']") + |> render_click() + + assert html =~ "Merge 1 contacts" or html =~ "1 selected" + refute html =~ "Jiyoung" + end + + test "choosing a conflicting value marks it selected", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + html = + live + |> element("button[phx-value-field='company'][phx-value-index='1']") + |> render_click() + + assert html =~ "Figma" + end + + test "changing the selection discards an explicit choice", ctx do + c = + ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Sarah", + last_name: "Kim", + company: "Stripe" + }) + + candidate!(ctx.account_id, ctx.b, c) + + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + live + |> element("button[phx-value-field='company'][phx-value-index='1']") + |> render_click() + + html = + live + |> element("input[phx-click='toggle-member'][phx-value-id='#{c.id}']") + |> render_click() + + # Back to the computed default rather than the override. + assert html =~ "Stripe" + end + + test "unchecking a value removes it from the merge", ctx do + email_type = + Kith.Repo.one!( + from(t in Kith.Contacts.ContactFieldType, where: like(t.protocol, "mailto%"), limit: 1) + ) + + field = ContactsFixtures.contact_field_fixture(ctx.b, email_type.id, %{value: "x@y.com"}) + + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + html = + live + |> element("input[phx-click='toggle-value'][phx-value-id='#{field.id}']") + |> render_click() + + assert html =~ "x@y.com" + end + + test "making another member primary moves the badge", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + html = + live + |> element("button[phx-click='set-primary'][phx-value-id='#{ctx.b.id}']") + |> render_click() + + assert html =~ "primary" + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs -k "interactions"` +Expected: FAIL — no matching `handle_event/3` clause. + +- [ ] **Step 3: Write minimal implementation** + +Add to `lib/kith_web/live/contact_live/cluster_merge.ex`, replacing any +temporary catch-all handler: + +```elixir + @impl true + def handle_event("toggle-member", %{"id" => id}, socket) do + id = String.to_integer(id) + + selected = + if MapSet.member?(socket.assigns.selected_ids, id) do + MapSet.delete(socket.assigns.selected_ids, id) + else + MapSet.put(socket.assigns.selected_ids, id) + end + + {:noreply, + socket + |> assign(:selected_ids, selected) + # Selection changed, so every derived value is stale — including choices + # the user made, which may no longer be held by any selected member. + |> assign(:overrides, %{}) + |> assign(:dropped, MapSet.new()) + |> recompute()} + end + + def handle_event("set-primary", %{"id" => id}, socket) do + {:noreply, socket |> assign(:primary_id, String.to_integer(id)) |> recompute()} + end + + def handle_event("choose-field", %{"field" => field, "index" => index}, socket) do + field = String.to_existing_atom(field) + index = String.to_integer(index) + + selected = + Enum.filter(socket.assigns.members, &MapSet.member?(socket.assigns.selected_ids, &1.id)) + + case Enum.at(MergeResolution.candidates_for(selected, field), index) do + nil -> + {:noreply, socket} + + candidate -> + {:noreply, + assign(socket, :overrides, Map.put(socket.assigns.overrides, field, candidate.value))} + end + end + + def handle_event("toggle-value", %{"type" => type, "id" => id}, socket) do + key = {type, cast_entry_id(id)} + + dropped = + if MapSet.member?(socket.assigns.dropped, key) do + MapSet.delete(socket.assigns.dropped, key) + else + MapSet.put(socket.assigns.dropped, key) + end + + {:noreply, assign(socket, :dropped, dropped)} + end + + defp cast_entry_id(id) do + case Integer.parse(id) do + {int, ""} -> int + _ -> id + end + end +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith_web/live/contact_live/cluster_merge.ex test/kith_web/live/contact_live/cluster_merge_test.exs +git commit -m "feat(duplicates): wire cluster merge screen interactions" +``` + +--- + +### Task 8: Submitting a merge and a rejection + +**Files:** +- Modify: `lib/kith_web/live/contact_live/cluster_merge.ex` +- Test: `test/kith_web/live/contact_live/cluster_merge_test.exs` + +**Interfaces:** +- Consumes: `Contacts.merge_cluster/4`, `DuplicateDetection.dismiss_selection/3`. +- Produces: `handle_event/3` clauses for `"merge"` and `"not-duplicates"`. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith_web/live/contact_live/cluster_merge_test.exs`: + +```elixir + describe "submitting" do + test "merging redirects to the survivor and trashes the rest", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + assert {:error, {:redirect, %{to: path}}} = + live |> element("button[phx-click='merge']") |> render_click() + + survivor_id = path |> String.split("/") |> List.last() |> String.to_integer() + loser_id = if survivor_id == ctx.a.id, do: ctx.b.id, else: ctx.a.id + + assert Kith.Repo.get!(Kith.Contacts.Contact, loser_id).deleted_at != nil + end + + test "merging with a member unchecked dismisses that pair", ctx do + c = ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Sarah", last_name: "Kim"}) + candidate!(ctx.account_id, ctx.b, c) + + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + live + |> element("input[phx-click='toggle-member'][phx-value-id='#{c.id}']") + |> render_click() + + assert {:error, {:redirect, _}} = + live |> element("button[phx-click='merge']") |> render_click() + + statuses = + Kith.Repo.all( + from(d in Kith.Contacts.DuplicateCandidate, + where: d.contact_id == ^min(ctx.b.id, c.id) and d.duplicate_contact_id == ^max(ctx.b.id, c.id), + select: d.status + ) + ) + + assert statuses == ["dismissed"] + end + + test "not duplicates dismisses the cluster and returns to the list", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + assert {:error, {:redirect, %{to: "/contacts/duplicates"}}} = + live |> element("button[phx-click='not-duplicates']") |> render_click() + + assert Kith.DuplicateDetection.list_clusters(ctx.account_id) == [] + end + + test "a member merged elsewhere produces an inline error, not a partial merge", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + # Another session trashes a member behind our back. + Kith.Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.b.id), + set: [deleted_at: DateTime.utc_now(:second)] + ) + + html = live |> element("button[phx-click='merge']") |> render_click() + + assert html =~ "changed since you opened" + assert Kith.Repo.get!(Kith.Contacts.Contact, ctx.a.id).deleted_at == nil + end + + test "a viewer cannot open the screen", ctx do + viewer = AccountsFixtures.user_fixture(%{role: "viewer"}) + + assert {:error, {:live_redirect, %{to: "/contacts"}}} = + live(log_in_user(build_conn(), viewer), cluster_path(ctx.a, ctx.b)) + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs -k "submitting"` +Expected: FAIL — no `"merge"` handler. + +- [ ] **Step 3: Write minimal implementation** + +Add to `lib/kith_web/live/contact_live/cluster_merge.ex`: + +```elixir + def handle_event("merge", _params, socket) do + selected = selected_members(socket) + survivor_id = socket.assigns.primary_id + loser_ids = selected |> Enum.map(& &1.id) |> Enum.reject(&(&1 == survivor_id)) + + resolution = %{ + fields: Map.merge(socket.assigns.resolution.fields, socket.assigns.overrides), + drop: build_drop(socket), + unchecked_ids: unchecked_ids(socket) + } + + case Contacts.merge_cluster(socket.assigns.current_scope, survivor_id, loser_ids, resolution) do + {:ok, survivor} -> + {:noreply, + socket + |> put_flash(:info, "Merged #{length(loser_ids) + 1} contacts") + |> redirect(to: ~p"/contacts/#{survivor.id}")} + + {:error, reason} -> + {:noreply, assign(socket, :error, error_message(reason))} + end + end + + def handle_event("not-duplicates", _params, socket) do + DuplicateDetection.dismiss_selection( + socket.assigns.current_scope.account.id, + socket.assigns.selected_ids |> MapSet.to_list(), + unchecked_ids(socket) + ) + + {:noreply, + socket + |> put_flash(:info, "Marked as not duplicates") + |> redirect(to: ~p"/contacts/duplicates")} + end + + defp unchecked_ids(socket) do + socket.assigns.members + |> Enum.map(& &1.id) + |> Enum.reject(&MapSet.member?(socket.assigns.selected_ids, &1)) + end + + # The screen groups dropped values by its own section labels; the engine wants + # them keyed by entity type. + defp build_drop(socket) do + Enum.reduce(socket.assigns.dropped, %{}, fn {type, id}, acc -> + case drop_key(type) do + nil -> acc + key -> Map.update(acc, key, [id], &[id | &1]) + end + end) + end + + defp drop_key("Fields"), do: :contact_fields + defp drop_key("Addresses"), do: :addresses + defp drop_key("Tags"), do: :tags + defp drop_key(_), do: nil + + defp error_message(:trashed), + do: "One of these contacts changed since you opened this page." + + defp error_message(:not_found), + do: "One of these contacts changed since you opened this page." + + defp error_message({:unknown_value, field}), + do: "The #{humanize(field)} you picked changed since you opened this page." + + defp error_message(reason), do: "Merge failed: #{inspect(reason)}" +``` + +Aliases are unchecked through the `"Aliases"` label but carried in `fields` +rather than `drop`, so extend the merge payload: after building `resolution`, +subtract dropped aliases from the resolved array: + +```elixir + dropped_aliases = + for {"Aliases", value} <- socket.assigns.dropped, do: value + + resolution = + update_in(resolution.fields[:aliases], fn + list when is_list(list) -> list -- dropped_aliases + other -> other + end) +``` + +Place that immediately after the `resolution = %{...}` assignment and before the +`case`. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test` +Expected: PASS across the suite. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix credo --strict +mix test +git add lib/kith_web/live/contact_live/cluster_merge.ex test/kith_web/live/contact_live/cluster_merge_test.exs +git commit -m "feat(duplicates): submit cluster merges and rejections" +``` + +--- + +### Task 9: The Immich row + +**Files:** +- Modify: `lib/kith_web/live/contact_live/cluster_merge.ex` +- Test: `test/kith_web/live/contact_live/cluster_merge_test.exs` + +**Interfaces:** +- Consumes: `MergeFields.immich_fields/0`, the resolver's Immich rules from slice 1 Task 3. +- Produces: `handle_event/3` clause for `"choose-immich"`. The Identity section renders one Immich row listing linked members by name and sync date. + +The Identity section iterates `MergeFields.choice_fields()`, which deliberately +excludes the Immich group, so without this task the four Immich columns are +resolved but never shown — B10 would be unmet. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith_web/live/contact_live/cluster_merge_test.exs`: + +```elixir + describe "immich" do + setup ctx do + Kith.Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.a.id), + set: [ + immich_person_id: "person-a", + immich_status: "linked", + immich_last_synced_at: ~U[2026-01-01 00:00:00Z] + ] + ) + + Kith.Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.b.id), + set: [ + immich_person_id: "person-b", + immich_status: "linked", + immich_last_synced_at: ~U[2026-02-01 00:00:00Z] + ] + ) + + ctx + end + + test "renders one row naming each linked member, not four id rows", ctx do + {:ok, _live, html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + assert html =~ "Photo library" + refute html =~ "person-a" + refute html =~ "immich_person_id" + end + + test "offers an unlink option and one option per linked member", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + assert has_element?(live, "button[phx-click='choose-immich'][phx-value-id='#{ctx.a.id}']") + assert has_element?(live, "button[phx-click='choose-immich'][phx-value-id='#{ctx.b.id}']") + assert has_element?(live, "button[phx-click='choose-immich'][phx-value-id='none']") + end + + test "choosing a member adopts that member's whole group", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + live + |> element("button[phx-click='choose-immich'][phx-value-id='#{ctx.b.id}']") + |> render_click() + + assert {:error, {:redirect, %{to: path}}} = + live |> element("button[phx-click='merge']") |> render_click() + + survivor_id = path |> String.split("/") |> List.last() |> String.to_integer() + survivor = Kith.Repo.get!(Kith.Contacts.Contact, survivor_id) + + assert survivor.immich_person_id == "person-b" + assert survivor.immich_status == "linked" + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs -k "immich"` +Expected: FAIL — `Photo library` is absent because no Immich row is rendered. + +- [ ] **Step 3: Write minimal implementation** + +Add these helpers to `lib/kith_web/live/contact_live/cluster_merge.ex`: + +```elixir + defp linked_members(assigns) do + assigns + |> selected_from_assigns() + |> Enum.filter(&(not is_nil(&1.immich_person_id))) + end + + # Which member's Immich group is currently winning, so the row can highlight + # it without exposing the opaque person id. + defp immich_source_id(assigns) do + case Map.get(assigns.overrides, :__immich__, :default) do + :default -> + chosen = Map.get(assigns.resolution.fields, :immich_person_id) + + case Enum.find(linked_members(assigns), &(&1.immich_person_id == chosen)) do + nil -> :none + member -> member.id + end + + other -> + other + end + end + + defp immich_sync_label(nil), do: "never synced" + defp immich_sync_label(at), do: "synced #{Calendar.strftime(at, "%d %b %Y")}" +``` + +Add this row to the Identity section's template, immediately after the +`:for={field <- MergeFields.choice_fields()}` block and inside the same +`
`: + +```heex +
+
Photo library
+
+ + +
+
+``` + +Add the event handler, beside the other `handle_event/3` clauses: + +```elixir + def handle_event("choose-immich", %{"id" => "none"}, socket) do + overrides = + Enum.reduce(MergeFields.immich_fields(), socket.assigns.overrides, fn field, acc -> + Map.put(acc, field, :clear) + end) + + {:noreply, assign(socket, :overrides, Map.put(overrides, :__immich__, :none))} + end + + def handle_event("choose-immich", %{"id" => id}, socket) do + id = String.to_integer(id) + + case Enum.find(socket.assigns.members, &(&1.id == id)) do + nil -> + {:noreply, socket} + + member -> + # All four columns move together or the survivor ends up with one + # record's person id and another's sync timestamp. + overrides = + Enum.reduce(MergeFields.immich_fields(), socket.assigns.overrides, fn field, acc -> + Map.put(acc, field, Map.fetch!(member, field) || :clear) + end) + + {:noreply, assign(socket, :overrides, Map.put(overrides, :__immich__, id))} + end + end +``` + +`:__immich__` is bookkeeping for the UI, not a contact field, so strip it before +submitting. In the `"merge"` handler, change the `fields` line to: + +```elixir + fields: + socket.assigns.resolution.fields + |> Map.merge(Map.delete(socket.assigns.overrides, :__immich__)), +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith_web/live/contact_live/cluster_merge.ex test/kith_web/live/contact_live/cluster_merge_test.exs +git commit -m "feat(duplicates): render immich links as one editable row" +``` + +--- + +## Slice 2 completion checklist + +- [ ] `mix precommit` passes. +- [ ] Spec scenarios covered: A1–A6, B1–B10, C1–C5, F1–F3, F5–F6, G2. +- [ ] The old wizard at `/contacts/:id/merge` still exists and still works. It is + deleted in slice 3. +- [ ] Manual check: open a real four-member cluster in the browser and confirm + the Identity section opens itself while Contact details stays folded. diff --git a/docs/superpowers/plans/2026-08-17-merge-revamp-slice-3-manual-merge.md b/docs/superpowers/plans/2026-08-17-merge-revamp-slice-3-manual-merge.md new file mode 100644 index 00000000..574e1773 --- /dev/null +++ b/docs/superpowers/plans/2026-08-17-merge-revamp-slice-3-manual-merge.md @@ -0,0 +1,719 @@ +# Merge Revamp — Slice 3: Manual Merge and Cleanup — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Let a user merge contacts detection never linked — by search from the cluster screen, or by selecting them on the contacts list — then delete the old wizard and the dead duplicates LiveView. + +**Architecture:** The cluster screen already renders a member strip from a list of contacts. This slice makes that list come from three sources instead of one: a detected cluster, a bare contact id plus search, or a set of ids passed from the contacts index. The engine and resolver do not change — a manually added member is indistinguishable from a detected one once it is in the strip. + +**Tech Stack:** Elixir, Ecto, Phoenix LiveView, ExUnit. + +**Spec:** `docs/superpowers/specs/2026-08-17-duplicate-merge-revamp-design.md` + +**Depends on:** slices 1 and 2 complete and merged. + +## Global Constraints + +- Account-scoped multitenancy: every lookup goes through `scope.account.id`. +- Never add `Co-Authored-By` lines to commits. +- Run `mix test` before every commit and ensure 0 failures. +- `mix format` after each task. +- Authorization uses `Kith.Policy.can?(user, :update, :contact)`. + +## File Structure + +| File | Responsibility | +|---|---| +| `lib/kith_web/live/contact_live/cluster_merge.ex` (modify) | Accepts a bare contact id and an id list; adds the Add contact search. | +| `lib/kith_web/live/contact_live/index.ex` (modify) | Selection state and the Merge selected action. | +| `lib/kith_web/live/contact_live/index.html.heex` (modify) | Row checkboxes and the selection bar. | +| `lib/kith_web/live/contact_live/show.html.heex` (modify) | Merge link points at the cluster screen. | +| `lib/kith_web/router.ex` (modify) | Removes `/contacts/:id/merge`. | +| `lib/kith_web/live/contact_live/merge.ex` (delete) | The wizard. | +| `lib/kith_web/live/contact_live/duplicates.ex` (delete) | Dead since before this work — unrouted and unreferenced. | + +--- + +### Task 1: A bare contact id opens a one-member screen + +**Files:** +- Modify: `lib/kith_web/live/contact_live/cluster_merge.ex` +- Test: `test/kith_web/live/contact_live/cluster_merge_test.exs` + +**Interfaces:** +- Consumes: `DuplicateDetection.get_cluster/2`, `Contacts.get_contact/3`. +- Produces: `handle_params/3` falls back to a synthetic single-member cluster when the id belongs to no pending cluster. Adds `:synthetic?` to assigns. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith_web/live/contact_live/cluster_merge_test.exs`: + +```elixir + describe "manual entry" do + test "a contact in no cluster renders alone", ctx do + loner = ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Dana"}) + + {:ok, _live, html} = live(ctx.conn, "/contacts/duplicates/cluster/#{loner.id}") + + assert html =~ "Dana" + assert html =~ "Add contact" + refute html =~ "Sarah" + end + + test "merge is disabled until a second member is added", ctx do + loner = ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Dana"}) + + {:ok, live, _html} = live(ctx.conn, "/contacts/duplicates/cluster/#{loner.id}") + + assert has_element?(live, "button[phx-click='merge'][disabled]") + end + + test "a contact id from another account is refused", ctx do + other = AccountsFixtures.user_fixture() + stranger = ContactsFixtures.contact_fixture(other.account_id, %{first_name: "Zed"}) + + assert {:error, {:live_redirect, %{to: "/contacts"}}} = + live(ctx.conn, "/contacts/duplicates/cluster/#{stranger.id}") + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs -k "manual entry"` +Expected: FAIL — the screen redirects to `/contacts` with "No duplicate cluster found". + +- [ ] **Step 3: Write minimal implementation** + +In `lib/kith_web/live/contact_live/cluster_merge.ex`, replace the `handle_params/3` +`cond` with: + +```elixir + def handle_params(%{"id" => id}, _uri, socket) do + scope = socket.assigns.current_scope + contact_id = String.to_integer(id) + + cond do + not Policy.can?(scope.user, :update, :contact) -> + {:noreply, + socket + |> put_flash(:error, "You don't have permission to merge contacts") + |> push_navigate(to: ~p"/contacts")} + + cluster = DuplicateDetection.get_cluster(scope.account.id, contact_id) -> + {:noreply, load_cluster(socket, cluster, false)} + + contact = Contacts.get_contact(scope.account.id, contact_id) -> + # Not a detected duplicate — the user came here to merge by hand. + {:noreply, load_cluster(socket, synthetic_cluster(contact), true)} + + true -> + {:noreply, + socket + |> put_flash(:error, "Contact not found") + |> push_navigate(to: ~p"/contacts")} + end + end + + defp synthetic_cluster(contact) do + %Kith.DuplicateDetection.Cluster{ + key: contact.id, + contacts: [contact], + pairs: [], + max_score: 0.0, + reasons: [] + } + end +``` + +Change `load_cluster/2` to `load_cluster/3` and add the two new assigns: + +```elixir + defp load_cluster(socket, cluster, synthetic?) do + members = cluster.contacts + primary = DuplicateDetection.default_primary(members) + + socket + |> assign(:page_title, "Merge duplicates") + |> assign(:cluster, cluster) + |> assign(:members, members) + |> assign(:selected_ids, MapSet.new(members, & &1.id)) + |> assign(:primary_id, primary.id) + |> assign(:overrides, %{}) + |> assign(:dropped, MapSet.new()) + |> assign(:error, nil) + |> assign(:synthetic?, synthetic?) + |> assign(:search_query, "") + |> assign(:search_results, []) + |> recompute() + end +``` + +In the member strip card, add the search affordance after the chips (Task 2 +wires its events; rendering the input now satisfies the "Add contact" assertion): + +```heex +
+
+ + +
+ +
+ +
+
+``` + +Also guard `default_primary/1` and the footer against a one-member screen: the +footer's `MapSet.size(@selected_ids) - 1` reads "0 contact(s) move to trash", +which is correct, and the merge button is already disabled below two members. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith_web/live/contact_live/cluster_merge.ex test/kith_web/live/contact_live/cluster_merge_test.exs +git commit -m "feat(duplicates): open the cluster screen for a contact with no cluster" +``` + +--- + +### Task 2: Adding a member by search + +**Files:** +- Modify: `lib/kith_web/live/contact_live/cluster_merge.ex` +- Test: `test/kith_web/live/contact_live/cluster_merge_test.exs` + +**Interfaces:** +- Consumes: `Contacts.search_contacts/2`. +- Produces: `handle_event/3` clauses for `"search"` and `"add-member"`. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith_web/live/contact_live/cluster_merge_test.exs`: + +```elixir + describe "add contact search" do + test "searching lists matches excluding current members", ctx do + ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Dana", last_name: "Reed"}) + + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + html = live |> form("form[phx-change='search']", %{query: "Dana"}) |> render_change() + + assert html =~ "Dana" + refute html =~ ~s(phx-click="add-member" phx-value-id="#{ctx.a.id}") + end + + test "adding a contact puts it in the strip, checked", ctx do + dana = ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Dana"}) + + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + live |> form("form[phx-change='search']", %{query: "Dana"}) |> render_change() + + html = + live |> element("button[phx-click='add-member'][phx-value-id='#{dana.id}']") + |> render_click() + + assert html =~ "Merge 3 contacts" + assert has_element?(live, "input[phx-click='toggle-member'][phx-value-id='#{dana.id}']") + end + + test "an added member contributes to conflicts", ctx do + dana = + ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Sarah", + last_name: "Kim", + occupation: "Designer" + }) + + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + live |> form("form[phx-change='search']", %{query: "Sarah"}) |> render_change() + + html = + live |> element("button[phx-click='add-member'][phx-value-id='#{dana.id}']") + |> render_click() + + assert html =~ "Designer" + end + + test "a short query returns nothing", ctx do + {:ok, live, _html} = live(ctx.conn, cluster_path(ctx.a, ctx.b)) + + html = live |> form("form[phx-change='search']", %{query: "D"}) |> render_change() + + refute html =~ "add-member" + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs -k "add contact search"` +Expected: FAIL — no `"search"` handler. + +- [ ] **Step 3: Write minimal implementation** + +Add to `lib/kith_web/live/contact_live/cluster_merge.ex`: + +```elixir + def handle_event("search", %{"query" => query}, socket) do + results = + if String.length(query) >= 2 do + current = MapSet.new(socket.assigns.members, & &1.id) + + socket.assigns.current_scope.account.id + |> Contacts.search_contacts(query) + |> Enum.reject(&MapSet.member?(current, &1.id)) + |> Enum.take(8) + else + [] + end + + {:noreply, socket |> assign(:search_query, query) |> assign(:search_results, results)} + end + + def handle_event("add-member", %{"id" => id}, socket) do + id = String.to_integer(id) + + case Contacts.get_contact(socket.assigns.current_scope.account.id, id) do + nil -> + {:noreply, socket} + + contact -> + # An added member is indistinguishable from a detected one from here on; + # the engine does not care how it got into the strip. + {:noreply, + socket + |> assign(:members, socket.assigns.members ++ [contact]) + |> assign(:selected_ids, MapSet.put(socket.assigns.selected_ids, contact.id)) + |> assign(:overrides, %{}) + |> assign(:dropped, MapSet.new()) + |> assign(:search_query, "") + |> assign(:search_results, []) + |> recompute()} + end + end +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith_web/live/contact_live/cluster_merge.ex test/kith_web/live/contact_live/cluster_merge_test.exs +git commit -m "feat(duplicates): add members to a merge by search" +``` + +--- + +### Task 3: Merging from the contacts list + +**Files:** +- Modify: `lib/kith_web/live/contact_live/index.ex` +- Modify: `lib/kith_web/live/contact_live/index.html.heex` +- Modify: `lib/kith_web/live/contact_live/cluster_merge.ex` +- Modify: `lib/kith_web/router.ex` +- Test: `test/kith_web/live/contact_live/merge_selected_test.exs` (create) + +**Interfaces:** +- Produces: the contacts index assigns `:selected_contact_ids` (MapSet) and handles `"toggle-select"` and `"clear-selection"`. "Merge selected" navigates to `/contacts/duplicates/cluster/:id?with=`. `ClusterMerge.handle_params/3` reads `with` and appends those contacts as members. + +This reuses the existing `:id` route rather than adding a second one, so every +entry point resolves through the same `handle_params/3`. + +- [ ] **Step 1: Write the failing test** + +Create `test/kith_web/live/contact_live/merge_selected_test.exs`: + +```elixir +defmodule KithWeb.ContactLive.MergeSelectedTest do + use KithWeb.ConnCase + + import Phoenix.LiveViewTest + + alias Kith.{AccountsFixtures, ContactsFixtures} + + setup %{conn: conn} do + ContactsFixtures.seed_reference_data!() + user = AccountsFixtures.user_fixture() + + a = ContactsFixtures.contact_fixture(user.account_id, %{first_name: "Ann"}) + b = ContactsFixtures.contact_fixture(user.account_id, %{first_name: "Bea"}) + c = ContactsFixtures.contact_fixture(user.account_id, %{first_name: "Cal"}) + + %{conn: log_in_user(conn, user), user: user, account_id: user.account_id, a: a, b: b, c: c} + end + + test "selecting contacts reveals a merge action", ctx do + {:ok, live, _html} = live(ctx.conn, ~p"/contacts") + + html = + live + |> element("input[phx-click='toggle-select'][phx-value-id='#{ctx.a.id}']") + |> render_click() + + refute html =~ "Merge selected" + + html = + live + |> element("input[phx-click='toggle-select'][phx-value-id='#{ctx.b.id}']") + |> render_click() + + assert html =~ "Merge selected" + assert html =~ "2 selected" + end + + test "merge selected opens the cluster screen with those members", ctx do + {:ok, live, _html} = live(ctx.conn, ~p"/contacts") + + for contact <- [ctx.a, ctx.b, ctx.c] do + live + |> element("input[phx-click='toggle-select'][phx-value-id='#{contact.id}']") + |> render_click() + end + + assert {:error, {:live_redirect, %{to: path}}} = + live |> element("a[data-role='merge-selected']") |> render_click() + + {:ok, _merge_live, html} = live(ctx.conn, path) + + assert html =~ "Merge 3 contacts" + for name <- ~w(Ann Bea Cal), do: assert(html =~ name) + end + + test "contacts from a URL are refused if they belong to another account", ctx do + other = AccountsFixtures.user_fixture() + stranger = ContactsFixtures.contact_fixture(other.account_id, %{first_name: "Zed"}) + + {:ok, _live, html} = + live(ctx.conn, "/contacts/duplicates/cluster/#{ctx.a.id}?with=#{stranger.id}") + + refute html =~ "Zed" + assert html =~ "Ann" + end +end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith_web/live/contact_live/merge_selected_test.exs` +Expected: FAIL — no `toggle-select` handler on the index. + +- [ ] **Step 3: Write minimal implementation** + +In `lib/kith_web/live/contact_live/index.ex`, add to `mount/3`: + +```elixir + |> assign(:selected_contact_ids, MapSet.new()) +``` + +and add these handlers: + +```elixir + def handle_event("toggle-select", %{"id" => id}, socket) do + id = String.to_integer(id) + selected = socket.assigns.selected_contact_ids + + selected = + if MapSet.member?(selected, id), + do: MapSet.delete(selected, id), + else: MapSet.put(selected, id) + + {:noreply, assign(socket, :selected_contact_ids, selected)} + end + + def handle_event("clear-selection", _params, socket) do + {:noreply, assign(socket, :selected_contact_ids, MapSet.new())} + end +``` + +In `lib/kith_web/live/contact_live/index.html.heex`, add a checkbox to each +contact row in the `:index` listing (place it as the first child of the row +container): + +```heex + +``` + +And add the selection bar directly above the contact list: + +```heex +
1} + class="flex items-center justify-between gap-4 rounded-[var(--radius-lg)] border border-[var(--color-accent)] bg-[var(--color-accent-subtle)] px-4 py-3" + > + + {MapSet.size(@selected_contact_ids)} selected + +
+ + <.link + data-role="merge-selected" + navigate={merge_selected_path(@selected_contact_ids)} + class="inline-flex items-center gap-1.5 rounded-[var(--radius-md)] bg-[var(--color-accent)] text-[var(--color-accent-foreground)] px-3 py-1.5 text-sm font-medium" + > + <.icon name="hero-arrows-right-left" class="size-4" /> Merge selected + +
+
+``` + +Add the path helper to `lib/kith_web/live/contact_live/index.ex`: + +```elixir + defp merge_selected_path(selected_ids) do + [first | rest] = Enum.sort(selected_ids) + ~p"/contacts/duplicates/cluster/#{first}?#{[with: Enum.join(rest, ",")]}" + end +``` + +In `lib/kith_web/live/contact_live/cluster_merge.ex`, accept the `with` +parameter. Change the `handle_params/3` head and add the appending step: + +```elixir + def handle_params(%{"id" => id} = params, _uri, socket) do + scope = socket.assigns.current_scope + contact_id = String.to_integer(id) + + cond do + not Policy.can?(scope.user, :update, :contact) -> + {:noreply, + socket + |> put_flash(:error, "You don't have permission to merge contacts") + |> push_navigate(to: ~p"/contacts")} + + cluster = DuplicateDetection.get_cluster(scope.account.id, contact_id) -> + {:noreply, socket |> load_cluster(cluster, false) |> append_from_params(params)} + + contact = Contacts.get_contact(scope.account.id, contact_id) -> + {:noreply, + socket |> load_cluster(synthetic_cluster(contact), true) |> append_from_params(params)} + + true -> + {:noreply, + socket + |> put_flash(:error, "Contact not found") + |> push_navigate(to: ~p"/contacts")} + end + end + + # Extra members passed from the contacts list. Every id is re-fetched through + # the account scope, so a hand-edited URL cannot pull in another account's + # contact. + defp append_from_params(socket, %{"with" => with_param}) when is_binary(with_param) do + account_id = socket.assigns.current_scope.account.id + existing = MapSet.new(socket.assigns.members, & &1.id) + + extra = + with_param + |> String.split(",", trim: true) + |> Enum.flat_map(fn raw -> + case Integer.parse(raw) do + {id, ""} -> [id] + _ -> [] + end + end) + |> Enum.reject(&MapSet.member?(existing, &1)) + |> Enum.map(&Contacts.get_contact(account_id, &1)) + |> Enum.reject(&is_nil/1) + + if extra == [] do + socket + else + members = socket.assigns.members ++ extra + + socket + |> assign(:members, members) + |> assign(:selected_ids, MapSet.new(members, & &1.id)) + |> recompute() + end + end + + defp append_from_params(socket, _params), do: socket +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test test/kith_web/live/contact_live/merge_selected_test.exs` +Expected: PASS, 3 tests. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix test +git add lib/kith_web/live/contact_live/index.ex lib/kith_web/live/contact_live/index.html.heex lib/kith_web/live/contact_live/cluster_merge.ex test/kith_web/live/contact_live/merge_selected_test.exs +git commit -m "feat(contacts): merge selected contacts from the contacts list" +``` + +--- + +### Task 4: Delete the wizard and the dead LiveView + +**Files:** +- Delete: `lib/kith_web/live/contact_live/merge.ex` +- Delete: `lib/kith_web/live/contact_live/duplicates.ex` +- Modify: `lib/kith_web/router.ex` +- Modify: `lib/kith_web/live/contact_live/show.html.heex` +- Modify: `lib/kith/contacts.ex` +- Test: `test/kith_web/live/contact_live/cluster_merge_test.exs` + +**Interfaces:** +- Removes: the `/contacts/:id/merge` route, `KithWeb.ContactLive.Merge`, `KithWeb.ContactLive.Duplicates`. `Contacts.merge_contacts/3` stays — the REST API still uses it. + +- [ ] **Step 1: Write the failing test** + +Append to `test/kith_web/live/contact_live/cluster_merge_test.exs`: + +```elixir + describe "old routes" do + test "the merge wizard route no longer exists", ctx do + assert_raise Phoenix.Router.NoRouteError, fn -> + get(ctx.conn, "/contacts/#{ctx.a.id}/merge") + end + end + + test "the contact page links to the cluster screen", ctx do + {:ok, _live, html} = live(ctx.conn, ~p"/contacts/#{ctx.a.id}") + + assert html =~ "/contacts/duplicates/cluster/#{ctx.a.id}" + refute html =~ "/contacts/#{ctx.a.id}/merge" + end + end +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mix test test/kith_web/live/contact_live/cluster_merge_test.exs -k "old routes"` +Expected: FAIL — the route still resolves and the show page still links to it. + +- [ ] **Step 3: Write minimal implementation** + +Remove the route from `lib/kith_web/router.ex`: + +```elixir + live "/contacts/:id/merge", ContactLive.Merge, :index +``` + +Delete both LiveViews: + +```bash +git rm lib/kith_web/live/contact_live/merge.ex +git rm lib/kith_web/live/contact_live/duplicates.ex +``` + +In `lib/kith_web/live/contact_live/show.html.heex`, find the Merge link — it +points at `~p"/contacts/#{@contact.id}/merge"` — and change its target to: + +```heex +~p"/contacts/duplicates/cluster/#{@contact.id}" +``` + +Search for any other reference before compiling: + +```bash +grep -rn "ContactLive.Merge\|/merge\"" lib/ test/ +``` + +Delete `test/kith_web/live/contact_live/merge_test.exs` if it exists — it tests +a removed module. Its behaviour is covered by +`cluster_merge_test.exs` and the slice 1 engine tests. + +`Contacts.merge_contacts/3` is **not** removed: `contact_controller.ex` still +calls it for `POST /api/contacts/merge`, which keeps its documented request +shape. Confirm with: + +```bash +grep -rn "merge_contacts" lib/ +``` + +Expect exactly two hits — the definition in `contacts.ex` and the API +controller. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `mix test` +Expected: PASS across the suite, with no compile warnings about undefined +modules. + +- [ ] **Step 5: Commit** + +```bash +mix format +mix compile --warnings-as-errors +mix credo --strict +mix test +git add -A lib/ test/ +git commit -m "refactor(duplicates): remove the merge wizard and dead duplicates live view" +``` + +--- + +## Slice 3 completion checklist + +- [ ] `mix precommit` passes. +- [ ] Spec scenarios covered: E1–E4. +- [ ] E4 (merging past a dismissal) is exercised end to end: dismiss two contacts + as not duplicates, then select both on the contacts list and merge. The + merge must succeed and the dismissed pair must end as `merged` — this is + the recovery path that replaces a dismissed-candidates view, so verify it + by hand as well as in tests. +- [ ] `grep -rn "ContactLive.Merge\|ContactLive.Duplicates" lib/ test/` returns + nothing. + +## Whole-feature verification + +After all three slices, walk the spec's §8 scenarios against the running app: + +- [ ] A four-member cluster appears as one entry and merges in one action. +- [ ] Unchecking one member and merging leaves the other pending pairs alone, and + the unchecked member does not come back. +- [ ] A five-member cluster holding two people resolves in two passes. +- [ ] `POST /api/contacts/merge` still works and now writes an audit entry. +- [ ] A `viewer` is refused the cluster screen. diff --git a/docs/superpowers/specs/2026-08-17-duplicate-merge-revamp-design.md b/docs/superpowers/specs/2026-08-17-duplicate-merge-revamp-design.md new file mode 100644 index 00000000..bb12daf7 --- /dev/null +++ b/docs/superpowers/specs/2026-08-17-duplicate-merge-revamp-design.md @@ -0,0 +1,883 @@ +# Duplicate Contact Merge Revamp — Design + +**Date:** 2026-08-17 +**Status:** Approved for planning +**Branch:** `worktree-merge-revamp` +**Reviewed:** `2026-08-17-duplicate-merge-revamp-review.md` — all findings folded +in except concern 8, rejected with evidence in §9. + +## Problem + +The duplicate merge tool has three defects that compound each other. + +**Too many steps.** `ContactLive.Merge` is a three-step wizard: search for a +contact, choose field values, review a preview. When entered from the duplicates +page — the only path most users take — step 1 is already answered and is skipped, +but the stepper still reports "Step 2 of 3". Steps 2 and 3 show disjoint data, so +the user picks field values on one screen and sees unrelated counts on the next. + +**Poor visibility.** `Contact.update_changeset` casts 27 fields. The merge wizard +exposes eight: `first_name`, `last_name`, `nickname`, `birthdate`, `description`, +`occupation`, `company`, `avatar`. The other nineteen — `middle_name`, `aliases`, +`gender_id`, `currency_id`, `deceased`, `favorite`, `is_archived`, the +`first_met_*` group, the `immich_*` group — are never shown and silently keep the +survivor's values. The preview is a list of counts ("3 notes will be combined") +with no sample of the underlying records and no rendering of the resulting +contact. + +**Pairs, not clusters.** `duplicate_candidates` stores pairs +(`contact_id`, `duplicate_contact_id`, with a check constraint enforcing +`contact_id < duplicate_contact_id`) and `Contacts.merge_contacts/3` merges +exactly two contacts. Four duplicates of one person surface as up to six +unrelated rows that must be resolved one at a time. + +## Goals + +1. One screen, no wizard. +2. Every mergeable contact field visible, with miscellaneous and historical data + summarised rather than expanded. +3. An N-contact cluster resolved in a single merge. +4. Fix the two data-loss bugs found during design (§5). + +## Non-goals + +- Changing the detection algorithm, its signals, or its scoring. +- Adding a `duplicate_groups` table. Clusters are derived, not stored. +- A "Dismissed candidates" view with restore. Dismissal remains permanent and + unlisted; the recovery path is instead the contacts-page multi-select merge + (§6), which lets a user merge two contacts directly without resurrecting the + suggestion. This gives a way to *act* on a mistaken dismissal, not a way to be + *reminded* of one — accepted. +- Automatic or bulk merging without review. + +--- + +## 1. Cluster layer + +### Derivation + +`Kith.DuplicateDetection.list_clusters(account_id, opts)` loads all `pending` and +`dismissed` candidates for the account and computes components with union-find in +memory. + +In-memory is the right call at this scale, but not because pair counts are +capped — only the name query carries `LIMIT 500` +(`duplicate_detection_worker.ex:107`); the email, phone and address queries are +uncapped. The real justification is that pending pairs per account are small in +practice and the negative-edge check below is awkward to express as a recursive +CTE. + +Pending pairs are positive edges; dismissed pairs are **negative edges** — a +record that the user already reviewed those two contacts and rejected the match. +Clustering must never place a negative edge inside a component, or a rejected +match would silently reappear by transitivity through some third contact. + +The build is therefore greedy rather than a plain union of all edges: + +1. Sort pending pairs by `{-score, contact_id, duplicate_contact_id}`. +2. For each pair, union its endpoints **only if** the resulting component would + contain no dismissed pair. +3. Skipped pairs stay `pending` and are reconsidered on the next run. + +**The sort must be fully deterministic.** Scores come from a small fixed set — +0.85 email, 0.75 phone, 0.60 address, plus 0.05 per extra signal, rounded to two +decimals — so ties are the common case, not the exception. Every email-only pair +scores exactly 0.85. Sorting on score alone leaves the winner to arbitrary row +order, which makes clustering nondeterministic between runs on identical data and +makes the F2/F3 tests flaky by construction. The contact ids break the tie. + +This yields "closest match wins". Given a dismissed A–D and a new contact E +matching A at 0.90 and D at 0.60, A–E unions first; D–E is then skipped because +it would put A and D in one component. The result is `{A, E}`, with D left +alone — not `{A, D, E}`. + +The negative-edge check is O(|C₁|×|C₂|) per candidate edge. Acceptable at this +scale; noted so it is a known cost rather than a surprise. + +A skipped edge is invisible to the user until it resolves itself: once `{A, E}` +merges, the D–E pair is repointed onto the already-dismissed A–D (§2). This is +accepted, not fixed. + +### Cluster identity + +Clusters have no database row, so the route `/contacts/duplicates/cluster/:id` +takes **any member contact id** and resolves it to the cluster containing that +contact. The canonical key used for display and generated links is the lowest +*active* member id. + +This makes the stability question moot. The canonical key does shift if a new +member with a lower id joins the cluster between runs — but because any member id +resolves, an old bookmark still lands on the right cluster. If the id belongs to +no pending cluster, the screen renders a synthetic one-member cluster for manual +merging (§6). + +Two tabs holding different derivations both recompute on load and both submit +against current data; the resolution validation in §2 catches the stale one and +aborts rather than merging something the user did not see. + +### Trashed members + +A pending pair survives its contact being trashed by ordinary deletion. Cluster +members are therefore filtered to active contacts (`deleted_at IS NULL`), and any +cluster left with fewer than two members is dropped from the listing. + +### Membership is a selection + +The cluster is fixed as detected. Every member carries a checkbox, checked by +default. **Both** the merge and the "Not duplicates" action operate on the +checked set only. + +**Merging commits a review of the checked members.** Unchecking a member and +merging the rest is a statement that the unchecked member is not one of them, so +its pairs to the merged members become `dismissed` (§2). The worker already skips +any pair recorded as `pending` or `dismissed`, so the rejected match is never +regenerated, and the negative-edge rule above stops it reappearing by +transitivity. The cluster does not come back. + +**Pairs between two unchecked members are never touched by either action.** This +is what lets a five-member cluster holding two distinct people be resolved in two +passes: merge person A's three records with person B's two unchecked, and B's +internal pair survives to recluster on the next run. + +Leaving the page without acting commits nothing — the cluster returns unchanged. +There is no per-member "not a duplicate" action; unchecking plus acting is that +action. + +### Primary selection + +The default primary is the member with the most attached records — the sum of its +notes, activities, calls, life events, photos, documents, addresses and contact +fields — tie-broken by earliest `inserted_at`. This moves the fewest rows and +preserves the contact id holding the most history, which matters because CardDAV +clients hold URLs keyed on it. The user can override by clicking a member chip. + +--- + +## 2. Merge engine + +`Kith.Contacts.merge_cluster(scope, survivor_id, loser_ids, resolution)` replaces +`merge_contacts/3`. One `Ecto.Multi`, one transaction. + +The engine does **not** derive anything. It validates the resolution it is handed +and applies it. Resolution is computed by a shared module (§3) called by every +caller, so what the user approved is exactly what is written, and a concurrent +edit between render and submit fails loudly instead of silently changing the +outcome. + +### Resolution payload + +```elixir +%{ + fields: %{ + first_name: "Sarah", + birthdate: ~D[1990-03-02], + nickname: :clear, + aliases: ["Sarah K.", "김지영"], + gender_id: 2, + first_met_through_id: :clear + }, + drop: %{contact_fields: [88, 91], addresses: [], tags: [12]} +} +``` + +`fields` is complete — every mergeable field, with a concrete value or `:clear`. +Every value must be held by one of the selected members, or be `:clear`. Array +fields such as `aliases` carry their fully resolved array; they are not +expressible through `drop`, which is id-based (see §3). + +`drop` keys are entity types and values are **owning-record ids** — for `tags` +the `tag_id`, since `contact_tags` is a bare join table with no Ecto schema. +Every id must belong to a selected member. Both maps are validated server-side; a +mismatch aborts the transaction. + +`drop` is destructive by design (§4): a dropped record belonging to a loser is +simply not remapped and rides to trash with its owner, but a dropped record +belonging to the survivor is deleted outright, with no trash behind it. + +### Steps + +1. **Lock and validate** — `SELECT ... FOR UPDATE` on all member rows, then + confirm they are in `scope.account`, all active (`deleted_at` is nil), + survivor not among losers, at least one loser, resolution well-formed. The + lock prevents two users merging overlapping clusters concurrently; the + "all active" check alone is not a lock. +2. **Apply scalars** to the survivor via changeset. +3. **Remap** every loser's owned records to the survivor. +4. **Deduplicate** the survivor's multi-valued records. +5. **Discard** records listed in `drop`. +6. **`last_talked_to`** — the maximum across all merged members. +7. **Cancel Oban jobs** for losers' reminders. +8. **Soft-delete** losers. +9. **Resolve candidate pairs** (below). +10. ~~**Enqueue** a `DisplayNameRecomputeWorker` job for the survivor, since + `display_name` is computed and never merged directly (§3).~~ + **Amended 2026-08-18 during slice 1 (ruling R15) — no job is enqueued.** This + step rested on a false premise about that worker. `DisplayNameRecomputeWorker` + is account-wide: its only `perform/1` clause matches + `%{"account_id" => _, "display_name_format" => _}` and it exists to rewrite + every contact in an account after a user changes their `display_name_format` + preference (`lib/kith/accounts.ex`, `update_user_profile/2`). A job enqueued + as `%{contact_id: survivor.id}` can never match it — it fails all three + attempts on every merge, leaving a discarded job and error-channel noise — + and its `unique: [keys: [:account_id]]` declaration makes two merges within + 60 seconds collide, so the enqueue is not even reliably present. + None of it is needed: `Contact.update_changeset/2` recomputes `display_name` + synchronously via `compute_display_name/1`, and step 2 applies the scalars + through exactly that changeset, so the survivor's display name is already + correct when the transaction commits. Adding a per-contact clause to the + worker was considered and rejected: that worker formats from first/last only, + so it would strip the middle name the changeset had just included. +11. **Audit log** — one `:contact_merged` entry naming every loser, the full + resolution, and every dropped record. + +The audit entry records each dropped record's type, value and original owner, so +a destructive `drop` leaves a trail. Note this writes contact values — emails, +phone numbers, addresses — into `audit_logs` metadata, where they sit outside the +contact soft-delete and purge lifecycle and are unbounded in size. Accepted as +the cost of an auditable destructive action. + +### Entities remapped + +Existing: notes, `activity_contacts`, calls, life events, documents, photos, +addresses, contact fields, `contact_tags`, reminders, relationships. + +**Added, and currently missing — see Bug 1:** debts, gifts, pets, tasks, +conversations, `immich_candidates`, `reminder_instances`, and inbound +`first_met_through` references. + +### Candidate pair resolution + +Both actions are scoped to the checked members. Let *S* be the checked set and +*U* the unchecked members of the cluster. + +| Pair | Merge | Not duplicates | +|---|---|---| +| Both endpoints in *S* | `merged` | `dismissed` — written as the full clique over *S* | +| One in *S*, one in *U* | `dismissed` | `dismissed` | +| Both endpoints in *U* | untouched | untouched | + +The clique matters because detection often links a cluster as a chain: A–B, B–C, +C–D is three pairs, not six. Dismissing only the detected pairs would leave no +negative edge between A and C, so a later contact matching both would reunite +them by transitivity. Writing the clique over *S* closes that hole. It is written +only over *S*, never over *U*, so it can never assert something about a pair the +user excluded. + +**Repointing.** Every remaining pair referencing a loser is rewritten to +reference the survivor, since the survivor now *is* that contact. Without this, a +dismissal recorded against a loser evaporates when the loser is trashed and the +rejected match returns on the next scan. + +Repointing must respect the table's constraints: normalize so +`contact_id < duplicate_contact_id`, drop self-pairs, and on collision with an +existing row keep the strongest status — `merged` over `dismissed` over +`pending`. Implemented as delete-then-insert rather than a bare `update_all`, +because the unique index on `(account_id, contact_id, duplicate_contact_id)` will +otherwise reject the rewrite. + +**Invariant: a `merged` row always has at least one trashed endpoint.** Rule 1 +only produces `merged` for pairs inside *S*, every non-survivor member of which +is soft-deleted in step 8; loser-to-loser pairs collapse to self-pairs during +repointing and are dropped. This matters because neither the clustering in §1 nor +the worker's existing-pair check considers `merged` rows — so a `merged` row +between two *active* contacts would be a permanent silent block, never clustered +and never re-detected. The invariant guarantees none exists. Rows are ultimately +removed by `on_delete: :delete_all` on both contact FKs when the loser is purged. + +The blanket `dismiss_candidates_for_contact/2` call is removed — see Bug 2. Note +the distinction: that call dismisses pairs the user never looked at, which is the +bug. These rules dismiss only pairs the user reviewed by unchecking. + +--- + +## 3. Auto-resolution rules + +`Kith.Contacts.MergeResolution.resolve(members)` returns the complete field map +for a set of members. It is the single implementation: the LiveView calls it to +render and re-calls it on every selection change; the API adapter calls it to +build its payload. The engine never calls it. + +Resolution is recomputed from scratch whenever the member selection changes. +Explicit user choices made before a selection change are **not** retained. + +### Scalar fields + +| Distinct non-null values | Behaviour | +|---|---| +| 0 | Empty. No decision. | +| 1 | Auto-resolved. Attribution shown ("only S. Kim has this"). | +| 2+ | Conflict. Toggle, defaulting to the value held by the most members, tie-broken by most recently updated. | + +Comparison trims whitespace. A resolved field remains user-changeable, including +an explicit "Leave empty" — except for non-clearable fields. + +**Non-clearable fields.** `first_name` is `validate_required` in both changesets +(`contact.ex:103`, `contact.ex:140`), so `:clear` on it would abort the +transaction at step 2 with an error the UI never predicted. The registry derives +the non-clearable set from the changeset's `validate_required` list rather than +maintaining it by hand, and §4's segmented control omits "Leave empty" for those +rows. Today that set is exactly `first_name`. + +**Excluded from the registry.** `display_name` is computed from the name fields +and recomputed asynchronously by `DisplayNameRecomputeWorker`; it is never a +mergeable field, and the merge enqueues a recompute instead (§2). Also excluded: +`account_id`, `deleted_at`, and the timestamps. + +### Array fields + +`aliases` is `{:array, :string}` on `contacts` — its elements have no ids, so it +cannot be expressed through the id-based `drop` map. It resolves as a **scalar +array field**: the union of all members' aliases, deduplicated on the string, +carried in `fields` as a complete array. The UI renders it as a checklist in +Contact details for consistency, but unchecking edits the submitted array rather +than adding to `drop`. + +### Association ids + +`gender_id`, `currency_id` and `first_met_through_id` are `belongs_to` ids and +follow the scalar rules above, resolving to an id rather than a label. The UI +renders the associated record's name. + +`first_met_through_id` additionally needs a **self-reference clear**. Bug 1 and +D3 cover *inbound* references — other contacts pointing at a loser. Two outbound +cases remain: + +- the survivor's `first_met_through_id` points at a loser, or +- a loser's `first_met_through_id` points at the survivor. + +After remap either becomes a contact met through itself, which +`validate_first_met_through_account` does not catch. When the resolved +`first_met_through_id` is any merged member, it is set to `nil`. + +### Multi-valued data + +Unioned, then deduplicated on a per-type key: + +| Type | Dedupe key | +|---|---| +| Contact fields | field type + normalized value | +| Addresses | `line1` + `postal_code`, case-insensitive, trimmed | +| Tags | `tag_id` | + +Dropped duplicates are shown struck through with their source, not hidden. + +### Boolean and flag fields + +Policy is "the most engaged interpretation wins": + +- `favorite` — true if any member is favorited. +- `deceased` — true if any member is marked deceased. `deceased_at` takes the + earliest **non-nil** date among members marked deceased; if every such member + has a nil date, it stays nil. +- `is_archived` — false if any member is active. Since the column defaults to + false, this means a merge involving any non-archived member un-archives the + result. Intended: a contact worth merging is a contact in use. + +### Immich fields + +`immich_person_id`, `immich_person_url`, `immich_status` and +`immich_last_synced_at` move **as one unit** — they are meaningless apart, and a +survivor marked `linked` while carrying another record's sync timestamp is +corrupt state. + +They render as a **single editable row** listing each linked member by display +name and last sync date, rather than four rows of opaque ids. The default is: +the survivor's link if it has one; otherwise the only linked member's; otherwise +the most recently synced. The user can pick another linked member or unlink. + +Per ADR-007 the integration is read-only, so nothing is written back to Immich +and a wrong choice is repairable by re-linking from the contact page. + +### History + +Notes, activities, calls, life events, photos, documents, reminders, gifts, +debts, pets, tasks and relationships are never a decision. They always move to +the survivor and are reported as counts. + +--- + +## 4. Screen + +One screen at `/contacts/duplicates/cluster/:id`, replacing the wizard. + +**Member strip** — a chip per member with a checkbox, avatar, display name, and +its best linking evidence ("86% · phone"). The primary chip is marked. Unchecking +dims the chip and recomputes every section below. An **Add contact** search sits +at the end of the strip for manually merging contacts detection did not link +(§6). + +**Three folded sections**, each an ordinary disclosure whose header states what it +holds, what the engine already did, and whether anything needs a decision: + +1. **Identity** — every mergeable scalar field as a row, plus the single grouped + Immich row. Resolved rows show the value plus attribution and are + click-to-change, opening in place as a segmented control including "Leave + empty" — omitted for non-clearable fields (§3). Conflicting rows are already + segmented controls, tinted and dotted, sitting in their natural schema + position rather than lifted to the top. +2. **Contact details** — emails, phones, addresses, tags and aliases as + checklists of real values with source labels. The list is the flat union; + unchecking any value excludes it from the merge, including values the survivor + already owns, which are deleted (§2). +3. **Carried over as-is** — history entity counts only. + +**A section holding an unresolved conflict opens itself.** Sections with nothing +contested stay folded and say so. The page is therefore short when nothing needs +attention and unfolds exactly where the user is needed. + +**Footer** — what will happen in plain language, a secondary **"Not duplicates"** +action, and the primary "Merge N contacts". Both act on the checked members only. + +The footer must state both halves of the outcome, because unchecking is a +permanent review (§1): what happens to the merged contacts ("2 contacts move to +trash and stay recoverable for 30 days") **and** what happens to the unchecked +ones ("Sarah J. Kim will be marked as not a duplicate and won't be suggested +again"). The second half is the consequence a user is most likely to trigger +without realising. + +**When validation aborts** — because another session edited or merged a member +between render and submit — the screen keeps the user's place rather than +discarding it: an inline error names what changed ("S. Kim was merged by another +session"), and a Reload action rebuilds the cluster from current data. Field +choices are lost on reload, consistent with §3's recompute rule. + +The duplicates index at `/contacts/duplicates` lists clusters instead of pairs. +Each row shows member avatars, the top score, match reasons, and links to the +cluster screen. + +### Design reference + +Mockups of the four interaction models considered, and the approved design: +https://claude.ai/code/artifact/e9e42da5-121f-4e76-96c8-ce3427c88774 + +--- + +## 5. Bugs to fix + +### Bug 1 — merging orphans six kinds of record + +`Contacts.merge_contacts/3` (`lib/kith/contacts.ex:1699`) does not remap `debts`, +`gifts`, `pets`, `tasks`, `conversations`, `immich_candidates`, or inbound +`first_met_through` references. Those records stay attached to the soft-deleted +non-survivor and disappear from the UI. Fixed by the expanded remap list in §2. + +### Bug 2 — pairs the user never saw are dismissed + +After a merge, `dismiss_candidates_for_contact/2` runs for both contacts +(`lib/kith_web/live/contact_live/merge.ex:174`), dismissing every other pending +pair involving the survivor. `DuplicateDetectionWorker` then skips those pairs +permanently, because it excludes anything already recorded as `pending` **or** +`dismissed` (`lib/kith/workers/duplicate_detection_worker.ex:70`). + +This destroys pairs the user never looked at. Merging A and B wipes the pending +A–C and A–D pairs even though the user was never shown them, and no rescan brings +them back. Fixed by the targeted pair resolution in §2, which only dismisses +pairs the user reviewed by unchecking. + +--- + +## 6. Compatibility and entry points + +**REST API.** `POST /api/contacts/merge` currently accepts `survivor_id` and +`non_survivor_id` and calls `merge_contacts/3` with no field choices. It keeps +its request shape: the controller calls `MergeResolution.resolve/1` on the two +contacts and passes the resulting complete map to +`merge_cluster(scope, survivor_id, [non_survivor_id], resolution)`. + +Two deliberate behaviour improvements for API clients: + +- Today a loser's `middle_name` is discarded even when the survivor has none; + under the shared resolver that gap is filled. A survivor's existing value is + never overridden, so no client loses data it previously kept. +- The API merge path writes **no audit entry at all** today + (`contact_controller.ex:218` — compare `restore` immediately below it, which + does log). Moving audit into the engine fixes that. + +`GET /api/duplicates` continues to return pairs. Cluster listing is a LiveView +concern for now. + +**Dead code.** `KithWeb.ContactLive.Duplicates` is unrouted and unreferenced — +the live page is `ContactLive.Index` with the `:duplicates` action. Delete it as +part of this work. + +**Routes.** `/contacts/:id/merge` is removed along with `ContactLive.Merge`, and +replaced by `/contacts/duplicates/cluster/:id`. + +**Manual merge — search.** Today the wizard's step 1 lets a user merge any two +contacts by search, independent of detection. That path must survive. The cluster +screen accepts any contact id (§1); when the id belongs to no pending cluster, it +renders that one contact as its only member with an **Add contact** search — +backed by the existing `Contacts.search_contacts/2` — that appends members to the +strip. The "Merge" action on the contact show page points here. + +**Manual merge — multi-select.** The contacts index gains selection checkboxes +and a **Merge selected** action that opens the cluster screen pre-populated with +the chosen contacts. This is the primary path for merging contacts the user +spotted themselves, and it is also the recovery path for a mistaken dismissal: +rather than restoring a dismissed suggestion, the user selects both contacts and +merges them directly. An explicit selection overrides any dismissed pair between +the members, and after the merge that pair resolves to `merged` like any other. + +Members added by search or selection behave exactly like detected ones: checked +by default, contributing to conflict computation, removable by unchecking. The +merge itself does not distinguish their origin. + +--- + +## 7. Testing + +**Resolution** (`test/kith/contacts/merge_resolution_test.exs`, new): + +- 0 / 1 / 2+ distinct values produce empty, resolved and conflict states. +- Conflict default is the most-held value; ties break by most recently updated. +- Boolean policy: `favorite`; `deceased` with earliest non-nil `deceased_at` and + the all-nil case; `is_archived`. +- `aliases` resolves to a deduplicated union array. +- `gender_id`, `currency_id` resolve as ids; `first_met_through_id` pointing at + any merged member resolves to `nil`. +- Immich fields resolve as a unit under each branch of the default rule. +- `display_name` never appears in the resolved map; `first_name` is reported + non-clearable. + +**Engine** (`test/kith/contacts_merge_test.exs`, extended): + +- Three- and four-contact merges remap every entity type, including the six + previously missed. +- Deduplication for each multi-valued type. +- `drop` excludes loser-owned records and deletes survivor-owned ones; both are + recorded in the audit entry. +- Invalid `drop` ids, and scalar values held by no selected member, abort. +- `:clear` on `first_name` is rejected before the changeset runs. +- `last_talked_to` takes the maximum across members. +- A `DisplayNameRecomputeWorker` job is enqueued for the survivor. +- Cross-account members are rejected; trashed members are rejected. +- Inbound `first_met_through` references are repointed; outbound self-references + are cleared. +- Every `merged` row produced has at least one trashed endpoint. + +**Clusters** (`test/kith/duplicate_detection_test.exs`, extended): + +- Transitive pairs form one cluster; disjoint pairs stay separate. +- Tied scores cluster deterministically — the same input produces the same + components across repeated runs and across insertion orders. +- Any member contact id resolves to its cluster; the canonical key is the lowest + active member id. +- Members trashed outside the merge flow are excluded, and clusters left with + fewer than two members are dropped. +- Pair resolution after merge and after "Not duplicates", for all three rows of + the §2 table. +- The clique is written over the checked set only, never over unchecked members. +- Repointing respects the ordering check constraint and the unique index, and + keeps the strongest status on collision. +- Pairs in other clusters are untouched by a merge — the regression test for + Bug 2. + +**The handled-cluster scenario**, end to end, as its own test: + +- Given a cluster `{A, B, C, D}`, merging `A + B + C` with D unchecked leaves + A–D `dismissed` and repoints B–D and C–D onto A. +- Re-running `DuplicateDetectionWorker` does not recreate A–D and does not + produce a cluster containing both A and D. +- A new contact E matching A at 0.90 and D at 0.60 clusters as `{A, E}`. D is + excluded, and the D–E pair remains `pending`. +- Reversing the scores puts E with D instead, and A is excluded. + +**The two-people scenario**, end to end, as its own test: + +- Given a cluster `{A, B, C, D, E}` where D–E are duplicates of each other, + merging `A + B + C` leaves D–E `pending`. +- The next detection run produces a `{D, E}` cluster, which merges independently. +- Neither resulting contact ever reclusters with the other. + +**LiveView** (`test/kith_web/live/contact_live/cluster_merge_test.exs`, new): + +- Conflicts render as toggles; resolved fields render with attribution. +- A section with a conflict renders open; a fully resolved one renders folded. +- Non-clearable fields render without a "Leave empty" option. +- Unchecking a member recomputes conflicts and counts, and discards any explicit + field choices. +- Merge submits the resolved payload and redirects to the survivor. +- "Not duplicates" dismisses the clique over the checked members and leaves + unchecked-to-unchecked pairs pending. +- A member merged by another session between render and submit produces an inline + error naming it, with a Reload action, rather than a silent partial merge. +- A bare contact id with no pending cluster renders a one-member screen, and + adding a contact by search appends it to the strip. +- Selecting contacts on the index and choosing Merge selected opens the cluster + screen pre-populated. +- A `viewer` is refused both the screen and the submit. + +--- + +## 8. User scenarios + +Written as the acceptance criteria for the implementation plan. Each maps to the +section that defines its behaviour. + +### A. Finding duplicates + +**A1 — Four duplicates appear as one entry.** *(§1)* +Given four contacts detected as duplicates of one person, when I open the +duplicates page, then I see one cluster entry showing all four members, not six +pair rows. + +**A2 — Unrelated duplicates stay separate.** *(§1)* +Given two contacts duplicated with each other and two unrelated contacts +duplicated with each other, when I open the duplicates page, then I see two +cluster entries, each with two members. + +**A3 — The queue is ordered by confidence.** *(§1)* +Given several clusters, when I open the duplicates page, then they are ordered by +their highest internal match score, and each shows member avatars, that score, +and the reasons for the match. + +**A4 — Trashed contacts do not appear.** *(§1)* +Given a member of a pending cluster is trashed from the contacts page, when I +open the duplicates page, then that member is not shown, and if the cluster is +left with one member it is not listed at all. + +**A5 — Clustering is stable.** *(§1)* +Given several pairs sharing the same score, when detection runs repeatedly over +unchanged data, then the same clusters are produced every time. + +**A6 — Any member's URL opens the cluster.** *(§1)* +Given a cluster of four, when I open the cluster URL for any one of its members, +then I see the whole cluster. + +### B. Reviewing the merge + +**B1 — Every field is visible.** *(§4)* +Given a cluster, when I open it, then every mergeable scalar field is present — +including `middle_name`, `aliases`, gender, the first-met group and the flags — +not the eight the old wizard exposed. + +**B2 — Agreement resolves itself.** *(§3)* +Given three members that all have the first name "Sarah", when I open the +cluster, then the First name row shows "Sarah" with the attribution "all 3 +agree" and asks me nothing. + +**B3 — A value only one member has is kept.** *(§3)* +Given only one member has a middle name, when I open the cluster, then the row +shows that value attributed to its source, and I am not asked to choose. + +**B4 — Disagreement becomes a choice in place.** *(§3, §4)* +Given two members hold different birthdates, when I open the cluster, then the +Birthdate row is a toggle in its normal position in the Identity section, marked +as needing a decision, defaulted to the value held by the most members. + +**B5 — A section with a conflict opens itself.** *(§4)* +Given a cluster whose only conflicts are identity fields, when I open it, then +the Identity section is expanded with a "2 need a decision" chip and the Contact +details section is folded and marked as having nothing to decide. + +**B6 — A resolved field can still be changed.** *(§3, §4)* +Given a field the engine resolved on its own, when I click its value, then it +opens in place as a choice between the candidate values and "Leave empty". + +**B7 — Required fields cannot be emptied.** *(§3, §4)* +Given the First name row, when I open it, then no "Leave empty" option is +offered, and a submitted `:clear` is rejected. + +**B8 — Multi-valued data is combined and prunable.** *(§3)* +Given members holding four distinct emails and one repeated email, when I expand +Contact details, then I see the four with their sources, the repeat struck +through as a dropped duplicate, and a checkbox on each so I can exclude one — +including values the survivor already owns. + +**B9 — History is summarised, not detailed.** *(§3, §4)* +Given members with notes, activities, calls and gifts, when I open the cluster, +then the Carried over section reports counts per type and asks me nothing. + +**B10 — Immich links are one choice, not four fields.** *(§3)* +Given two members linked to different Immich people, when I open the cluster, +then I see one row listing each linked member by name and last sync date, not +four rows of opaque identifiers. + +### C. Membership and review + +**C1 — Excluding a member changes the result.** *(§1, §3)* +Given a four-member cluster, when I uncheck one member, then conflicts, counts +and attributions recompute against the remaining three, and the merge button +reads "Merge 3 contacts". + +**C2 — The consequence of excluding is stated.** *(§4)* +Given I have unchecked a member, when I read the footer, then it tells me both +that the merged contacts move to trash recoverably and that the unchecked member +will be marked as not a duplicate and not suggested again. + +**C3 — Choosing the primary.** *(§1)* +Given a cluster, when I open it, then the member with the most attached records +is the primary by default, tie-broken by earliest creation, and I can make +another member primary by clicking its chip. + +**C4 — Leaving without acting changes nothing.** *(§1)* +Given I uncheck a member and navigate away without merging, when the duplicates +page reloads, then the cluster is unchanged with all members checked. + +**C5 — Changing the selection resets field choices.** *(§3)* +Given I have overridden a conflicting field, when I then uncheck a member, then +the resolution recomputes from defaults and my override is not retained. + +### D. Merging + +**D1 — An N-way merge is one action.** *(§2)* +Given a four-member cluster, when I merge, then one transaction produces one +surviving contact, three trashed contacts, and one audit entry naming all three. + +**D2 — Nothing is orphaned.** *(§2, Bug 1)* +Given losers holding debts, gifts, pets, tasks, conversations, Immich candidates +and reminder instances, when I merge, then all of them appear on the survivor. +*Regression test for Bug 1.* + +**D3 — Inbound references follow.** *(§2)* +Given another contact whose `first_met_through` points at a loser, when I merge, +then that reference points at the survivor. + +**D4 — No contact is met through itself.** *(§3)* +Given the survivor was first met through one of the losers, when I merge, then +the survivor's first-met-through is cleared rather than pointing at itself. + +**D5 — Duplicated sub-records collapse.** *(§2, §3)* +Given two members share an email address and a tag, when I merge, then the +survivor has one of each. + +**D6 — Excluded values are not carried.** *(§2)* +Given I unchecked an address in Contact details, when I merge, then that address +is not on the survivor, and the audit entry records what was dropped and who +owned it. + +**D7 — The merge is atomic.** *(§2)* +Given any step fails, when I merge, then nothing is changed and I am told the +merge failed. + +**D8 — Tampering is rejected.** *(§2)* +Given a submitted scalar value that no selected member holds, or a dropped record +id belonging to no selected member, when the merge is submitted, then it is +rejected server-side. + +**D9 — Concurrent merges do not interleave.** *(§2, §4)* +Given another session merges one of my members while I have the screen open, when +I submit, then my merge aborts with an inline error naming what changed and a +Reload action — not a partial merge. + +**D10 — The display name is recomputed.** *(§2, §3)* +Given a merge that changes the survivor's name fields, when it completes, then the +survivor's `display_name` reflects those fields. +*Amended 2026-08-18 (ruling R15): originally "then a `DisplayNameRecomputeWorker` +job is enqueued for the survivor". That worker is account-wide and cannot act on a +single contact — see §2 step 10. The merge's own changeset recomputes +`display_name` synchronously, so the criterion is now asserted on the result +rather than on an enqueued job.* + +### E. Manual merge + +**E1 — Merging two contacts detection never linked.** *(§6)* +Given a contact with no pending cluster, when I choose Merge from its page, then +the cluster screen opens with that contact as its only member and an Add contact +search. + +**E2 — A manually added member behaves like a detected one.** *(§6)* +Given I add a contact by search, when it joins the strip, then it is checked, +contributes to conflicts and counts, and can be unchecked. + +**E3 — Merging from the contacts list.** *(§6)* +Given I select three contacts on the contacts page, when I choose Merge selected, +then the cluster screen opens with those three as its members. + +**E4 — Merging past a dismissal.** *(§6)* +Given I previously dismissed A and D as not duplicates and later realise they are +the same person, when I select both on the contacts page and merge, then the +merge proceeds and the dismissed pair resolves to `merged`. + +### F. After the merge + +**F1 — A handled cluster does not return.** *(§1, §2)* +Given cluster `{A, B, C, D}` where I merged A, B and C with D unchecked, when the +detection worker runs again, then A–D is not recreated and no cluster contains +both A and D. + +**F2 — A new match joins the closest contact only.** *(§1)* +Given A–D was dismissed by F1 and a new contact E matches A at 0.90 and D at +0.60, when the worker runs, then the cluster is `{A, E}` — D is excluded, and A +and D are not reunited by transitivity through E. + +**F3 — The reverse holds.** *(§1)* +Given the same setup with E matching D more strongly than A, when the worker +runs, then the cluster is `{D, E}` and A is excluded. + +**F4 — Other clusters are untouched.** *(§2, Bug 2)* +Given a merge in one cluster, when it completes, then pending pairs belonging to +unrelated clusters are still pending and still cluster. +*Regression test for Bug 2.* + +**F5 — Two people in one cluster resolve in two passes.** *(§1, §2)* +Given a cluster `{A, B, C, D, E}` where D and E are duplicates of each other but +not of A, when I merge A, B and C with D and E unchecked, then the D–E pair is +untouched and the next detection run offers `{D, E}` as its own cluster. + +**F6 — Marking the checked members as not duplicates.** *(§1, §2)* +Given the checked members do not belong together, when I choose Not duplicates, +then every pair among them is dismissed, pairs to unchecked members are +dismissed, pairs between two unchecked members are left alone, and the group does +not return. + +### G. Compatibility + +**G1 — The existing API keeps working, and improves.** *(§6)* +Given a client posting `survivor_id` and `non_survivor_id` to +`/api/contacts/merge`, when it is called, then the merge succeeds, the survivor +gains any field it had no value for, no existing survivor value is overridden, an +audit entry is written, and the survivor is returned. + +**G2 — Permissions are enforced.** *(§2)* +Given a `viewer`, when they attempt to open or submit a cluster merge, then it is +refused by `Kith.Policy`. + +**G3 — Account isolation holds.** *(§2)* +Given contact ids from another account, when a merge is submitted, then it is +rejected. + +--- + +## 9. Review disposition + +All findings from `2026-08-17-duplicate-merge-revamp-review.md` are folded in +except one. + +**Concern 8 — "G2 is an acceptance criterion for a feature that does not exist" — +is rejected.** The claim is that no authorization enforcement exists. It does: +`Kith.Policy` (`lib/kith/policy.ex`) is a complete role-based module — `admin` +full access, `editor` CRUD minus account and user management, `viewer` +read-only — exposing `can?/3`, an `authorize!/3` that raises, and a +`Plug.Exception` implementation mapping to 403. It is called in 122 places across +`lib/`, including `contact_controller.ex:102` for contact updates and +`merge.ex:32` in the very LiveView being replaced. A `viewer` is already denied +`:update, :contact`, so merge is gated today. G2 costs one `Policy.can?/3` call +and stays in scope. + +The review's other correction to §1 is accepted and applied: the claim that +pending pairs are bounded by 500 was wrong — only the name query carries that +limit. + +### Slicing + +The review's restructure is adopted; the original four slices were not +independently shippable. Slice 1 assumed the existing wizard could adapt to the +new engine, but the wizard passes a two-contact chooser +(`%{"first_name" => "survivor"}`), not a resolution map — real work that a later +slice deletes. Slice 2 shipped a cluster list linking to a screen that did not +exist yet. + +1. **Engine, resolver and bug fixes.** `MergeResolution`, `merge_cluster/4`, the + expanded remap list, pair resolution, repointing, locking. API adapted. The + wizard left calling a thin `merge_contacts/3` shim over the new engine, so no + wizard rework is done and then thrown away. + Covers D1–D10, F4, G1, G3, Bug 1, Bug 2. +2. **Clusters and screen, together.** `list_clusters/2`, negative-edge building, + deterministic ordering, trashed-member filtering, the index flip and the + cluster LiveView in one deploy. + Covers A1–A6, B1–B10, C1–C5, F1–F3, F5–F6, G2. +3. **Manual merge and cleanup.** Add-contact search, contacts-index multi-select + with Merge selected, route changes, deleting `ContactLive.Merge` and the dead + `ContactLive.Duplicates`. + Covers E1–E4. diff --git a/docs/superpowers/specs/2026-08-17-duplicate-merge-revamp-review.md b/docs/superpowers/specs/2026-08-17-duplicate-merge-revamp-review.md new file mode 100644 index 00000000..28537624 --- /dev/null +++ b/docs/superpowers/specs/2026-08-17-duplicate-merge-revamp-review.md @@ -0,0 +1,247 @@ +# Duplicate Merge Revamp — Design Review + +**Date:** 2026-08-17 +**Reviews:** `2026-08-17-duplicate-merge-revamp-design.md` +**Status:** Open — items below are unresolved unless marked otherwise +**Verified against:** worktree `merge-revamp`, branch `worktree-merge-revamp` + +Short version: a strong design. The best parts are genuinely good. It has one +under-specified core (cluster identity / route), a handful of concrete gaps that +will bite during implementation, and a slicing plan that does not hold up. + +--- + +## What checks out + +All three code citations in the design are exact: + +| Design claim | Verified | +|---|---| +| `Contacts.merge_contacts/3` at `lib/kith/contacts.ex:1699` | ✅ exact line | +| `dismiss_candidates_for_contact/2` at `lib/kith_web/live/contact_live/merge.ex:174` | ✅ exact line | +| Worker skips `pending` **or** `dismissed` at `lib/kith/workers/duplicate_detection_worker.ex:70` | ✅ `where: d.status in ["pending", "dismissed"]` | + +**Bug 1 is real.** The remap chain in `merge_contacts/3` covers notes, +`activity_contacts`, calls, life events, documents, photos, addresses, contact +fields, `contact_tags`, reminders and relationships. It omits `debts`, `gifts`, +`pets`, `tasks`, `conversations`, `immich_candidates`, `reminder_instances` and +`first_met_through` — all of which exist as associations on `Kith.Contacts.Contact`. + +**Bug 2 is real and nasty.** Worker line 70 means a dismissal is permanent, and +the blanket post-merge dismissal wipes pairs the user never saw, with no rescan +to recover them. + +**The negative-edge clustering rule is the strongest idea in the document.** +Plain union-find over pending edges would resurrect dismissed matches by +transitivity. §1 catches that, works the example, and accepts the residual cost +explicitly ("A skipped edge is invisible to the user until it resolves itself"). + +**The clique-over-*S* argument in §2 is likewise subtle and correct.** Detection +produces chains (A–B, B–C), so dismissing only the detected pairs leaves no +negative edge between A and C. + +**The design undersells one win.** The current API merge path +(`lib/kith_web/controllers/api/contact_controller.ex:218`) writes **no** audit +entry at all — only the LiveView does. Moving audit into the engine fixes that +silently. Worth stating in §6 as a behaviour improvement. + +--- + +## Blocking gaps + +### 1. Cluster identity is never defined — and it is the route + +§4 routes to `/contacts/duplicates/cluster/:id` while §1 insists clusters are +derived, not stored. §6 says the screen "accepts either a detected cluster key or +a bare contact id", disambiguating by whether the id belongs to a pending +cluster. So `:id` is presumably the minimum member contact id — never stated, and +load-bearing: + +- Does the URL change when a new member joins and the min id shifts? +- Is the URL bookmarkable across detection runs? +- What happens when two tabs hold different derivations of the same cluster? + +**Action:** define the cluster key explicitly in §1, and state its stability +guarantees. + +### 2. `first_name` is `validate_required` — "Leave empty" cannot apply to every field + +§3 says a resolved field "remains user-changeable, including an explicit 'Leave +empty'", and the resolution payload allows `:clear` for any field. +`Contact.update_changeset` requires `first_name`. Clearing it fails at step 2 +with a changeset error the UI never predicted. + +**Action:** add a non-clearable field set to the registry in §3, and have §4's +segmented control omit "Leave empty" for those rows. + +### 3. `aliases` cannot be expressed in the `drop` payload + +§4 puts aliases in the Contact details checklist alongside emails, phones, +addresses and tags; §3 gives them a dedupe key. But `aliases` is +`{:array, :string}` on `contacts` — elements have no ids — while `drop` is an id +map: `%{contact_fields: [88], addresses: [], tags: [12]}`. + +**Action:** either move aliases to the scalar registry as a merged array value, +or give `drop` a non-id shape for them. + +Related, smaller: `tags: [12]` is ambiguous. `contact_tags` is a bare join table +with no Ecto schema, so is `12` a `tag_id` or a join row id? Say which. + +### 4. Outbound `first_met_through` self-reference + +Bug 1 and D3 cover *inbound* references being repointed. Nothing covers: + +- the survivor's own `first_met_through_id` pointing at a loser, or +- a loser's `first_met_through_id` pointing at the survivor. + +After remap, that is a contact met through itself. + +`first_met_through_id` is also not discussed as a mergeable scalar at all — nor +are `gender_id` / `currency_id` explicitly, though §1 name-drops `gender` and +`currency` as unexposed fields. + +**Action:** give `belongs_to` ids their own rule in §3, including the self- +reference clear. + +### 5. Score ties make greedy clustering nondeterministic + +§1's worked example assumes distinct scores (0.90 vs 0.60). In practice +`compute_merged_score/1` emits a small fixed set of values: + +| Signal | Base | +|---|---| +| email | 0.85 | +| phone | 0.75 | +| address | 0.60 | +| name | `similarity(display_name)` | + +plus `0.05 × (signal_count - 1)`, rounded to 2 decimals. Ties are common, and +"sort pending pairs by score, descending" leaves the winner to arbitrary row +order — so which component an edge lands in can change between runs on identical +data. + +**Action:** add a deterministic tiebreak (score, then `contact_id`, then +`duplicate_contact_id`) and state it. Otherwise the F2/F3 tests are flaky by +construction. + +### 6. `merged`-status rows are invisible to both layers + +§2's collision precedence keeps `merged` over `dismissed`. But §1 clusters over +`pending` + `dismissed` only, and the worker's dedupe check (line 70) also looks +only at `pending`/`dismissed`, with the unique index plus `on_conflict: :nothing` +behind it. Any `merged` row between two *active* contacts is therefore a +permanent silent block: never clustered, never re-detected, never surfaced. + +No case was found where such a row survives — the loser is trashed, then +FK-cascaded on purge (`on_delete: :delete_all` on both contact FKs in +`20260321173108_create_duplicate_candidates.exs`). + +**Action:** state the invariant ("a `merged` row always has at least one trashed +endpoint") rather than leaving it to be rediscovered. + +### 7. No locking; the concurrency story is thinner than claimed + +§2 says a concurrent edit "fails loudly instead of silently changing the +outcome" — true, since the submitted value would no longer be held by any member. +But: + +- No recovery path is described. The user sees an abort and must redo every choice. +- Nothing prevents two users merging overlapping clusters simultaneously. Step 1's + "all active" check is not a lock. + +**Action:** `SELECT ... FOR UPDATE` on the member rows in step 1, and a sentence +in §4 on what the user sees when validation aborts. + +### 8. G2 is an acceptance criterion for a feature that does not exist + +"Given a user without contact update permission…" — roles exist +(`admin` / `editor` / `viewer` on `Kith.Accounts.User`, `@valid_roles` at +`user.ex:52`), but `viewer` appears in the web layer **only** in +`kith_ui.ex` as a badge colour. There is no authorization enforcement anywhere. + +G2 therefore either quietly expands this work to include building an authz layer, +or it gets dropped during implementation. + +**Action:** decide which, now. If out of scope, delete G2. + +--- + +## Smaller notes + +- **§1 field count.** "Roughly thirty other contact fields" overstates it — the + castable field set is ~28 total, so about 20 are unexposed. The point stands; + the number does not. +- **§2 audit PII.** "every dropped record with its type, **value** and original + owner" writes emails, phone numbers and addresses into `audit_logs` metadata, + unbounded in size, outside the contact soft-delete/purge lifecycle. Probably + fine — deserves an acknowledging sentence rather than being a side effect of + "leave a trail". +- **§3 `deceased`.** "true if any member is marked deceased, taking the earliest + `deceased_at`" is undefined when the deceased member has a nil `deceased_at` + and another has a date. +- **§3 `is_archived`.** "false if any member is active" means a merge almost + always un-archives, since the column defaults to false. Presumably intended; + worth one line. +- **§1 the 500 cap.** "name matching already caps at 500 rows per scan" is true + (`LIMIT 500` on the name query, `duplicate_detection_worker.ex:107`), but the + email, phone and address queries are uncapped — total pending pairs are *not* + bounded by 500. In-memory union-find is still the right call; the justification + is wrong. +- **§1 greedy union cost.** The "would the component contain a dismissed pair" + check is O(|C₁|×|C₂|) per edge. Fine at this scale; worth stating. + +--- + +## The slicing does not work + +§8's suggested slicing claims four "independently shippable" slices. Two are not. + +**Slice 1** says "the existing wizard and API adapt to the new functions". The +wizard passes `%{"first_name" => "non_survivor"}` — a per-field *chooser* keyed to +exactly two contacts. The new engine demands a complete resolution map with +concrete values. Adapting the three-step wizard to that is real work that slice 3 +then deletes. + +**Slice 2** ships a cluster *list* whose rows link to a cluster screen that does +not exist until slice 3. + +**Proposed restructure:** + +1. **Engine + resolver + bug fixes.** `MergeResolution`, `merge_cluster/4`, + expanded remap list, pair resolution, repointing. API adapted. The wizard left + calling a thin `merge_contacts/3` shim over the new engine — no wizard rework. + Covers D1–D8, F4, G1, G3, Bug 1, Bug 2. +2. **Clusters + screen, together.** `list_clusters/2`, negative-edge building, + trashed-member filtering, the index flip and the cluster LiveView in one + deploy. Covers A1–A4, B1–B9, C1–C5, F1–F3, F5–F6. +3. **Manual merge and cleanup.** Add-contact search, route changes, deleting + `ContactLive.Merge` and the dead `ContactLive.Duplicates`. Covers E1–E2. + +--- + +## The one thing worth arguing with + +**The non-goal on dismissal undo should be pulled into scope.** + +§1 makes unchecking a checkbox a permanent, irreversible statement. §2 amplifies +it into a full clique of negative edges. §4's footer text is the only thing +standing between a user and silently destroying a correct future match — with no +dismissed-pairs view and no recovery short of `psql`. + +The design names this risk honestly, which is to its credit. But the risk is +*created by this design*: today's dismissals are per-pair and far less +consequential. A read-only "Dismissed" list with a restore button is small next +to the rest of this work. + +**Recommendation:** add it as a fourth slice rather than leaving it as "the +natural follow-up". + +--- + +## Open questions for the next session + +1. What is the cluster key, concretely, and how stable is it? +2. Non-clearable fields: just `first_name`, or is there a broader rule? +3. Aliases — scalar array merge, or extend `drop`? +4. Is G2 (permissions) in scope, or deleted? +5. Is the dismissal-undo slice in or out? diff --git a/lib/kith/accounts.ex b/lib/kith/accounts.ex index 8b0c90b2..ba939a55 100644 --- a/lib/kith/accounts.ex +++ b/lib/kith/accounts.ex @@ -21,15 +21,6 @@ defmodule Kith.Accounts do WebauthnCredential } - # Suppress Ecto.Multi opaque type warnings (Dialyzer false positives) - @dialyzer [ - {:nowarn_function, register_user: 1}, - {:nowarn_function, confirm_user: 1}, - {:nowarn_function, register_oauth_user: 4}, - {:nowarn_function, accept_invitation: 2}, - {:nowarn_function, do_remove_user: 1} - ] - ## Database getters @doc """ diff --git a/lib/kith/accounts/scope.ex b/lib/kith/accounts/scope.ex index 10f0e374..ee9758be 100644 --- a/lib/kith/accounts/scope.ex +++ b/lib/kith/accounts/scope.ex @@ -25,4 +25,18 @@ defmodule Kith.Accounts.Scope do end def for_user(nil), do: nil + + @doc """ + Builds a full-privilege scope from a bare account id, for background and + adapter callers that have already established which account they are acting + on. `user` is nil, so callers needing an audit actor must supply one. + + This is **not** an authorization scope: it grants access to `account_id` + without checking that anyone is entitled to it, so the id must be derived + from an already-authorized record (as `Kith.Contacts.merge_contacts/3` does, + reading it off a contact it just fetched) — never from user input. + """ + def system_for_account_id(account_id) do + %__MODULE__{user: nil, account: Kith.Repo.get!(Kith.Accounts.Account, account_id)} + end end diff --git a/lib/kith/activities.ex b/lib/kith/activities.ex index ed7c6769..bc9d9e0b 100644 --- a/lib/kith/activities.ex +++ b/lib/kith/activities.ex @@ -14,13 +14,6 @@ defmodule Kith.Activities do alias Kith.Contacts.Contact alias Kith.Repo - # Suppress Ecto.Multi opaque type warnings (Dialyzer false positives) - @dialyzer [ - {:nowarn_function, create_activity: 4}, - {:nowarn_function, update_activity: 4}, - {:nowarn_function, create_call: 2} - ] - ## Activities def list_activities(account_id, opts \\ []) do diff --git a/lib/kith/contacts.ex b/lib/kith/contacts.ex index d5c7bef6..88ca6389 100644 --- a/lib/kith/contacts.ex +++ b/lib/kith/contacts.ex @@ -21,6 +21,9 @@ defmodule Kith.Contacts do Gender, ImmichCandidate, LifeEventType, + Merge, + MergeFields, + MergeResolution, Note, PhoneFormatter, Photo, @@ -32,15 +35,6 @@ defmodule Kith.Contacts do alias Kith.Activities.{Activity, Call} alias Kith.Storage - # Suppress Ecto.Multi opaque type warnings (Dialyzer false positives) - @dialyzer [ - {:nowarn_function, set_avatar: 2}, - {:nowarn_function, confirm_immich_link: 3}, - {:nowarn_function, unlink_immich: 1}, - {:nowarn_function, merge_tags: 3}, - {:nowarn_function, merge_contacts: 3} - ] - ## Contacts def list_contacts(account_id, opts \\ []) do @@ -1686,197 +1680,151 @@ defmodule Kith.Contacts do # ── Contact Merge ────────────────────────────────────────────────────── @doc """ - Merges two contacts. The survivor keeps chosen field values and receives - all sub-entities from the non-survivor. The non-survivor is soft-deleted. - - `field_choices` is a map of `%{"field_name" => "survivor" | "non_survivor"}` - indicating which contact's value to keep for each identity field. + Merges `loser_ids` into `survivor_id`, applying `resolution`. + + See `Kith.Contacts.MergeResolution` for how a resolution is produced. + `resolution` may include an optional `:unchecked_ids` key (default `[]`) + naming every candidate contact the caller's UI presented but the user did + not include in this merge — see `Kith.Contacts.Merge.run/4` for what + omitting it costs. + + Returns `{:ok, contact}` or `{:error, reason}`, where `reason` is one of + `:not_found`, `:trashed`, `:different_accounts`, `:survivor_in_losers`, + `:no_losers`, `{:unknown_value, field}`, `{:unknown_field, field}`, + `{:not_clearable, field}`, `{:unknown_drop, key}`, or `{:invalid_fields, changeset}`. + """ + def merge_cluster(scope, survivor_id, loser_ids, resolution) do + Merge.run(scope, survivor_id, loser_ids, resolution) + end - Both contacts must belong to the same account. + @doc """ + Merges `non_survivor_id` into `survivor_id`. - Returns `{:ok, %{survivor: contact}}` or `{:error, step, changeset, changes}`. + Retained for the existing wizard and REST API. `field_choices` uses the old + `%{"field" => "survivor" | "non_survivor"}` shape; everything not named is + resolved by `legacy_resolution_fields/2` — see there for how a survivor's + existing value is protected — before `field_choices` overrides are applied + on top. """ def merge_contacts(survivor_id, non_survivor_id, field_choices \\ %{}) do - alias Kith.Activities.{Call, LifeEvent} - with {:ok, survivor} <- fetch_active_contact(survivor_id), {:ok, non_survivor} <- fetch_active_contact(non_survivor_id), :ok <- validate_merge(survivor, non_survivor) do - account_id = survivor.account_id + scope = Kith.Accounts.Scope.system_for_account_id(survivor.account_id) - Ecto.Multi.new() - # (a) Update survivor identity fields - |> Ecto.Multi.run(:update_survivor_fields, fn _repo, _changes -> - update_survivor_fields(survivor, non_survivor, field_choices) - end) - # (b) Remap notes - |> Ecto.Multi.update_all( - :remap_notes, - fn _changes -> - from(n in Note, where: n.contact_id == ^non_survivor.id) - end, - set: [contact_id: survivor.id] - ) - # Remap activity_contacts - |> Ecto.Multi.run(:remap_activity_contacts, fn repo, _changes -> - # Delete activity_contacts that would create duplicates - repo.query( - "DELETE FROM activity_contacts WHERE contact_id = $1 AND activity_id IN (SELECT activity_id FROM activity_contacts WHERE contact_id = $2)", - [non_survivor.id, survivor.id] - ) + fields = + survivor + |> legacy_resolution_fields(non_survivor) + |> apply_legacy_choices(survivor, non_survivor, field_choices) - repo.update_all( - from(ac in "activity_contacts", where: ac.contact_id == ^non_survivor.id), - set: [contact_id: survivor.id] - ) + merge_cluster(scope, survivor.id, [non_survivor.id], %{fields: fields, drop: %{}}) + end + end - {:ok, :done} - end) - # Remap calls - |> Ecto.Multi.update_all( - :remap_calls, - fn _changes -> - from(c in Call, where: c.contact_id == ^non_survivor.id) - end, - set: [contact_id: survivor.id] - ) - # Remap life_events - |> Ecto.Multi.update_all( - :remap_life_events, - fn _changes -> - from(le in LifeEvent, where: le.contact_id == ^non_survivor.id) - end, - set: [contact_id: survivor.id] - ) - # Remap documents - |> Ecto.Multi.update_all( - :remap_documents, - fn _changes -> - from(d in Document, where: d.contact_id == ^non_survivor.id) - end, - set: [contact_id: survivor.id] - ) - # Remap photos (delete duplicates by content_hash first, then move remaining) - |> Ecto.Multi.run(:remap_photos, fn repo, _changes -> - # Delete photos from non-survivor that already exist on survivor (same content_hash) - repo.query( - """ - DELETE FROM photos - WHERE contact_id = $1 - AND content_hash IS NOT NULL - AND content_hash IN ( - SELECT content_hash FROM photos WHERE contact_id = $2 AND content_hash IS NOT NULL - ) - """, - [non_survivor.id, survivor.id] - ) + @doc """ + Resolves a two-contact legacy merge via `Kith.Contacts.MergeResolution`, + then protects the survivor's own values: for every + `MergeFields.choice_fields/0` entry the survivor already holds a non-nil + value for, that value wins over whatever the resolver's conflict-default + picked — a survivor's existing value is never silently overridden, only + gap-filled when the survivor has none (design spec §6). + + The one exception is a field the resolver cleared because it pointed back + at a member being merged away (e.g. `first_met_through_id` referencing + `loser`) — restoring the survivor's raw value there would leave the field + pointing at a contact about to be soft-deleted, so the resolver's `:clear` + is left standing instead (design spec scenario D4). + + A date's year-unknown flag follows whichever date survives this protection: + restoring the survivor's `birthdate` also restores the survivor's + `birthdate_year_unknown`, so a gap-filled placeholder year is never marked + as a known one. + + Shared by the legacy wizard shim (`merge_contacts/3`) and the REST API + merge endpoint so the two two-contact callers cannot drift apart. + """ + def legacy_resolution_fields(survivor, loser) do + fields = + [survivor, loser] + |> MergeResolution.resolve(survivor.id) + |> Map.fetch!(:fields) - # Move remaining photos - {count, _} = - from(p in Photo, where: p.contact_id == ^non_survivor.id) - |> repo.update_all(set: [contact_id: survivor.id]) + member_ids = [survivor.id, loser.id] - {:ok, count} - end) - # Remap addresses - |> Ecto.Multi.update_all( - :remap_addresses, - fn _changes -> - from(a in Address, where: a.contact_id == ^non_survivor.id) - end, - set: [contact_id: survivor.id] - ) - # Remap contact_fields (then deduplicate) - |> Ecto.Multi.run(:remap_contact_fields, fn repo, _changes -> - # Move all contact fields - repo.update_all( - from(cf in ContactField, where: cf.contact_id == ^non_survivor.id), - set: [contact_id: survivor.id] - ) + MergeFields.choice_fields() + |> Enum.reduce(fields, fn field, acc -> + keep_survivor_value(acc, field, Map.fetch!(survivor, field), member_ids) + end) + |> keep_survivor_coupled_flags(survivor) + end - # Deduplicate: remove exact dupes (same type + same value) - repo.query( - """ - DELETE FROM contact_fields - WHERE id IN ( - SELECT cf.id FROM contact_fields cf - WHERE cf.contact_id = $1 - AND EXISTS ( - SELECT 1 FROM contact_fields cf2 - WHERE cf2.contact_id = $1 - AND cf2.contact_field_type_id = cf.contact_field_type_id - AND cf2.value = cf.value - AND cf2.id < cf.id - ) - ) - """, - [survivor.id] - ) + # `keep_survivor_value/4` may have just restored the survivor's date over the + # resolver's pick, in which case the flag the resolver attached describes the + # *other* date. Re-derive each flag from whichever date actually survived. + defp keep_survivor_coupled_flags(fields, survivor) do + Enum.reduce(MergeFields.coupled_fields(), fields, fn {date_field, flag_field}, acc -> + survivor_date = Map.fetch!(survivor, date_field) - {:ok, :done} - end) - # Remap contact_tags (handle duplicates) - |> Ecto.Multi.run(:remap_contact_tags, fn repo, _changes -> - # Delete tags that already exist on survivor - repo.query( - "DELETE FROM contact_tags WHERE contact_id = $1 AND tag_id IN (SELECT tag_id FROM contact_tags WHERE contact_id = $2)", - [non_survivor.id, survivor.id] - ) + if not is_nil(survivor_date) and Map.get(acc, date_field) == survivor_date do + Map.put(acc, flag_field, Map.fetch!(survivor, flag_field)) + else + acc + end + end) + end - # Move remaining tags - repo.update_all( - from(ct in "contact_tags", where: ct.contact_id == ^non_survivor.id), - set: [contact_id: survivor.id] - ) + # The survivor holds nothing here, so whatever the resolver produced stands + # (that is the gap fill). A value the resolver cleared because it points at a + # member being merged away also stands cleared — restoring it would leave the + # field referencing a contact about to be soft-deleted. + defp keep_survivor_value(fields, _field, nil, _member_ids), do: fields - {:ok, :done} - end) - # Remap reminders - |> Ecto.Multi.update_all( - :remap_reminders, - fn _changes -> - from(r in Kith.Reminders.Reminder, where: r.contact_id == ^non_survivor.id) - end, - set: [contact_id: survivor.id] - ) - # (c) Remap relationships - |> Ecto.Multi.run(:remap_relationships, fn repo, _changes -> - remap_relationships(repo, survivor, non_survivor) - end) - # (d) Cancel Oban jobs for non-survivor reminders - |> Ecto.Multi.run(:cancel_oban_jobs, fn _repo, _changes -> - Kith.Reminders.cancel_all_for_contact(non_survivor.id, account_id) - end) - # (f) Update survivor's last_talked_to - |> Ecto.Multi.run(:update_last_talked_to, fn repo, _changes -> - merge_last_talked_to(repo, survivor, non_survivor) - end) - # (e) Soft-delete non-survivor - |> Ecto.Multi.run(:soft_delete_non_survivor, fn repo, _changes -> - non_survivor - |> Ecto.Changeset.change(%{deleted_at: DateTime.utc_now(:second)}) - |> repo.update() - end) - |> Repo.transaction() - |> case do - {:ok, %{update_survivor_fields: survivor}} -> {:ok, survivor} - {:error, step, reason, _changes} -> {:error, {step, reason}} - end + defp keep_survivor_value(fields, field, value, member_ids) do + if Map.get(fields, field) == :clear and value in member_ids do + fields + else + Map.put(fields, field, value) end end - defp merge_last_talked_to(repo, survivor, non_survivor) do - more_recent = most_recent_date(survivor.last_talked_to, non_survivor.last_talked_to) + # Iterates the known field registry rather than the caller-supplied + # `choices` map, and never converts a client-controlled key to an atom — + # the wizard's "choose-field" LiveView event puts the raw client string + # into `field_choices` with no allowlist, so a crafted/unknown field name + # must be silently dropped rather than crashing the merge (and the + # LiveView process) on `String.to_existing_atom/1`. Coupled flags are not + # choice fields, so a choice naming one is dropped here too — writing + # `:clear` to a `null: false` boolean raises inside the transaction. + # + # An explicit choice pins that contact's value even when it is nil (the + # user asked for the field to be empty); a field with no choice keeps + # whatever the resolver produced, which is what gap-fills the survivor. + # Choosing a date also pins that same contact's year-unknown flag: to the + # user the two are one value. + defp apply_legacy_choices(fields, survivor, non_survivor, choices) do + Enum.reduce(MergeFields.choice_fields(), fields, fn field, acc -> + case Map.get(choices, Atom.to_string(field)) do + "non_survivor" -> pin_choice(acc, field, non_survivor) + "survivor" -> pin_choice(acc, field, survivor) + _ -> acc + end + end) + end + + defp pin_choice(fields, field, contact) do + fields = Map.put(fields, field, clearable_value(Map.fetch!(contact, field))) - if more_recent != survivor.last_talked_to do - survivor - |> Ecto.Changeset.change(%{last_talked_to: more_recent}) - |> repo.update() - else - {:ok, survivor} + case List.keyfind(MergeFields.coupled_fields(), field, 0) do + {^field, flag_field} -> Map.put(fields, flag_field, Map.fetch!(contact, flag_field)) + nil -> fields end end + # Only `nil` means "unset". A `false` is a value, and coercing it to + # `:clear` writes NULL to a `null: false` column. + defp clearable_value(nil), do: :clear + defp clearable_value(value), do: value + defp fetch_active_contact(id) do contact = Repo.get(Contact, id) @@ -1895,91 +1843,6 @@ defmodule Kith.Contacts do end end - defp update_survivor_fields(survivor, non_survivor, field_choices) do - mergeable_fields = ~w(first_name last_name nickname birthdate description - occupation company avatar)a - - changes = - Enum.reduce(mergeable_fields, %{}, fn field, acc -> - field_str = Atom.to_string(field) - - case Map.get(field_choices, field_str, "survivor") do - "non_survivor" -> - Map.put(acc, field, Map.get(non_survivor, field)) - - _ -> - acc - end - end) - - if map_size(changes) > 0 do - survivor - |> Ecto.Changeset.change(changes) - |> Repo.update() - else - {:ok, survivor} - end - end - - defp remap_relationships(repo, survivor, non_survivor) do - # Remap forward relationships (contact_id = non_survivor) - # First delete any that would create duplicates or self-references - repo.query( - """ - DELETE FROM relationships WHERE contact_id = $1 - AND ( - related_contact_id = $2 - OR (related_contact_id, relationship_type_id) IN ( - SELECT related_contact_id, relationship_type_id - FROM relationships WHERE contact_id = $2 - ) - ) - """, - [non_survivor.id, survivor.id] - ) - - repo.update_all( - from(r in Relationship, where: r.contact_id == ^non_survivor.id), - set: [contact_id: survivor.id] - ) - - # Remap reverse relationships (related_contact_id = non_survivor) - repo.query( - """ - DELETE FROM relationships WHERE related_contact_id = $1 - AND ( - contact_id = $2 - OR (contact_id, relationship_type_id) IN ( - SELECT contact_id, relationship_type_id - FROM relationships WHERE related_contact_id = $2 - ) - ) - """, - [non_survivor.id, survivor.id] - ) - - repo.update_all( - from(r in Relationship, where: r.related_contact_id == ^non_survivor.id), - set: [related_contact_id: survivor.id] - ) - - # Clean up any self-referential relationships - repo.delete_all( - from(r in Relationship, - where: r.contact_id == ^survivor.id and r.related_contact_id == ^survivor.id - ) - ) - - {:ok, :done} - end - - defp most_recent_date(nil, date), do: date - defp most_recent_date(date, nil), do: date - - defp most_recent_date(date1, date2) do - if DateTime.compare(date1, date2) == :gt, do: date1, else: date2 - end - @doc """ Returns a dry-run preview of what a merge would do. Does NOT modify any data. diff --git a/lib/kith/contacts/merge.ex b/lib/kith/contacts/merge.ex new file mode 100644 index 00000000..f69833ed --- /dev/null +++ b/lib/kith/contacts/merge.ex @@ -0,0 +1,1052 @@ +defmodule Kith.Contacts.Merge do + @moduledoc """ + The N-way contact merge engine. + + Applies a resolution it is handed; it never derives one. Everything the merge + writes was computed by `Kith.Contacts.MergeResolution` and approved by the + caller, so a concurrent edit between resolution and submission fails + validation rather than silently changing the result. + + ## Contract: `resolution.drop` must name every row backing a dropped value + + `:dedupe_owned` (which collapses `contact_fields`/`addresses` rows the + survivor and a loser hold with an equal, normalized value) runs *before* + `:apply_drop`. If the caller's drop list names only one side of such a + duplicate pair, the outcome depends on which row `:dedupe_owned` happened to + keep by row id: + + * if the kept row is the one named in `drop` — it is then deleted by + `:apply_drop`, and the value is gone, as the caller intended. + * if the *other* row (the one **not** named in `drop`) is the one kept — + `:apply_drop`'s id matches nothing, and the value the caller meant to + exclude survives on the merged contact, unconditionally. + + A caller building a drop list (e.g. a merge-review UI) must therefore + include the ids of every member's row backing a value the user chose to + drop, not just one. This is a caller contract, not a merge bug: reordering + `:apply_drop` ahead of `:dedupe_owned` does not fix it — it makes the + survivor's row disappear into the loser's not-yet-deduped duplicate, which + then gets remapped in and the dropped value comes back unconditionally. + """ + + import Ecto.Query, warn: false + + alias Ecto.Multi + alias Kith.AuditLogs + alias Kith.Contacts.{Address, Contact, ContactField, MergeFields, Tag} + alias Kith.Repo + + # Policy, array, Immich and coupled-flag fields are computed rather than + # picked from a member, so `held_by_member?/3` accepts any value for them + # without a match. Two of those groups are resolved as a unit and can + # therefore synthesise a value no member stores: + # + # * the Immich group — an id from one record paired with another's sync + # timestamp is corrupt state. With every member at `needs_review` and + # no `immich_person_id`, the three nullable columns clear while + # `immich_status` keeps the strongest status any member held, because + # the column is `null: false` with a CHECK constraint and has no unset + # state to clear to. + # * the date/flag pairs — with no member holding a date, the flag + # resolves to `false` for that same reason, even when every member + # stores `true`. + # + # Mirrors `MergeFields.policy_fields/0 ++ MergeFields.array_fields/0 ++ + # MergeFields.immich_fields/0 ++ MergeFields.coupled_flags/0` at compile + # time (guards cannot call remote functions) — keep this in sync with that + # registry. + @computed_fields MergeFields.policy_fields() ++ + MergeFields.array_fields() ++ + MergeFields.immich_fields() ++ MergeFields.coupled_flags() + + # Every schema owning a contact_id that must follow the survivor. A schema + # missing from this list is silently orphaned by a merge, so it must stay + # exhaustive. `import_records` is deliberately absent: it references a + # contact through an untyped `local_entity_id` bigint with no FK, so it gets + # its own step (`remap_import_records_step/4`) rather than the blanket + # `contact_id` move. + @owned_schemas [ + Kith.Contacts.Note, + Kith.Contacts.Address, + Kith.Contacts.ContactField, + Kith.Contacts.Document, + Kith.Contacts.Photo, + Kith.Activities.Call, + Kith.Activities.LifeEvent, + Kith.Reminders.Reminder, + Kith.Reminders.ReminderInstance, + Kith.Contacts.Debt, + Kith.Contacts.Gift, + Kith.Contacts.Pet, + Kith.Tasks.Task, + Kith.Conversations.Conversation, + Kith.Contacts.ImmichCandidate + ] + + @doc false + # Exposed for `Kith.Contacts.MergeInvariantsTest`, which asserts this + # registry still covers every table carrying a contact_id. + def __owned_schemas__, do: @owned_schemas + + # Which schema backs each key of `resolution.drop`. Tags are the exception: + # `contact_tags` is a bare join table, so its ids are tag ids. + @drop_schemas %{ + contact_fields: ContactField, + addresses: Address + } + + @doc """ + Merges `loser_ids` into `survivor_id` inside one transaction. + + `resolution` is `%{fields: %{atom => value | :clear}, drop: %{atom => [id]}, + unchecked_ids: [id]}`. `:unchecked_ids` is optional (default `[]`) and + names every contact the caller's UI presented as a duplicate candidate but + the user did not include in this merge — see + `Kith.DuplicateDetection.resolve_after_merge/4`, which this calls to + settle and repoint that cluster's candidate pairs. Omitting it (or passing + an incomplete list) does not fail the merge, but it means a pair the user + actually rejected is left `pending` instead of `dismissed`, so it comes + back as a suggestion on the next duplicate scan — the caller is + responsible for passing every id it showed the user, not just the ones + the user visibly unchecked. + + `scope.user` is used as the actor recorded on the merge's audit entry. + + Returns `{:ok, contact}` or `{:error, reason}`, where `reason` is one of + `:not_found`, `:trashed`, `:different_accounts`, `:survivor_in_losers`, + `:no_losers`, `{:unknown_value, field}`, `{:unknown_field, field}`, `{:not_clearable, field}`, + `{:unknown_drop, key}`, or `{:invalid_fields, changeset}` if the resolved + values fail changeset validation on the survivor. + """ + def run(scope, survivor_id, loser_ids, resolution) do + Repo.transaction(fn -> + with {:ok, members} <- lock_and_load(scope, survivor_id, loser_ids), + survivor = Enum.find(members, &(&1.id == survivor_id)), + {:ok, resolution} <- validate_fields(members, resolution), + :ok <- validate_drop(members, resolution) do + run_merge(scope, members, survivor, resolution) + else + {:error, reason} -> Repo.rollback(reason) + end + end) + end + + # Everything below this point runs inside the `SELECT ... FOR UPDATE` + # transaction opened by `run/4` above — the lock and every validation and + # write must stay inside that one enclosing `Repo.transaction`. + defp run_merge(scope, members, survivor, resolution) do + losers = Enum.reject(members, &(&1.id == survivor.id)) + + # Captured now, before `:remap_owned` reassigns contact_fields and + # addresses from losers to the survivor — read any later than this + # and every dropped row's owner_id would read back as the survivor + # regardless of which member actually held it. + member_ids = Enum.map(members, & &1.id) + dropped_records = describe_dropped(Repo, resolution, member_ids) + + # A dropped record owned by a loser must not be remapped onto the + # survivor at all — it stays put and rides to trash with its owner + # when the loser is soft-deleted below. Only a dropped record already + # owned by the survivor is deleted outright, in `:apply_drop`. These + # ids are what the remap steps below exclude from their moves. + drop = drop_map(resolution) + + dropped_ids = %{ + contact_fields: drop |> Map.get(:contact_fields, []) |> Enum.uniq(), + addresses: drop |> Map.get(:addresses, []) |> Enum.uniq(), + tags: drop |> Map.get(:tags, []) |> Enum.uniq() + } + + scope + |> build_multi(members, survivor, losers, resolution, dropped_records, dropped_ids) + |> Repo.transaction() + |> case do + {:ok, %{last_talked_to: survivor}} -> survivor + {:error, _step, reason, _changes} -> Repo.rollback(reason) + end + end + + # Multi step ORDER is load-bearing: every dedupe stays ahead of the move it + # guards, the drop capture happens before this is even called (in + # `run_merge/3`, ahead of any mutation), `:resolve_pairs` stays immediately + # after `:soft_delete_losers` (a `merged` pair row must have a trashed + # endpoint), and `run/4` returns the survivor struct from `:last_talked_to`, + # not the earlier `:survivor` step. + defp build_multi(scope, members, survivor, losers, resolution, dropped_records, dropped_ids) do + loser_ids = Enum.map(losers, & &1.id) + + account_id = scope.account.id + + Multi.new() + |> Multi.run(:survivor, fn _repo, _changes -> apply_fields(survivor, resolution) end) + |> Multi.run(:remap_photos, fn repo, _changes -> + # unique_index(:photos, [:contact_id, :content_hash]) means a blanket + # move would raise whenever any two members share a hash — including + # two losers colliding with each other, not just a loser vs. the + # survivor. Dedupe across the whole cluster before moving. + dedupe_and_move(repo, "photos", "content_hash", loser_ids, survivor.id, account_id) + {:ok, :done} + end) + |> Multi.run(:remap_immich_candidates, fn repo, _changes -> + # unique_index(:immich_candidates, [:contact_id, :immich_photo_id]) — + # duplicate contacts routinely have the same Immich photo suggested + # for both, so this collides in practice more than most. + dedupe_and_move( + repo, + "immich_candidates", + "immich_photo_id", + loser_ids, + survivor.id, + account_id + ) + + {:ok, :done} + end) + |> Multi.run(:remap_birthday_reminders, fn repo, %{survivor: survivor} -> + remap_birthday_reminders_step(repo, survivor, loser_ids, account_id) + end) + |> Multi.run(:remap_stay_in_touch_reminders, fn repo, _changes -> + remap_stay_in_touch_reminders_step(repo, loser_ids, survivor.id) + end) + |> Multi.run(:remap_owned, fn repo, _changes -> + remap_owned_step(repo, loser_ids, survivor.id, dropped_ids) + end) + |> Multi.run(:remap_contact_tags, fn repo, _changes -> + # contact_tags is a bare join table with no Ecto schema, so it is + # invisible to @owned_schemas but must still follow the survivor. + # unique_index on (contact_id, tag_id); two losers can share a tag the + # survivor doesn't have, so this dedupes across the whole cluster too. + # Dropped tag ids are excluded from the move: a loser's link to a + # dropped tag stays on the loser and rides to trash with it. + dedupe_and_move_tags(repo, loser_ids, survivor.id, dropped_ids.tags) + {:ok, :done} + end) + |> Multi.run(:remap_activity_contacts, fn repo, _changes -> + # unique_index(:activity_contacts, [:activity_id, :contact_id]) — + # two losers can be tagged on the same activity the survivor isn't. + dedupe_and_move( + repo, + "activity_contacts", + "activity_id", + loser_ids, + survivor.id, + account_id + ) + + {:ok, :done} + end) + |> Multi.run(:remap_inbound_first_met, fn repo, _changes -> + remap_inbound_first_met_step(repo, loser_ids, survivor.id, account_id) + end) + |> Multi.run(:remap_import_records, fn repo, _changes -> + remap_import_records_step(repo, loser_ids, survivor.id, account_id) + end) + |> Multi.run(:remap_me_contact, fn repo, _changes -> + remap_me_contact_step(repo, loser_ids, survivor.id, account_id) + end) + |> Multi.run(:dedupe_owned, fn repo, _changes -> dedupe_owned_step(repo, survivor.id) end) + |> Multi.run(:remap_relationships, fn repo, _changes -> + remap_relationships(repo, survivor.id, loser_ids, account_id) + end) + |> Multi.run(:apply_drop, fn repo, _changes -> + apply_drop_step(repo, resolution, survivor.id) + end) + |> Multi.run(:last_talked_to, fn repo, %{survivor: survivor} -> + last_talked_to_step(repo, members, survivor) + end) + |> Multi.run(:soft_delete_losers, fn repo, _changes -> + soft_delete_losers_step(repo, loser_ids) + end) + |> Multi.run(:resolve_pairs, fn _repo, _changes -> + resolve_pairs_step(scope, survivor.id, loser_ids, resolution) + end) + |> Multi.run(:audit, fn _repo, changes -> + audit_step(scope, changes, losers, resolution, dropped_records) + end) + end + + # unique_index(:reminders, [:contact_id], where: "type = 'birthday'") — + # birthday reminders are auto-created whenever a birthdate is set, so + # merging any two contacts that both have one is the single most common + # collision. Keep the survivor's own birthday reminder if it has one; + # otherwise keep the lowest-id one among the losers. Delete the rest; + # reminder_instances cascades (on_delete: :delete_all). + # + # The kept reminder is then re-dated from the *merged* birthdate. `:survivor` + # runs before this step and may have just replaced the survivor's birthdate + # with a loser's — nothing else in the app re-syncs a birthday reminder from + # `contacts.birthdate`, so without this the survivor keeps a reminder firing + # on a date it no longer has, permanently, while the reminder that matched + # the surviving birthdate has just been deleted. + defp remap_birthday_reminders_step(repo, survivor, loser_ids, account_id) do + all_ids = [survivor.id | loser_ids] + + %{rows: rows} = + repo.query!( + "SELECT id, contact_id FROM reminders WHERE type = 'birthday' AND contact_id = ANY($1)", + [all_ids] + ) + + case rows do + [] -> + {:ok, :done} + + _ -> + {:ok, :done} = delete_extra_birthday_reminders(repo, rows, survivor.id) + resync_birthday_reminder(repo, rows, survivor, account_id) + end + end + + # No merged birthdate means there is no date to re-derive from, so the kept + # reminder is left exactly as it was. Deleting it here would be a different + # change than this one: a merge that clears a birthdate is not the same event + # as a user removing one (which routes through + # `Reminders.delete_birthday_reminder/2`), and the engine's contract is that + # a merge destroys birthday reminders only to resolve the unique-index + # collision. + defp resync_birthday_reminder(_repo, _rows, %Contact{birthdate: nil}, _account_id) do + {:ok, :done} + end + + defp resync_birthday_reminder(repo, rows, survivor, account_id) do + keep_id = reminder_keep_id(rows, survivor.id) + next_date = Kith.TimeHelper.next_birthday_date(survivor.birthdate) + reminder = repo.get!(Kith.Reminders.Reminder, keep_id) + + cond do + reminder.next_reminder_date == next_date -> + {:ok, :done} + + # A deactivated reminder still tracks the merged birthdate — re-enabling + # it later must schedule from the right day — but it must not be put back + # on the queue. Deactivation deliberately empties `enqueued_oban_job_ids`, + # and enqueuing here would refill it while `active` stays false, leaving + # the row inconsistent with itself. `ReminderNotificationWorker.perform/1` + # discards an inactive reminder's job anyway, so the enqueue is pure noise. + not reminder.active -> + repo.update_all( + from(r in Kith.Reminders.Reminder, where: r.id == ^keep_id), + set: [next_reminder_date: next_date] + ) + + {:ok, :done} + + true -> + cancel_reminder_jobs(repo, [keep_id]) + + reminder = + reminder + |> Ecto.Changeset.change(%{next_reminder_date: next_date, enqueued_oban_job_ids: []}) + |> repo.update!() + + account = repo.get!(Kith.Accounts.Account, account_id) + {:ok, job_ids} = Kith.Reminders.enqueue_jobs_for_reminder(reminder, account) + + repo.update_all( + from(r in Kith.Reminders.Reminder, where: r.id == ^keep_id), + set: [enqueued_oban_job_ids: job_ids] + ) + + {:ok, :done} + end + end + + defp delete_extra_birthday_reminders(repo, rows, survivor_id) do + keep_id = reminder_keep_id(rows, survivor_id) + delete_ids = for [id, _contact_id] <- rows, id != keep_id, do: id + + if delete_ids != [] do + cancel_reminder_jobs(repo, delete_ids) + repo.query!("DELETE FROM reminders WHERE id = ANY($1)", [delete_ids]) + end + + {:ok, :done} + end + + # Design spec §2 step 7: these duplicate birthday reminders, and the + # duplicate stay-in-touch reminders collapsed in + # `remap_stay_in_touch_reminders_step/3`, are the only reminders a merge + # destroys. Every other reminder a loser owns moves to the survivor at + # `:remap_owned` and must keep its scheduled job, so there is nothing to + # cancel per-contact — cancelling here, where the doomed ids are still + # known, is the correct scope. `ReminderNotificationWorker.perform/1` also + # discards a job whose reminder is gone, so this trims the queue rather + # than being the only guard. + defp cancel_reminder_jobs(repo, reminder_ids) do + %{rows: rows} = + repo.query!("SELECT enqueued_oban_job_ids FROM reminders WHERE id = ANY($1)", [reminder_ids]) + + rows |> List.flatten() |> Kith.Reminders.cancel_jobs() + end + + defp reminder_keep_id(rows, survivor_id) do + case Enum.find(rows, fn [_id, contact_id] -> contact_id == survivor_id end) do + [id, _contact_id] -> id + nil -> rows |> Enum.map(&hd/1) |> Enum.min() + end + end + + # Stay-in-touch reminders have no unique index (only `reminders_birthday_ + # unique_idx` exists), so the blanket `:remap_owned` move happily lands two + # active rows on the survivor. Nothing raises at write time — the failure + # surfaces later and elsewhere, in `Reminders.resolve_stay_in_touch_instance/1`, + # whose `Repo.one/1` raises `Ecto.MultipleResultsError` the next time an + # interaction is logged. Collapse to one here, where the doomed ids are + # still known and their jobs can be cancelled. Same keep rule as birthdays: + # the survivor's own row wins, else the lowest id. + defp remap_stay_in_touch_reminders_step(repo, loser_ids, survivor_id) do + %{rows: rows} = + repo.query!( + """ + SELECT id, contact_id FROM reminders + WHERE contact_id = ANY($1) + AND type = 'stay_in_touch' + AND active = true + """, + [[survivor_id | loser_ids]] + ) + + case rows do + [] -> + {:ok, :done} + + [_only_one] -> + {:ok, :done} + + _ -> + keep_id = reminder_keep_id(rows, survivor_id) + delete_ids = for [id, _contact_id] <- rows, id != keep_id, do: id + + cancel_reminder_jobs(repo, delete_ids) + repo.query!("DELETE FROM reminders WHERE id = ANY($1)", [delete_ids]) + + {:ok, :done} + end + end + + defp remap_owned_step(repo, loser_ids, survivor_id, dropped_ids) do + @owned_schemas + |> List.delete(Kith.Contacts.Photo) + |> List.delete(Kith.Contacts.ImmichCandidate) + |> List.delete(ContactField) + |> List.delete(Address) + |> Enum.each(fn schema -> + repo.update_all( + from(r in schema, where: r.contact_id in ^loser_ids), + set: [contact_id: survivor_id] + ) + end) + + # ContactField and Address get their own call: dropped ids owned by + # a loser must not move, so they stay behind and are soft-deleted + # with that loser instead of riding to the survivor. + remap_excluding(repo, ContactField, loser_ids, survivor_id, dropped_ids.contact_fields) + remap_excluding(repo, Address, loser_ids, survivor_id, dropped_ids.addresses) + + {:ok, :done} + end + + # Contacts met *through* a loser now point at the survivor. The survivor's + # own row is excluded: `clear_member_self_reference/2` only coerces when the + # caller's field map names `:first_met_through_id`, so a partial resolution + # can reach here with the survivor pointing at a loser — and rewriting that + # to `survivor_id` produces the self-reference + # `Contact.validate_first_met_through_account/1` exists to reject, through an + # `update_all` that runs no changeset. + defp remap_inbound_first_met_step(repo, loser_ids, survivor_id, account_id) do + {count, _} = + repo.update_all( + from(c in Contact, + where: c.account_id == ^account_id, + where: c.id != ^survivor_id, + where: c.first_met_through_id in ^loser_ids + ), + set: [first_met_through_id: survivor_id] + ) + + {:ok, count} + end + + # `import_records.local_entity_id` is an untyped bigint with no foreign key, + # so nothing repoints it on its own and `ContactPurgeWorker` will hard-delete + # the loser it names 30 days from now. The unique index on + # (account_id, source, source_entity_type, source_entity_id) means a + # re-import would then map that source contact to a row that no longer + # exists instead of to the survivor. + defp remap_import_records_step(repo, loser_ids, survivor_id, account_id) do + {count, _} = + repo.update_all( + from(ir in Kith.Imports.ImportRecord, + where: ir.account_id == ^account_id, + where: ir.local_entity_type == "contact", + where: ir.local_entity_id in ^loser_ids + ), + set: [local_entity_id: survivor_id] + ) + + {:ok, count} + end + + # `users.me_contact_id` is a real FK with `on_delete: :nilify_all`, so a + # merge that absorbs a user's "me" card leaves the pointer naming a trashed + # contact and nothing complains. Thirty days later `ContactPurgeWorker` hard- + # deletes that row, the FK nilifies, and the user loses their own card with + # no event to trace it back to. Repoint it here, with the rest of the merge. + defp remap_me_contact_step(repo, loser_ids, survivor_id, account_id) do + {count, _} = + repo.update_all( + from(u in Kith.Accounts.User, + where: u.account_id == ^account_id, + where: u.me_contact_id in ^loser_ids + ), + set: [me_contact_id: survivor_id] + ) + + {:ok, count} + end + + # contact_fields and addresses have no unique index — collapsing + # duplicates here is a product decision (design spec §3), not a + # constraint guard. Both keys are normalized (lower + trimmed) so + # case/whitespace-only differences still collapse. Tags and photos + # are deduped by their own remap steps above already, via + # `dedupe_and_move/6`; nothing to repeat here. + defp dedupe_owned_step(repo, survivor_id) do + repo.query!( + """ + DELETE FROM contact_fields + WHERE id IN ( + SELECT cf.id FROM contact_fields cf + WHERE cf.contact_id = $1 + AND EXISTS ( + SELECT 1 FROM contact_fields other + WHERE other.contact_id = $1 + AND other.contact_field_type_id = cf.contact_field_type_id + AND lower(btrim(other.value)) = lower(btrim(cf.value)) + AND other.id < cf.id + ) + ) + """, + [survivor_id] + ) + + # A row with neither line1 nor postal_code is not a duplicate of + # another such row — addresses carry city/region/country/label too, + # so two blank-key rows can be different places. Only dedupe when + # at least one of the key columns actually has content. + repo.query!( + """ + DELETE FROM addresses + WHERE id IN ( + SELECT a.id FROM addresses a + WHERE a.contact_id = $1 + AND (lower(btrim(coalesce(a.line1, ''))) <> '' + OR lower(btrim(coalesce(a.postal_code, ''))) <> '') + AND EXISTS ( + SELECT 1 FROM addresses other + WHERE other.contact_id = $1 + AND lower(btrim(coalesce(other.line1, ''))) = + lower(btrim(coalesce(a.line1, ''))) + AND lower(btrim(coalesce(other.postal_code, ''))) = + lower(btrim(coalesce(a.postal_code, ''))) + AND other.id < a.id + ) + ) + """, + [survivor_id] + ) + + {:ok, :done} + end + + # Only rows currently owned by the survivor are deleted outright. + # A dropped id owned by a loser was excluded from the remap steps + # above, so it is still sitting on that loser here — left alone, it + # rides to trash with the loser at `:soft_delete_losers` instead of + # being destroyed with no trash behind it (design spec §2). + defp apply_drop_step(repo, resolution, survivor_id) do + Enum.each(drop_map(resolution), &apply_drop_entry(repo, &1, survivor_id)) + {:ok, :done} + end + + defp apply_drop_entry(_repo, {_key, []}, _survivor_id), do: :ok + + defp apply_drop_entry(repo, {:tags, tag_ids}, survivor_id) do + repo.delete_all( + from(ct in "contact_tags", + where: ct.contact_id == ^survivor_id and ct.tag_id in ^tag_ids + ) + ) + end + + defp apply_drop_entry(repo, {key, ids}, survivor_id) do + case Map.fetch(@drop_schemas, key) do + {:ok, schema} -> + repo.delete_all(from(r in schema, where: r.id in ^ids and r.contact_id == ^survivor_id)) + + :error -> + :ok + end + end + + defp last_talked_to_step(repo, members, survivor) do + latest = latest_last_talked_to(members) + + if latest && latest != survivor.last_talked_to do + survivor |> Ecto.Changeset.change(%{last_talked_to: latest}) |> repo.update() + else + {:ok, survivor} + end + end + + defp latest_last_talked_to(members) do + members + |> Enum.map(& &1.last_talked_to) + |> Enum.reject(&is_nil/1) + |> case do + [] -> nil + dates -> Enum.max(dates, DateTime) + end + end + + defp soft_delete_losers_step(repo, loser_ids) do + now = DateTime.utc_now(:second) + + {count, _} = + repo.update_all(from(c in Contact, where: c.id in ^loser_ids), set: [deleted_at: now]) + + {:ok, count} + end + + defp resolve_pairs_step(scope, survivor_id, loser_ids, resolution) do + Kith.DuplicateDetection.resolve_after_merge( + scope.account.id, + survivor_id, + loser_ids, + Map.get(resolution, :unchecked_ids, []) + ) + + {:ok, :done} + end + + defp audit_step(scope, changes, losers, resolution, dropped_records) do + survivor = changes.last_talked_to + + if scope.user do + AuditLogs.log_event(scope.account.id, scope.user, :contact_merged, + contact_id: survivor.id, + contact_name: survivor.display_name, + metadata: %{ + survivor_id: survivor.id, + loser_ids: Enum.map(losers, & &1.id), + fields: inspect_fields(resolution), + dropped: dropped_records + } + ) + else + {:ok, :skipped} + end + end + + defp lock_and_load(scope, survivor_id, loser_ids) do + account_id = scope.account.id + ids = [survivor_id | loser_ids] + + cond do + loser_ids == [] -> + {:error, :no_losers} + + survivor_id in loser_ids -> + {:error, :survivor_in_losers} + + true -> + # FOR UPDATE must run inside the same transaction as the writes below: + # a row lock taken outside a transaction is released the instant the + # statement returns, so it would not serialise two sessions merging + # overlapping clusters (design-spec §2 step 1, scenario D9). + # + # order_by: c.id fixes a deterministic lock acquisition order across + # sessions, so two merges over overlapping clusters always contend + # for locks in the same order instead of deadlocking. + members = + from(c in Contact, where: c.id in ^ids, order_by: c.id, lock: "FOR UPDATE") + |> Repo.all() + + cond do + length(members) != length(Enum.uniq(ids)) -> {:error, :not_found} + Enum.any?(members, &(&1.account_id != account_id)) -> {:error, :different_accounts} + Enum.any?(members, &(&1.deleted_at != nil)) -> {:error, :trashed} + true -> {:ok, members} + end + end + end + + # Returns the resolution to apply, which is not always the one handed in: + # `first_met_through_id` is coerced (see `clear_member_self_reference/2`). + defp validate_fields(members, resolution) do + member_ids = Enum.map(members, & &1.id) + fields = clear_member_self_reference(resolution.fields, member_ids) + + result = + Enum.reduce_while(fields, :ok, fn {field, value}, :ok -> + cond do + not MergeFields.known?(field) -> + {:halt, {:error, {:unknown_field, field}}} + + value == :clear and MergeFields.non_clearable?(field) -> + {:halt, {:error, {:not_clearable, field}}} + + value == :clear -> + {:cont, :ok} + + held_by_member?(members, field, value) -> + {:cont, :ok} + + true -> + {:halt, {:error, {:unknown_value, field}}} + end + end) + + case result do + :ok -> {:ok, Map.put(resolution, :fields, fields)} + {:error, reason} -> {:error, reason} + end + end + + # Design spec §3 / scenario D4: "when the resolved `first_met_through_id` is + # any merged member, it is set to `nil`". A contact cannot be met through a + # record it just absorbed, and pointing at a loser is worse than useless — + # `:remap_inbound_first_met` would rewrite it to the survivor anyway, + # producing a self-reference. Enforced here, at the engine, so every caller + # gets it: `MergeResolution.clear_self_reference/2` covers the resolver-driven + # paths, but a legacy `%{"first_met_through_id" => "survivor"}` choice and a + # cluster screen offering a loser's id as a value both route around it. + # Coercing rather than rejecting matches §3's wording and is a no-op for the + # resolver-driven callers, which already pass `:clear`. + defp clear_member_self_reference(fields, member_ids) do + case Map.fetch(fields, :first_met_through_id) do + {:ok, value} when value != :clear -> + if value in member_ids, + do: Map.put(fields, :first_met_through_id, :clear), + else: fields + + _ -> + fields + end + end + + # Array, policy and Immich fields are computed rather than picked, so they + # are not required to match a single member's stored value. + defp held_by_member?(_members, field, _value) + when field in @computed_fields, + do: true + + defp held_by_member?(members, field, value) do + Enum.any?(members, fn member -> + stored = Map.fetch!(member, field) + stored == value or (is_binary(stored) and String.trim(stored) == value) + end) + end + + # Moves every `table` row owned by a loser onto the survivor, first deleting + # whichever rows collide on `(contact_id, key_column)` — mirroring a unique + # index on that pair. Collisions are checked across the *whole* cluster, not + # just loser-vs-survivor: two losers can hold the same key value while the + # survivor holds none, and a plain survivor-vs-loser check would miss that, + # then raise when both surviving loser rows land on the same contact_id. + # `ctid` (Postgres's physical row identifier) breaks ties between two + # colliding loser rows deterministically without needing an `id` column, so + # this works for bare join tables (`contact_tags`) as well as tables with a + # primary key (`photos`, `activity_contacts`, `immich_candidates`). + defp dedupe_and_move(repo, table, key_column, loser_ids, survivor_id, account_id) do + {scope_delete, scope_update, scope_param} = account_scope(table, account_id) + + repo.query!( + """ + DELETE FROM #{table} t + WHERE t.contact_id = ANY($1)#{scope_delete} + AND t.#{key_column} IS NOT NULL + AND EXISTS ( + SELECT 1 FROM #{table} o + WHERE o.#{key_column} = t.#{key_column} + AND (o.contact_id = $2 + OR (o.contact_id = ANY($1) AND o.ctid < t.ctid)) + ) + """, + [loser_ids, survivor_id] ++ scope_param + ) + + repo.query!( + "UPDATE #{table} SET contact_id = $1 WHERE contact_id = ANY($2)#{scope_update}", + [survivor_id, loser_ids] ++ scope_param + ) + + :ok + end + + # Tables `dedupe_and_move/6` drives that carry their own account_id column. + # `activity_contacts` (and `contact_tags`, in `dedupe_and_move_tags/4`) are + # bare join tables with no account column, so they are scoped transitively + # by the account-validated contact ids `lock_and_load/3` produced. The + # scoping is defence in depth either way — those ids are already checked — + # but every other query in this codebase carries its scope. + @account_column_tables ~w(photos immich_candidates) + + defp account_scope(table, account_id) do + if table in @account_column_tables do + {"\n AND t.account_id = $3", " AND account_id = $3", [account_id]} + else + {"", "", []} + end + end + + # Relationships are directional and carry a 4-column unique index + # (account_id, contact_id, related_contact_id, relationship_type_id), so a + # collision can appear on either endpoint, and — as with the other dedupes + # above — between two losers that neither one shares with the survivor. + # + # Step 1 handles self-references: any relationship with both endpoints + # inside the cluster (survivor or loser, in either order — including a + # relationship between two different losers) becomes contact_id == + # related_contact_id once both sides land on the survivor, so it is + # dropped outright rather than deduped. What remains has exactly one + # cluster endpoint; the other points outside the cluster. + # + # Steps 2-3 dedupe + move the forward direction (contact_id is a loser), + # steps 4-5 mirror that for the reverse direction (related_contact_id is a + # loser) — each pass drops a loser's row when either another loser's row + # (ctid tie-break) or the survivor's own row already covers the same + # (other endpoint, type) pair, then moves what's left onto the survivor. + defp remap_relationships(repo, survivor_id, loser_ids, account_id) do + cluster_ids = [survivor_id | loser_ids] + + repo.query!( + """ + DELETE FROM relationships + WHERE account_id = $2 + AND contact_id = ANY($1) + AND related_contact_id = ANY($1) + """, + [cluster_ids, account_id] + ) + + repo.query!( + """ + DELETE FROM relationships r + WHERE r.account_id = $3 + AND r.contact_id = ANY($1) + AND EXISTS ( + SELECT 1 FROM relationships o + WHERE o.related_contact_id = r.related_contact_id + AND o.relationship_type_id = r.relationship_type_id + AND ( + o.contact_id = $2 + OR (o.contact_id = ANY($1) AND o.ctid < r.ctid) + ) + ) + """, + [loser_ids, survivor_id, account_id] + ) + + repo.query!( + "UPDATE relationships SET contact_id = $1 WHERE contact_id = ANY($2) AND account_id = $3", + [survivor_id, loser_ids, account_id] + ) + + repo.query!( + """ + DELETE FROM relationships r + WHERE r.account_id = $3 + AND r.related_contact_id = ANY($1) + AND EXISTS ( + SELECT 1 FROM relationships o + WHERE o.contact_id = r.contact_id + AND o.relationship_type_id = r.relationship_type_id + AND ( + o.related_contact_id = $2 + OR (o.related_contact_id = ANY($1) AND o.ctid < r.ctid) + ) + ) + """, + [loser_ids, survivor_id, account_id] + ) + + repo.query!( + """ + UPDATE relationships SET related_contact_id = $1 + WHERE related_contact_id = ANY($2) AND account_id = $3 + """, + [survivor_id, loser_ids, account_id] + ) + + {:ok, :done} + end + + # `resolution.drop` may legitimately be absent — dot access would raise + # `KeyError` in that case, so every drop-related function goes through this. + defp drop_map(resolution), do: Map.get(resolution, :drop) || %{} + + defp validate_drop(members, resolution) do + member_ids = Enum.map(members, & &1.id) + + Enum.reduce_while(drop_map(resolution), :ok, fn entry, :ok -> + validate_drop_entry(entry, member_ids) + end) + end + + defp validate_drop_entry({_key, []}, _member_ids), do: {:cont, :ok} + + defp validate_drop_entry({:tags, ids}, member_ids) do + ids = Enum.uniq(ids) + + owned = + from(ct in "contact_tags", + where: ct.tag_id in ^ids and ct.contact_id in ^member_ids, + select: ct.tag_id, + distinct: true + ) + |> Repo.all() + + drop_validation_result(owned, ids, :tags) + end + + defp validate_drop_entry({key, ids}, member_ids) do + case Map.fetch(@drop_schemas, key) do + {:ok, schema} -> + ids = Enum.uniq(ids) + + owned = + from(r in schema, + where: r.id in ^ids and r.contact_id in ^member_ids, + select: r.id + ) + |> Repo.all() + + drop_validation_result(owned, ids, key) + + :error -> + {:halt, {:error, {:unknown_drop, key}}} + end + end + + defp drop_validation_result(owned, ids, key) do + if Enum.sort(owned) == Enum.sort(ids) do + {:cont, :ok} + else + {:halt, {:error, {:unknown_drop, key}}} + end + end + + # Captured before deletion so the audit trail survives the rows it describes. + # + # `member_ids` matters only for the `:tags` clause: a tag_id is many-to-one + # with `contact_tags` rows, so without filtering by the cluster's members, + # dropping a widely-used tag would describe every contact in the account + # that happens to hold it — not just the merge's own members. The + # `:contact_fields`/`:addresses` clauses don't need it: their ids are row + # ids already pinned to a member by `validate_drop/2`. + defp describe_dropped(repo, resolution, member_ids) do + Enum.flat_map(drop_map(resolution), &describe_dropped_entry(repo, &1, member_ids)) + end + + defp describe_dropped_entry(_repo, {_key, []}, _member_ids), do: [] + + defp describe_dropped_entry(repo, {:tags, ids}, member_ids) do + ids = Enum.uniq(ids) + + from(ct in "contact_tags", + join: t in Tag, + on: t.id == ct.tag_id, + where: ct.tag_id in ^ids and ct.contact_id in ^member_ids, + select: {t.name, ct.contact_id} + ) + |> repo.all() + |> Enum.map(fn {name, owner_id} -> + %{type: "tags", value: name, owner_id: owner_id} + end) + end + + defp describe_dropped_entry(repo, {:addresses, ids}, _member_ids) do + ids = Enum.uniq(ids) + + from(a in Address, where: a.id in ^ids, select: {a.line1, a.contact_id}) + |> repo.all() + |> Enum.map(fn {line1, owner_id} -> + %{type: "addresses", value: line1, owner_id: owner_id} + end) + end + + defp describe_dropped_entry(repo, {key, ids}, _member_ids) do + case Map.fetch(@drop_schemas, key) do + {:ok, schema} -> + ids = Enum.uniq(ids) + + from(r in schema, where: r.id in ^ids, select: {r.value, r.contact_id}) + |> repo.all() + |> Enum.map(fn {value, owner_id} -> + %{type: to_string(key), value: value, owner_id: owner_id} + end) + + :error -> + [] + end + end + + # Moves every `schema` row owned by a loser onto the survivor, except rows + # whose id is in `excluded_ids` — those are dropped ids owned by a loser, + # which must stay on that loser (see the `:remap_owned` step). An empty + # `excluded_ids` list compiles to a no-op filter, not a special case. + defp remap_excluding(repo, schema, loser_ids, survivor_id, excluded_ids) do + from(r in schema, where: r.contact_id in ^loser_ids, where: r.id not in ^excluded_ids) + |> repo.update_all(set: [contact_id: survivor_id]) + + :ok + end + + # Same collision handling as `dedupe_and_move/6`, specialised to + # `contact_tags` with one addition: a loser's link to a tag_id in + # `excluded_tag_ids` is neither deduped away nor moved — it is left in + # place so it rides to trash with that loser. An empty `excluded_tag_ids` + # behaves exactly like the unfiltered move. + defp dedupe_and_move_tags(repo, loser_ids, survivor_id, excluded_tag_ids) do + repo.query!( + """ + DELETE FROM contact_tags t + WHERE t.contact_id = ANY($1) + AND t.tag_id IS NOT NULL + AND t.tag_id <> ALL($3::bigint[]) + AND EXISTS ( + SELECT 1 FROM contact_tags o + WHERE o.tag_id = t.tag_id + AND (o.contact_id = $2 + OR (o.contact_id = ANY($1) AND o.ctid < t.ctid)) + ) + """, + [loser_ids, survivor_id, excluded_tag_ids] + ) + + repo.query!( + "UPDATE contact_tags SET contact_id = $1 WHERE contact_id = ANY($2) AND tag_id <> ALL($3::bigint[])", + [survivor_id, loser_ids, excluded_tag_ids] + ) + + :ok + end + + defp inspect_fields(%{fields: fields}) do + Map.new(fields, fn {field, value} -> {to_string(field), inspect(value)} end) + end + + defp apply_fields(survivor, %{fields: fields}) do + changes = + Map.new(fields, fn + {field, :clear} -> {field, nil} + {field, value} -> {field, value} + end) + + case survivor |> Contact.update_changeset(changes) |> Repo.update() do + {:ok, contact} -> {:ok, contact} + {:error, changeset} -> {:error, {:invalid_fields, changeset}} + end + end +end diff --git a/lib/kith/contacts/merge_fields.ex b/lib/kith/contacts/merge_fields.ex new file mode 100644 index 00000000..774118ce --- /dev/null +++ b/lib/kith/contacts/merge_fields.ex @@ -0,0 +1,83 @@ +defmodule Kith.Contacts.MergeFields do + @moduledoc """ + Declares which `Kith.Contacts.Contact` fields participate in a merge and how + each is treated. + + Four groups: + + * `choice_fields` — ordinary scalars the user picks between. + * `policy_fields` — booleans and their companions, resolved by policy + rather than by choice (see `Kith.Contacts.MergeResolution`). + * `array_fields` — array columns resolved as a deduplicated union. + * `immich_fields` — integration state that moves as one unit. + * `coupled_fields` — a date and its year-unknown flag, resolved together. + + `display_name` is deliberately absent: it is computed by + `Contact.compute_display_name/1` and recomputed asynchronously by + `Kith.Workers.DisplayNameRecomputeWorker`. + """ + + alias Kith.Contacts.Contact + + @choice_fields ~w( + first_name middle_name last_name nickname + birthdate + description avatar occupation company + gender_id currency_id + first_met_at first_met_where + first_met_through_id first_met_additional_info + )a + + @policy_fields ~w(favorite is_archived deceased deceased_at)a + @array_fields ~w(aliases)a + @immich_fields ~w(immich_person_id immich_person_url immich_status immich_last_synced_at)a + + # A date and the flag saying whether its year is real. Resolved as one unit: + # taking the date from one member and the flag from another produces a + # contact that claims a placeholder year (1900) is a known birth year. Both + # flags are `null: false` with `default: false`, so they must never resolve + # to `:clear`. + @coupled_fields [ + {:birthdate, :birthdate_year_unknown}, + {:first_met_at, :first_met_year_unknown} + ] + + @doc "Scalars the user picks between." + def choice_fields, do: @choice_fields + + @doc "Fields resolved by policy rather than by user choice." + def policy_fields, do: @policy_fields + + @doc "Array columns resolved as a deduplicated union." + def array_fields, do: @array_fields + + @doc "Immich integration state, moved as a single unit." + def immich_fields, do: @immich_fields + + @doc "Date columns paired with the flag describing their year." + def coupled_fields, do: @coupled_fields + + @doc "The flag half of every coupled pair." + def coupled_flags, do: Enum.map(@coupled_fields, &elem(&1, 1)) + + @doc "Every mergeable field." + def all do + @choice_fields ++ @policy_fields ++ @array_fields ++ @immich_fields ++ coupled_flags() + end + + @doc "Whether `field` is a field the merge engine knows how to resolve." + def known?(field), do: field in all() + + @doc """ + Fields that cannot be set to `:clear`, derived from the contact update + changeset's `validate_required/2` list rather than maintained by hand. + """ + def non_clearable do + %Contact{} + |> Contact.update_changeset(%{}) + |> Map.fetch!(:required) + end + + @doc "Whether `field` may be cleared." + def non_clearable?(field), do: field in non_clearable() +end diff --git a/lib/kith/contacts/merge_resolution.ex b/lib/kith/contacts/merge_resolution.ex new file mode 100644 index 00000000..4194516f --- /dev/null +++ b/lib/kith/contacts/merge_resolution.ex @@ -0,0 +1,252 @@ +defmodule Kith.Contacts.MergeResolution do + @moduledoc """ + Turns a set of member contacts into the complete field map a merge will apply. + + Pure: reads the structs it is given and writes nothing. This is the single + implementation of merge resolution — the cluster LiveView calls it to render, + the REST API adapter calls it to build its payload, and the engine never calls + it at all. That separation is what makes a merge apply exactly what the user + approved: the engine validates a concrete map rather than re-deriving one at + transaction time. + + Resolution is recomputed from scratch whenever the member selection changes; + explicit user choices are not carried across a change. + """ + + alias Kith.Contacts.MergeFields + + defstruct fields: %{}, conflicts: %{}, attributions: %{} + + @type t :: %__MODULE__{ + fields: %{atom() => term() | :clear}, + conflicts: %{atom() => [%{value: term(), member_ids: [integer()], count: integer()}]}, + attributions: %{atom() => :all_agree | {:only, integer()} | {:some, integer()} | :none} + } + + @doc """ + Resolves `members` into a complete field map. + + `survivor_id` must be the id of one of `members`. It is needed by rules that + are relative to the surviving record rather than to the set as a whole. + """ + def resolve(members, survivor_id) when is_list(members) and members != [] do + member_ids = MapSet.new(members, & &1.id) + + MergeFields.choice_fields() + |> Enum.reduce(%__MODULE__{}, fn field, acc -> + put_resolution(acc, field, resolve_scalar(members, field)) + end) + |> resolve_coupled(members) + |> clear_self_reference(member_ids) + |> resolve_arrays(members) + |> resolve_policy_fields(members) + |> resolve_immich(members, survivor_id) + end + + # A contact cannot be met through a record it just absorbed. + defp clear_self_reference(acc, member_ids) do + case Map.get(acc.fields, :first_met_through_id) do + id when is_integer(id) -> + if MapSet.member?(member_ids, id) do + %{acc | fields: Map.put(acc.fields, :first_met_through_id, :clear)} + else + acc + end + + _ -> + acc + end + end + + defp resolve_arrays(acc, members) do + Enum.reduce(MergeFields.array_fields(), acc, fn field, acc -> + union = + members + |> Enum.flat_map(&(Map.fetch!(&1, field) || [])) + |> Enum.uniq() + + %{acc | fields: Map.put(acc.fields, field, union)} + end) + end + + # A date and its year-unknown flag move together. The date is resolved by the + # ordinary scalar rules in the reduce above (so it keeps its conflicts and + # attribution for the cluster screen); the flag is then read off the members + # holding the winning date rather than resolved on its own. Taking the flag + # from a different member than the date is how a 1900 placeholder ends up + # marked as a real birth year. + defp resolve_coupled(acc, members) do + Enum.reduce(MergeFields.coupled_fields(), acc, fn {date_field, flag_field}, acc -> + flag = coupled_flag(members, date_field, Map.fetch!(acc.fields, date_field), flag_field) + %{acc | fields: Map.put(acc.fields, flag_field, flag)} + end) + end + + # Both flags are `null: false` with `default: false`, so an absent date + # resolves the flag to `false`, never `:clear`. + defp coupled_flag(_members, _date_field, :clear, _flag_field), do: false + + defp coupled_flag(members, date_field, value, flag_field) do + # When members agree on the date but disagree on the flag, "year unknown" + # wins: asserting a placeholder year is real is the worse of the two errors. + members + |> Enum.filter(&(Map.fetch!(&1, date_field) == value)) + |> Enum.any?(&Map.fetch!(&1, flag_field)) + end + + defp resolve_policy_fields(acc, members) do + favorite = Enum.any?(members, & &1.favorite) + is_archived = not Enum.any?(members, &(not &1.is_archived)) + deceased = Enum.any?(members, & &1.deceased) + + deceased_at = + members + |> Enum.filter(& &1.deceased) + |> Enum.map(& &1.deceased_at) + |> Enum.reject(&is_nil/1) + |> case do + [] -> :clear + dates -> Enum.min(dates, Date) + end + + fields = + acc.fields + |> Map.put(:favorite, favorite) + |> Map.put(:is_archived, is_archived) + |> Map.put(:deceased, deceased) + |> Map.put(:deceased_at, deceased_at) + + %{acc | fields: fields} + end + + # `immich_status` is a `null: false` column with a check constraint + # (`unlinked | needs_review | linked`), ordered here by how much state it + # stands for. A merge must never move a contact *down* this ladder: dropping + # a `needs_review` survivor to `unlinked` hides it from + # `Contacts.list_needs_review/1` just as `:remap_immich_candidates` finishes + # moving every member's pending suggestion onto it. + @immich_status_rank %{"unlinked" => 0, "needs_review" => 1, "linked" => 2} + + # The four Immich columns move together: an id from one record paired with + # another record's sync timestamp is corrupt state. + defp resolve_immich(acc, members, survivor_id) do + linked = Enum.filter(members, &(not is_nil(&1.immich_person_id))) + survivor = Enum.find(members, &(&1.id == survivor_id)) + + source = + cond do + survivor && survivor.immich_person_id -> survivor + linked == [] -> nil + true -> Enum.max_by(linked, &sync_key/1) + end + + fields = + Enum.reduce(MergeFields.immich_fields(), acc.fields, fn field, fields -> + Map.put(fields, field, resolve_immich_field(field, source, members)) + end) + + %{acc | fields: fields} + end + + # With no linked source the three nullable columns have nothing to carry, but + # the status still does — it is the strongest status any member held. + defp resolve_immich_field(:immich_status, nil, members), do: strongest_immich_status(members) + defp resolve_immich_field(_field, nil, _members), do: :clear + defp resolve_immich_field(field, source, _members), do: Map.fetch!(source, field) || :clear + + defp strongest_immich_status(members) do + members + |> Enum.max_by(&Map.get(@immich_status_rank, &1.immich_status, 0)) + |> Map.fetch!(:immich_status) + end + + defp sync_key(%{immich_last_synced_at: nil}), do: 0 + defp sync_key(%{immich_last_synced_at: at}), do: DateTime.to_unix(at) + + defp put_resolution(acc, field, {value, attribution, candidates}) do + acc = %{ + acc + | fields: Map.put(acc.fields, field, value), + attributions: Map.put(acc.attributions, field, attribution) + } + + if candidates == [] do + acc + else + %{acc | conflicts: Map.put(acc.conflicts, field, candidates)} + end + end + + # Returns {resolved_value, attribution, conflict_candidates} + defp resolve_scalar(members, field) do + held = + members + |> Enum.map(fn member -> {member.id, normalize(Map.fetch!(member, field))} end) + |> Enum.reject(fn {_id, value} -> is_nil(value) end) + + case held |> Enum.map(&elem(&1, 1)) |> Enum.uniq() do + [] -> + {:clear, :none, []} + + [only] -> + ids = Enum.map(held, &elem(&1, 0)) + {only, attribution_for(length(ids), length(members), ids), []} + + _many -> + candidates = candidates(held) + value = default_value(members, candidates) + chosen = Enum.find(candidates, &(&1.value == value)) + {value, attribution_for(chosen.count, length(members), chosen.member_ids), candidates} + end + end + + defp normalize(value) when is_binary(value) do + case String.trim(value) do + "" -> nil + trimmed -> trimmed + end + end + + defp normalize(value), do: value + + # Attribution reflects how many of *all* members hold the resolved value — + # not how many members hold any value at all. A value held by 2 of 3 + # members is `{:some, 2}` even though all 3 members have some company set. + defp attribution_for(count, total_members, ids) do + case count do + 0 -> :none + 1 -> {:only, hd(ids)} + ^total_members -> :all_agree + n -> {:some, n} + end + end + + defp candidates(held) do + held + |> Enum.group_by(fn {_id, value} -> value end, fn {id, _value} -> id end) + |> Enum.map(fn {value, ids} -> %{value: value, member_ids: ids, count: length(ids)} end) + end + + # Most-held value wins; ties break toward the most recently updated member + # holding it, then toward the lowest member id. `updated_at` is + # second-precision, so exact ties are common in tests and in practice — the + # id tie-break makes the winner deterministic instead of falling out of + # map-iteration order. + defp default_value(members, candidates) do + updated_at = Map.new(members, &{&1.id, &1.updated_at}) + + candidates + |> Enum.max_by(fn %{member_ids: ids, count: count} -> + newest = + ids + |> Enum.map(&Map.fetch!(updated_at, &1)) + |> Enum.max(DateTime) + |> DateTime.to_unix() + + lowest_id = Enum.min(ids) + + {count, newest, -lowest_id} + end) + |> Map.fetch!(:value) + end +end diff --git a/lib/kith/conversations.ex b/lib/kith/conversations.ex index 971dc407..41204971 100644 --- a/lib/kith/conversations.ex +++ b/lib/kith/conversations.ex @@ -10,8 +10,6 @@ defmodule Kith.Conversations do alias Kith.Conversations.{Conversation, Message} alias Kith.Repo - @dialyzer {:nowarn_function, add_message: 2} - def list_conversations(account_id, contact_id) do Conversation |> scope_to_account(account_id) diff --git a/lib/kith/debts.ex b/lib/kith/debts.ex index be2cf65d..e5479703 100644 --- a/lib/kith/debts.ex +++ b/lib/kith/debts.ex @@ -5,8 +5,6 @@ defmodule Kith.Debts do alias Kith.Contacts.{Debt, DebtPayment} alias Kith.Repo - @dialyzer {:nowarn_function, add_payment: 2} - def list_debts(account_id, contact_id) do Debt |> scope_to_account(account_id) diff --git a/lib/kith/duplicate_detection.ex b/lib/kith/duplicate_detection.ex index 359417e5..6ae52641 100644 --- a/lib/kith/duplicate_detection.ex +++ b/lib/kith/duplicate_detection.ex @@ -54,12 +54,148 @@ defmodule Kith.DuplicateDetection do |> Repo.all() end - def dismiss_candidates_for_contact(account_id, contact_id) do - from(dc in DuplicateCandidate, - where: dc.account_id == ^account_id, - where: dc.status == "pending", - where: dc.contact_id == ^contact_id or dc.duplicate_contact_id == ^contact_id + @status_rank %{"pending" => 0, "dismissed" => 1, "merged" => 2} + + @doc """ + Settles candidate pairs after a merge. + + Three rules, from the design spec §2. Let `S` be the merged set (survivor + + losers) and `U` the unchecked ids: + + * both endpoints in `S` → `merged` + * one endpoint in `S`, one in `U` → `dismissed` (the user reviewed and + rejected that match) + * both endpoints in `U` → untouched + + Then every remaining pair referencing a loser is repointed onto the + survivor, because the survivor now *is* that contact. Without repointing, a + dismissal recorded against a loser evaporates when the loser is trashed and + the rejected match returns on the next scan. + + Callers must run this inside their own transaction (the engine's `Multi` + does) — this function does not open one itself, since wrapping it here + would only nest inside that outer transaction and swallow a rollback + raised by any step, silently losing it as this function's result is + discarded and always `:ok`. + + `unchecked_ids` is defensively subtracted from the merged set: if a caller + passes an id in both, treating it as "merged" wins (it's a member being + merged) — an id can't simultaneously be unchecked, so the overlap is + dropped from `unchecked_ids` before either rule runs. This guards the + trashed-endpoint invariant: without it, an overlapping id would let the + second call downgrade a `merged` row to `dismissed`. + """ + def resolve_after_merge(account_id, survivor_id, loser_ids, unchecked_ids) do + merged_ids = [survivor_id | loser_ids] + unchecked_ids = unchecked_ids -- merged_ids + now = DateTime.utc_now() |> DateTime.truncate(:second) + + set_status(account_id, merged_ids, merged_ids, "merged", now) + set_status(account_id, merged_ids, unchecked_ids, "dismissed", now) + repoint(account_id, survivor_id, loser_ids) + + :ok + end + + defp set_status(_account_id, _left, [], _status, _now), do: :ok + + defp set_status(account_id, left, right, status, now) do + from(d in DuplicateCandidate, + where: d.account_id == ^account_id, + where: + (d.contact_id in ^left and d.duplicate_contact_id in ^right) or + (d.contact_id in ^right and d.duplicate_contact_id in ^left) ) - |> Repo.update_all(set: [status: "dismissed", resolved_at: DateTime.utc_now()]) + |> Repo.update_all(set: [status: status, resolved_at: now]) + end + + # Delete-then-insert rather than update_all: the unique index on + # (account_id, contact_id, duplicate_contact_id) and the contact_id + # ordering check constraint both reject an in-place rewrite. + # + # Only rows with exactly one endpoint in the merged set (survivor + losers) + # qualify: a row with both endpoints already inside the merged set was just + # settled to "merged" by `set_status/5` above, and repointing it here would + # collapse it onto a self-pair (survivor, survivor) and silently drop it — + # destroying the very row `set_status/5` just wrote. + defp repoint(_account_id, _survivor_id, []), do: :ok + + defp repoint(account_id, survivor_id, loser_ids) do + merged_ids = [survivor_id | loser_ids] + + query = + from(d in DuplicateCandidate, + where: d.account_id == ^account_id, + where: + (d.contact_id in ^loser_ids and d.duplicate_contact_id not in ^merged_ids) or + (d.duplicate_contact_id in ^loser_ids and d.contact_id not in ^merged_ids) + ) + + rows = Repo.all(query) + Repo.delete_all(query) + + rows + |> Enum.map(&repoint_row(&1, survivor_id, loser_ids)) + |> Enum.reject(&is_nil/1) + |> Enum.group_by(fn row -> {row.contact_id, row.duplicate_contact_id} end) + |> Enum.each(fn {{low, high}, candidates} -> + strongest = Enum.max_by(candidates, &Map.fetch!(@status_rank, &1.status)) + merge_or_insert(account_id, low, high, strongest) + end) + end + + defp repoint_row(row, survivor_id, loser_ids) do + one = if row.contact_id in loser_ids, do: survivor_id, else: row.contact_id + + two = + if row.duplicate_contact_id in loser_ids, do: survivor_id, else: row.duplicate_contact_id + + if one == two do + nil + else + {low, high} = if one < two, do: {one, two}, else: {two, one} + %{row | contact_id: low, duplicate_contact_id: high} + end + end + + defp merge_or_insert(account_id, low, high, candidate) do + existing = + Repo.one( + from(d in DuplicateCandidate, + where: + d.account_id == ^account_id and d.contact_id == ^low and + d.duplicate_contact_id == ^high + ) + ) + + # The winner is a whole row, not just a status: a row's score, reasons and + # timestamps describe the match that produced *that* status, so pairing the + # existing row's status with the repointed row's evidence would misreport + # both. + winner = + if existing && + Map.fetch!(@status_rank, existing.status) >= Map.fetch!(@status_rank, candidate.status) do + existing + else + candidate + end + + attrs = %{ + contact_id: low, + duplicate_contact_id: high, + score: winner.score, + reasons: winner.reasons, + status: winner.status, + detected_at: winner.detected_at, + resolved_at: winner.resolved_at + } + + if existing do + existing |> DuplicateCandidate.changeset(attrs) |> Repo.update!() + else + %DuplicateCandidate{account_id: account_id} + |> DuplicateCandidate.changeset(attrs) + |> Repo.insert!() + end end end diff --git a/lib/kith/imports/sources/monica_api.ex b/lib/kith/imports/sources/monica_api.ex index 36056161..c268fcd0 100644 --- a/lib/kith/imports/sources/monica_api.ex +++ b/lib/kith/imports/sources/monica_api.ex @@ -917,7 +917,7 @@ defmodule Kith.Imports.Sources.MonicaApi do end end - defp try_merge_candidate(survivor, candidate, account_id, import_job, {c, e, s}) do + defp try_merge_candidate(survivor, candidate, _account_id, _import_job, {c, e, s}) do cond do MapSet.member?(s, candidate.id) -> {c, e, s} @@ -928,7 +928,9 @@ defmodule Kith.Imports.Sources.MonicaApi do true -> case Contacts.merge_contacts(survivor.id, candidate.id) do {:ok, _} -> - update_import_records_after_merge(account_id, import_job, candidate.id, survivor.id) + # No import_record fixup here: `Kith.Contacts.Merge` repoints + # them onto the survivor as part of the merge itself, at + # `:remap_import_records`. {c + 1, e, MapSet.put(s, candidate.id)} {:error, reason} -> @@ -1034,24 +1036,6 @@ defmodule Kith.Imports.Sources.MonicaApi do v |> String.trim() |> String.downcase() |> String.replace(~r/\s+/, " ") end - defp update_import_records_after_merge(account_id, import_job, old_contact_id, new_contact_id) do - from(ir in Imports.ImportRecord, - where: - ir.import_id == ^import_job.id and - ir.source_entity_type == "contact" and - ir.local_entity_id == ^old_contact_id - ) - |> Repo.update_all(set: [local_entity_id: new_contact_id]) - - # Also update any non-contact records that reference the old contact - # (This is handled by merge_contacts which remaps sub-entities, - # but import_records still need updating for Phase 2 cross-refs) - Logger.info( - "[MonicaApi] Remapped import records from contact #{old_contact_id} to #{new_contact_id} " <> - "(account #{account_id})" - ) - end - # ── Phase 2: Cross-reference resolution ────────────────────────────── defp resolve_cross_references(account_id, deferred, import_job) do diff --git a/lib/kith/reminders.ex b/lib/kith/reminders.ex index 09f11dcb..95336af8 100644 --- a/lib/kith/reminders.ex +++ b/lib/kith/reminders.ex @@ -25,17 +25,6 @@ defmodule Kith.Reminders do ReminderRule } - # Suppress Ecto.Multi opaque type warnings (Dialyzer false positives) - @dialyzer [ - {:nowarn_function, create_reminder: 3}, - {:nowarn_function, update_reminder: 2}, - {:nowarn_function, delete_reminder: 1}, - {:nowarn_function, resolve_stay_in_touch_instance: 1}, - {:nowarn_function, resolve_instance: 1}, - {:nowarn_function, dismiss_instance: 1}, - {:nowarn_function, cancel_enqueued_jobs_step: 2} - ] - # ── Reminders CRUD ────────────────────────────────────────────────────── @doc """ @@ -199,9 +188,17 @@ defmodule Kith.Reminders do end end + # `limit: 1` rather than a bare `Repo.one/1`: nothing in the schema enforces + # one active stay-in-touch reminder per contact (the only partial unique + # index on `reminders` covers birthdays), so any path that creates a second + # one — a merge, the API, a restored contact — would otherwise turn this + # into an `Ecto.MultipleResultsError` at the caller. `Kith.Contacts.Merge` + # dedupes on merge; this keeps the read total regardless. defp get_stay_in_touch_reminder(contact_id) do from(r in Reminder, - where: r.contact_id == ^contact_id and r.type == "stay_in_touch" and r.active == true + where: r.contact_id == ^contact_id and r.type == "stay_in_touch" and r.active == true, + order_by: [asc: r.id], + limit: 1 ) |> Repo.one() end diff --git a/lib/kith_web/controllers/api/contact_controller.ex b/lib/kith_web/controllers/api/contact_controller.ex index 1bd4fabc..4e4d7cb6 100644 --- a/lib/kith_web/controllers/api/contact_controller.ex +++ b/lib/kith_web/controllers/api/contact_controller.ex @@ -223,9 +223,10 @@ defmodule KithWeb.API.ContactController do with true <- Policy.can?(user, :update, :contact), :ok <- validate_merge_ids(survivor_id, non_survivor_id), survivor when not is_nil(survivor) <- Contacts.get_contact(account_id, survivor_id), - non_survivor when not is_nil(non_survivor) <- - Contacts.get_contact(account_id, non_survivor_id), - {:ok, merged} <- Contacts.merge_contacts(survivor.id, non_survivor.id) do + loser when not is_nil(loser) <- Contacts.get_contact(account_id, non_survivor_id), + fields = Contacts.legacy_resolution_fields(survivor, loser), + {:ok, merged} <- + Contacts.merge_cluster(scope, survivor.id, [loser.id], %{fields: fields, drop: %{}}) do json(conn, %{data: ContactJSON.data(merged)}) else false -> {:error, :forbidden} diff --git a/lib/kith_web/live/contact_live/merge.ex b/lib/kith_web/live/contact_live/merge.ex index f45dbbc9..e7c50520 100644 --- a/lib/kith_web/live/contact_live/merge.ex +++ b/lib/kith_web/live/contact_live/merge.ex @@ -170,10 +170,6 @@ defmodule KithWeb.ContactLive.Merge do DuplicateDetection.mark_merged(candidate) end - # Dismiss any other pending candidates involving either contact - DuplicateDetection.dismiss_candidates_for_contact(account_id, contact_a.id) - DuplicateDetection.dismiss_candidates_for_contact(account_id, contact_b.id) - # Log the merge Kith.AuditLogs.log_event(account_id, scope.user, :contact_merged, contact_id: contact_a.id, @@ -330,7 +326,7 @@ defmodule KithWeb.ContactLive.Merge do phx-value-source="survivor" class={[ "text-start text-sm px-3 py-2 rounded-[var(--radius-md)] border transition-colors cursor-pointer", - if(Map.get(@field_choices, field) == "survivor", + if(effective_choice(@field_choices, @contact_a, @contact_b, field) == "survivor", do: "border-[var(--color-accent)] bg-[var(--color-accent-subtle)] text-[var(--color-accent)]", else: "border-[var(--color-border)] hover:border-[var(--color-border-focus)]/50" @@ -345,7 +341,8 @@ defmodule KithWeb.ContactLive.Merge do phx-value-source="non_survivor" class={[ "text-start text-sm px-3 py-2 rounded-[var(--radius-md)] border transition-colors cursor-pointer", - if(Map.get(@field_choices, field) == "non_survivor", + if( + effective_choice(@field_choices, @contact_a, @contact_b, field) == "non_survivor", do: "border-[var(--color-accent)] bg-[var(--color-accent-subtle)] text-[var(--color-accent)]", else: "border-[var(--color-border)] hover:border-[var(--color-border-focus)]/50" @@ -434,10 +431,27 @@ defmodule KithWeb.ContactLive.Merge do defp step_label(2), do: "Choose fields" defp step_label(3), do: "Review & merge" - defp default_field_choices do - @mergeable_fields - |> Enum.map(&{&1, "survivor"}) - |> Map.new() + # Seeds no choices at all. A field the user never clicks must stay absent + # from this map: `Contacts.apply_legacy_choices/4` treats any present key as + # a deliberate pin and writes `:clear` when the chosen contact's value is + # nil, which would discard the resolver's gap fill. Absent means "let the + # resolver decide". `effective_choice/4` supplies the radio's display state + # so the UI still shows which value survives. + defp default_field_choices, do: %{} + + # Which column the radio should show as selected. An explicit click wins. + # Otherwise mirror what `MergeResolution` will do: the survivor's value + # stands, unless the survivor has none and the loser has one. + defp effective_choice(choices, survivor, non_survivor, field) do + case Map.get(choices, field) do + nil -> + if is_nil(field_value(survivor, field)) and not is_nil(field_value(non_survivor, field)), + do: "non_survivor", + else: "survivor" + + choice -> + choice + end end defp field_value(contact, field) do diff --git a/mix.exs b/mix.exs index e97f5e26..a90d9830 100644 --- a/mix.exs +++ b/mix.exs @@ -32,11 +32,29 @@ defmodule Kith.MixProject do ] end + # `:no_opaque` is project-wide on purpose. Every `Ecto.Multi` pipeline we + # build trips `call_without_opaque`, and the cause is entirely upstream: + # `Ecto.Multi`'s `names` field is typed `MapSet.t()` + # (deps/ecto/lib/ecto/multi.ex:150), `MapSet.t/0` is opaque, and Dialyzer + # constant-folds `Multi.new/0` to the literal `%Ecto.Multi{names: %MapSet{map: + # %{}}}` — a structurally built MapSet where an opaque one is expected. Note + # `Ecto.Multi`'s own `t/0` is NOT opaque, so this is not the "don't reach into + # Multi's internals" case; we only ever go through `Multi.new/0` and + # `Multi.run/3`. + # + # No code shape fixes it. Routing `Multi.new/0` through a `@spec _ :: + # Multi.t()` boundary only relocates the warning to `contract_with_opaque`, + # because `Multi.new/0`'s success typing cannot itself satisfy `Multi.t()`. + # Ecto is already current (3.13.5), so a version bump is not the remedy + # either. + # + # The cost of the project-wide flag over per-function escapes: a genuine + # opaque-type misuse anywhere in the project goes unreported. defp dialyzer do [ plt_file: {:no_warn, "priv/plts/dialyzer.plt"}, plt_add_apps: [:mix, :ex_unit], - flags: [:error_handling, :underspecs, :unknown], + flags: [:error_handling, :underspecs, :unknown, :no_opaque], ignore_warnings: ".dialyzer_ignore.exs" ] end diff --git a/mix.lock b/mix.lock index e228409c..a46fb54a 100644 --- a/mix.lock +++ b/mix.lock @@ -2,7 +2,6 @@ "asn1_compiler": {:hex, :asn1_compiler, "0.1.1", "64a4e52b59d1f225878445ace2c75cd2245b13a5a81182304fd9dc5acfc8994e", [:mix], [], "hexpm", "c250d24c22f1a3f305d88864400f9ac2df55c6886e1e3a030e2946efeb94695e"}, "assent": {:hex, :assent, "0.3.1", "7e7b04f4ab5d07b497b80c04f009a0f1efc409787e68b8e086512b9098f3095b", [:mix], [{:certifi, ">= 0.0.0", [hex: :certifi, repo: "hexpm", optional: true]}, {:finch, "~> 0.15", [hex: :finch, repo: "hexpm", optional: true]}, {:jose, "~> 1.8", [hex: :jose, repo: "hexpm", optional: true]}, {:req, "~> 0.4", [hex: :req, repo: "hexpm", optional: true]}, {:ssl_verify_fun, ">= 0.0.0", [hex: :ssl_verify_fun, repo: "hexpm", optional: true]}], "hexpm", "3597b31f9eb556d97e64cf60c00d3451f7353d7b465a71d33530b870ebed1ff1"}, "bandit": {:hex, :bandit, "1.10.3", "1e5d168fa79ec8de2860d1b4d878d97d4fbbe2fdbe7b0a7d9315a4359d1d4bb9", [:mix], [{:hpax, "~> 1.0", [hex: :hpax, repo: "hexpm", optional: false]}, {:plug, "~> 1.18", [hex: :plug, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}, {:thousand_island, "~> 1.0", [hex: :thousand_island, repo: "hexpm", optional: false]}, {:websock, "~> 0.5", [hex: :websock, repo: "hexpm", optional: false]}], "hexpm", "99a52d909c48db65ca598e1962797659e3c0f1d06e825a50c3d75b74a5e2db18"}, - "bcrypt_elixir": {:hex, :bcrypt_elixir, "3.3.2", "d50091e3c9492d73e17fc1e1619a9b09d6a5ef99160eb4d736926fd475a16ca3", [:make, :mix], [{:comeonin, "~> 5.3", [hex: :comeonin, repo: "hexpm", optional: false]}, {:elixir_make, "~> 0.6", [hex: :elixir_make, repo: "hexpm", optional: false]}], "hexpm", "471be5151874ae7931911057d1467d908955f93554f7a6cd1b7d804cac8cef53"}, "bunt": {:hex, :bunt, "1.0.0", "081c2c665f086849e6d57900292b3a161727ab40431219529f13c4ddcf3e7a44", [:mix], [], "hexpm", "dc5f86aa08a5f6fa6b8096f0735c4e76d54ae5c9fa2c143e5a1fc7c1cd9bb6b5"}, "cachex": {:hex, :cachex, "4.1.1", "574c5cd28473db313a0a76aac8c945fe44191659538ca6a1e8946ec300b1a19f", [:mix], [{:eternal, "~> 1.2", [hex: :eternal, repo: "hexpm", optional: false]}, {:ex_hash_ring, "~> 6.0", [hex: :ex_hash_ring, repo: "hexpm", optional: false]}, {:jumper, "~> 1.0", [hex: :jumper, repo: "hexpm", optional: false]}, {:sleeplocks, "~> 1.1", [hex: :sleeplocks, repo: "hexpm", optional: false]}, {:unsafe, "~> 1.0", [hex: :unsafe, repo: "hexpm", optional: false]}], "hexpm", "d6b7449ff98d6bb92dda58bd4fc3189cae9f99e7042054d669596f56dc503cd8"}, "castore": {:hex, :castore, "1.0.17", "4f9770d2d45fbd91dcf6bd404cf64e7e58fed04fadda0923dc32acca0badffa2", [:mix], [], "hexpm", "12d24b9d80b910dd3953e165636d68f147a31db945d2dcb9365e441f8b5351e5"}, diff --git a/test/kith/contacts/merge_fields_test.exs b/test/kith/contacts/merge_fields_test.exs new file mode 100644 index 00000000..8fe94afc --- /dev/null +++ b/test/kith/contacts/merge_fields_test.exs @@ -0,0 +1,99 @@ +defmodule Kith.Contacts.MergeFieldsTest do + use Kith.DataCase, async: true + + alias Kith.Contacts.MergeFields + + describe "classification" do + test "display_name is never mergeable" do + refute :display_name in MergeFields.all() + end + + test "the fields the old wizard exposed are all choice fields" do + for field <- [ + :first_name, + :last_name, + :nickname, + :birthdate, + :description, + :occupation, + :company, + :avatar + ] do + assert field in MergeFields.choice_fields(), "#{field} missing" + end + end + + test "the fields the old wizard hid are now covered" do + for field <- [ + :middle_name, + :gender_id, + :currency_id, + :first_met_at, + :first_met_where, + :first_met_through_id + ] do + assert field in MergeFields.choice_fields(), "#{field} missing" + end + end + + test "policy, array and immich fields are separate from choice fields" do + assert :favorite in MergeFields.policy_fields() + assert :aliases in MergeFields.array_fields() + assert :immich_person_id in MergeFields.immich_fields() + + refute :favorite in MergeFields.choice_fields() + refute :aliases in MergeFields.choice_fields() + refute :immich_person_id in MergeFields.choice_fields() + end + + test "all/0 is the union of the five groups" do + expected = + MergeFields.choice_fields() ++ + MergeFields.policy_fields() ++ + MergeFields.array_fields() ++ + MergeFields.immich_fields() ++ MergeFields.coupled_flags() + + assert Enum.sort(MergeFields.all()) == Enum.sort(expected) + end + end + + describe "coupled_fields/0" do + test "pairs each date with the flag describing it" do + assert MergeFields.coupled_fields() == [ + {:birthdate, :birthdate_year_unknown}, + {:first_met_at, :first_met_year_unknown} + ] + end + + test "flags are not choice fields — they are never picked independently" do + refute :birthdate_year_unknown in MergeFields.choice_fields() + refute :first_met_year_unknown in MergeFields.choice_fields() + end + + test "dates remain choice fields so the UI still shows their conflicts" do + assert :birthdate in MergeFields.choice_fields() + assert :first_met_at in MergeFields.choice_fields() + end + + test "all/0 still covers every mergeable column" do + assert :birthdate_year_unknown in MergeFields.all() + assert :first_met_year_unknown in MergeFields.all() + end + end + + describe "non_clearable" do + test "matches the contact update changeset's required fields" do + required = + %Kith.Contacts.Contact{} + |> Kith.Contacts.Contact.update_changeset(%{}) + |> Map.fetch!(:required) + + assert Enum.sort(MergeFields.non_clearable()) == Enum.sort(required) + end + + test "first_name cannot be cleared" do + assert MergeFields.non_clearable?(:first_name) + refute MergeFields.non_clearable?(:nickname) + end + end +end diff --git a/test/kith/contacts/merge_invariants_test.exs b/test/kith/contacts/merge_invariants_test.exs new file mode 100644 index 00000000..4f705757 --- /dev/null +++ b/test/kith/contacts/merge_invariants_test.exs @@ -0,0 +1,129 @@ +defmodule Kith.Contacts.MergeInvariantsTest do + @moduledoc """ + Structural guards on the merge engine. These do not exercise a merge — they + assert that the engine's static registries still match the schema, so adding + a table or a pointer to a contact fails here rather than silently orphaning + rows at merge time. + + ## What these tests do NOT cover + + The first test below only catches a pointer named `contact_id`; the second + only catches a pointer backed by a foreign key constraint. A pointer that is + BOTH unnamed `contact_id` AND has no FK escapes both queries entirely. + `import_records.local_entity_id` is exactly that shape today — an untyped, + deliberately FK-less bigint paired with a `local_entity_type` string (a + polymorphic pointer; see the comment on `@owned_schemas` in + `lib/kith/contacts/merge.ex`) — and it is pinned individually by the third + test below. A *new* column of this same shape (untyped, no FK, not named + `contact_id`) would stay invisible to both structural tests here; catching + that class in general was considered and rejected as too noisy for a + static registry check, so it is left to code review rather than automated + here. + """ + use Kith.DataCase, async: true + + alias Kith.Repo + + # Every table with a contact_id column must be handled by the engine: either + # in @owned_schemas (blanket move), or by a dedicated step. Findings 2 and 3 + # were both "the engine does not know about this pointer". + @handled_by_dedicated_step ~w( + contacts + contact_tags + activity_contacts + photos + immich_candidates + relationships + duplicate_candidates + import_records + audit_logs + ) + + test "every table with a contact_id is handled by the merge engine" do + %{rows: rows} = + Repo.query!(""" + SELECT table_name FROM information_schema.columns + WHERE column_name = 'contact_id' + AND table_schema = 'public' + ORDER BY table_name + """) + + tables = List.flatten(rows) + + owned_tables = + Kith.Contacts.Merge.__owned_schemas__() + |> Enum.map(& &1.__schema__(:source)) + + unhandled = tables -- (owned_tables ++ @handled_by_dedicated_step) + + assert unhandled == [], + """ + These tables have a contact_id the merge engine does not repoint: + #{inspect(unhandled)} + + Add each to @owned_schemas in lib/kith/contacts/merge.ex, or give it + a dedicated Multi step and list it in @handled_by_dedicated_step here. + """ + end + + # Finding 3's class: a pointer to a contact that is not named contact_id. + test "every foreign key referencing contacts is repointed or intentionally not" do + %{rows: rows} = + Repo.query!(""" + SELECT tc.table_name, kcu.column_name + FROM information_schema.table_constraints tc + JOIN information_schema.key_column_usage kcu + ON tc.constraint_name = kcu.constraint_name + JOIN information_schema.constraint_column_usage ccu + ON tc.constraint_name = ccu.constraint_name + WHERE tc.constraint_type = 'FOREIGN KEY' + AND ccu.table_name = 'contacts' + AND kcu.column_name != 'contact_id' + ORDER BY tc.table_name, kcu.column_name + """) + + # Each entry is {table, column} the engine knowingly handles. + known = [ + # remap_me_contact_step/4 + ["users", "me_contact_id"], + # remap_inbound_first_met_step/4 + ["contacts", "first_met_through_id"], + # remap_relationships/4 (raw SQL). The table is `relationships` — + # see lib/kith/contacts/relationship.ex's `schema "relationships"`. + ["relationships", "related_contact_id"], + # resolve_after_merge/4 in DuplicateDetection, which repoints both + # loser ids onto the survivor. `duplicate_contact_id` is the schema's + # second FK to contacts; the first is `contact_id`, excluded above. + ["duplicate_candidates", "duplicate_contact_id"] + ] + + assert rows -- known == [], + """ + These columns point at contacts and the merge engine does not + repoint them: #{inspect(rows -- known)} + """ + end + + # import_records.local_entity_id is the one known pointer neither test + # above can see — see the moduledoc's "What these tests do NOT cover". A + # behavioural test already covers this ("import records follow the + # survivor" in test/kith/contacts_merge_test.exs), so this doesn't run a + # second merge to prove the same thing; it pins the wiring structurally + # instead, so removing `remap_import_records_step/4` or disconnecting it + # from `build_multi/7`'s Multi chain fails here too. + test "the untyped import_records pointer has a dedicated, wired step" do + source = + __ENV__.file + |> Path.dirname() + |> Path.join("../../../lib/kith/contacts/merge.ex") + |> Path.expand() + |> File.read!() + + assert source =~ + ~r/defp remap_import_records_step\(repo, loser_ids, survivor_id, account_id\)/, + "remap_import_records_step/4 is missing from lib/kith/contacts/merge.ex" + + assert source =~ ~r/Multi\.run\(:remap_import_records,.*?remap_import_records_step\(/s, + "remap_import_records_step/4 exists but is no longer wired into build_multi/7's Multi chain" + end +end diff --git a/test/kith/contacts/merge_resolution_test.exs b/test/kith/contacts/merge_resolution_test.exs new file mode 100644 index 00000000..10816c14 --- /dev/null +++ b/test/kith/contacts/merge_resolution_test.exs @@ -0,0 +1,449 @@ +defmodule Kith.Contacts.MergeResolutionTest do + use Kith.DataCase, async: false + + import Ecto.Query + + alias Kith.Contacts.MergeResolution + alias Kith.ContactsFixtures + alias Kith.AccountsFixtures + + setup do + ContactsFixtures.seed_reference_data!() + user = AccountsFixtures.user_fixture() + %{account_id: user.account_id} + end + + defp contact(account_id, attrs), do: ContactsFixtures.contact_fixture(account_id, attrs) + + describe "scalar fields" do + test "no member holds a value — field is cleared, no attribution", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", occupation: nil}) + b = contact(ctx.account_id, %{first_name: "Sarah", occupation: nil}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.occupation == :clear + assert res.attributions.occupation == :none + refute Map.has_key?(res.conflicts, :occupation) + end + + test "every member agrees — value kept, attributed to all", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah"}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.first_name == "Sarah" + assert res.attributions.first_name == :all_agree + end + + test "only one member holds a value — gap is filled", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", middle_name: nil}) + b = contact(ctx.account_id, %{first_name: "Sarah", middle_name: "Jiyoung"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.middle_name == "Jiyoung" + assert res.attributions.middle_name == {:only, b.id} + refute Map.has_key?(res.conflicts, :middle_name) + end + + test "whitespace-only differences are not a conflict", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Figma"}) + b = contact(ctx.account_id, %{first_name: "Sarah", company: " Figma "}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.company == "Figma" + assert res.attributions.company == :all_agree + end + + test "disagreement is a conflict, defaulting to the most-held value", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Figma"}) + b = contact(ctx.account_id, %{first_name: "Sarah", company: "Stripe"}) + c = contact(ctx.account_id, %{first_name: "Sarah", company: "Stripe"}) + + res = MergeResolution.resolve([a, b, c], a.id) + + assert res.fields.company == "Stripe" + assert res.attributions.company == {:some, 2} + + candidates = res.conflicts.company + assert length(candidates) == 2 + stripe = Enum.find(candidates, &(&1.value == "Stripe")) + assert stripe.count == 2 + assert Enum.sort(stripe.member_ids) == Enum.sort([b.id, c.id]) + end + + test "a tied conflict breaks toward the most recently updated member", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Figma"}) + b = contact(ctx.account_id, %{first_name: "Sarah", company: "Stripe"}) + + # Make b unambiguously newer. + b = + b + |> Ecto.Changeset.change(%{updated_at: DateTime.add(a.updated_at, 3600, :second)}) + |> Repo.update!() + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.company == "Stripe" + end + + test "a conflict tied on count and updated_at breaks toward the lowest member id", ctx do + # candidates() groups by Map.group_by, whose resulting map iterates in + # Erlang term order of the *values* (lexicographic for strings) — + # independent of member-list order or member id. "Alpha" always sorts + # ahead of "Zeta", so Enum.max_by/2's first-wins tie-break would hand a + # tie to "Alpha" regardless of which member holds it. Assigning "Alpha" + # to the higher-id member and "Zeta" to the lower-id one means that + # picking the alphabetically-first value would give the WRONG (higher + # id) winner — so only the explicit `-lowest_id` term in + # default_value/2's sort key can make this assertion pass. + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Zeta"}) + b = contact(ctx.account_id, %{first_name: "Sarah", company: "Alpha"}) + + # a is created first so a.id < b.id. + assert a.id < b.id + + # Force identical updated_at so only the id tie-break can decide. + same_time = DateTime.utc_now() |> DateTime.truncate(:second) + + a = a |> Ecto.Changeset.change(%{updated_at: same_time}) |> Repo.update!() + b = b |> Ecto.Changeset.change(%{updated_at: same_time}) |> Repo.update!() + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.company == a.company + end + + test "resolving a single member is a no-op copy", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", company: "Figma"}) + + res = MergeResolution.resolve([a], a.id) + + assert res.fields.first_name == "Sarah" + assert res.fields.company == "Figma" + assert res.conflicts == %{} + end + end + + describe "array fields" do + test "aliases are unioned and deduplicated", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", aliases: ["Sarah K.", "SK"]}) + b = contact(ctx.account_id, %{first_name: "Sarah", aliases: ["SK", "김지영"]}) + + res = MergeResolution.resolve([a, b], a.id) + + assert Enum.sort(res.fields.aliases) == Enum.sort(["Sarah K.", "SK", "김지영"]) + end + + test "no aliases anywhere resolves to an empty list", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah"}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.aliases == [] + end + end + + describe "association ids" do + test "gender_id resolves like any other scalar", ctx do + gender = Repo.one!(from(g in Kith.Contacts.Gender, limit: 1)) + a = contact(ctx.account_id, %{first_name: "Sarah", gender_id: nil}) + b = contact(ctx.account_id, %{first_name: "Sarah", gender_id: gender.id}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.gender_id == gender.id + end + + test "first_met_through pointing at a merged member is cleared", ctx do + b = contact(ctx.account_id, %{first_name: "Sarah"}) + a = contact(ctx.account_id, %{first_name: "Sarah", first_met_through_id: b.id}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.first_met_through_id == :clear + end + + test "first_met_through pointing outside the cluster is kept", ctx do + outsider = contact(ctx.account_id, %{first_name: "Dana"}) + a = contact(ctx.account_id, %{first_name: "Sarah", first_met_through_id: outsider.id}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.first_met_through_id == outsider.id + end + end + + describe "policy fields" do + test "favorite is true if any member is favorited", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", favorite: false}) + b = contact(ctx.account_id, %{first_name: "Sarah", favorite: true}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.favorite == true + end + + test "is_archived is false if any member is active", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", is_archived: true}) + b = contact(ctx.account_id, %{first_name: "Sarah", is_archived: false}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.is_archived == false + end + + test "deceased_at takes the earliest non-nil date", ctx do + a = + contact(ctx.account_id, %{ + first_name: "Sarah", + deceased: true, + deceased_at: ~D[2024-05-01] + }) + + b = + contact(ctx.account_id, %{ + first_name: "Sarah", + deceased: true, + deceased_at: ~D[2023-02-11] + }) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.deceased == true + assert res.fields.deceased_at == ~D[2023-02-11] + end + + test "deceased with no dates anywhere leaves deceased_at nil", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", deceased: true, deceased_at: nil}) + b = contact(ctx.account_id, %{first_name: "Sarah", deceased: false}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.deceased == true + assert res.fields.deceased_at == :clear + end + end + + describe "coupled date/flag resolution" do + setup do + Kith.ContactsFixtures.seed_reference_data!() + user = Kith.AccountsFixtures.user_fixture() + %{account_id: user.account_id} + end + + test "the flag travels with the date that won", %{account_id: account_id} do + known = + Kith.ContactsFixtures.contact_fixture(account_id, %{ + first_name: "A", + birthdate: ~D[1985-04-12], + birthdate_year_unknown: false + }) + + placeholder = + Kith.ContactsFixtures.contact_fixture(account_id, %{ + first_name: "A", + birthdate: ~D[1900-04-12], + birthdate_year_unknown: true + }) + + fields = MergeResolution.resolve([known, placeholder], known.id).fields + + # Whichever date the tie-break picked, its own flag must come with it. + case fields.birthdate do + ~D[1985-04-12] -> assert fields.birthdate_year_unknown == false + ~D[1900-04-12] -> assert fields.birthdate_year_unknown == true + end + end + + test "gap-filling a date brings its flag along", %{account_id: account_id} do + empty = Kith.ContactsFixtures.contact_fixture(account_id, %{first_name: "A"}) + + holder = + Kith.ContactsFixtures.contact_fixture(account_id, %{ + first_name: "A", + birthdate: ~D[1900-06-15], + birthdate_year_unknown: true + }) + + fields = MergeResolution.resolve([empty, holder], empty.id).fields + + assert fields.birthdate == ~D[1900-06-15] + assert fields.birthdate_year_unknown == true + end + + test "no member holds the date, so the flag is false rather than :clear", + %{account_id: account_id} do + a = Kith.ContactsFixtures.contact_fixture(account_id, %{first_name: "A"}) + b = Kith.ContactsFixtures.contact_fixture(account_id, %{first_name: "A"}) + + fields = MergeResolution.resolve([a, b], a.id).fields + + assert fields.birthdate == :clear + assert fields.birthdate_year_unknown == false + assert fields.first_met_at == :clear + assert fields.first_met_year_unknown == false + end + + test "members agreeing on the date but disagreeing on the flag keep the year unknown", + %{account_id: account_id} do + a = + Kith.ContactsFixtures.contact_fixture(account_id, %{ + first_name: "A", + birthdate: ~D[1900-04-12], + birthdate_year_unknown: true + }) + + b = + Kith.ContactsFixtures.contact_fixture(account_id, %{ + first_name: "A", + birthdate: ~D[1900-04-12], + birthdate_year_unknown: false + }) + + fields = MergeResolution.resolve([a, b], a.id).fields + + assert fields.birthdate == ~D[1900-04-12] + assert fields.birthdate_year_unknown == true + end + + test "the date still reports conflicts for the UI", %{account_id: account_id} do + a = + Kith.ContactsFixtures.contact_fixture(account_id, %{ + first_name: "A", + birthdate: ~D[1985-04-12] + }) + + b = + Kith.ContactsFixtures.contact_fixture(account_id, %{ + first_name: "A", + birthdate: ~D[1990-01-01] + }) + + resolution = MergeResolution.resolve([a, b], a.id) + + assert Map.has_key?(resolution.conflicts, :birthdate) + assert length(resolution.conflicts.birthdate) == 2 + end + end + + describe "immich fields" do + test "the survivor's link wins when it has one", ctx do + a = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "survivor-person", + immich_status: "linked" + }) + + b = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "loser-person", + immich_status: "linked" + }) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.immich_person_id == "survivor-person" + end + + test "an unlinked survivor adopts the only linked member's whole group", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", immich_person_id: nil}) + + b = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "loser-person", + immich_person_url: "https://immich.example/people/loser-person", + immich_status: "linked" + }) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.immich_person_id == "loser-person" + assert res.fields.immich_person_url == "https://immich.example/people/loser-person" + assert res.fields.immich_status == "linked" + end + + test "with several linked members the most recently synced wins", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", immich_person_id: nil}) + + b = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "older", + immich_status: "linked", + immich_last_synced_at: ~U[2025-01-01 00:00:00Z] + }) + + c = + contact(ctx.account_id, %{ + first_name: "Sarah", + immich_person_id: "newer", + immich_status: "linked", + immich_last_synced_at: ~U[2026-01-01 00:00:00Z] + }) + + res = MergeResolution.resolve([a, b, c], a.id) + + assert res.fields.immich_person_id == "newer" + end + + test "nobody linked leaves the group cleared", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah"}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.immich_person_id == :clear + end + + # ImmichSyncWorker sets immich_status: "needs_review" *without* an + # immich_person_id, and it scans both duplicates — so this is the ordinary + # state of a duplicate pair after a sync, not an edge case. No member counts + # as linked, so the three nullable columns clear; the status must not clear + # with them. Dropping the survivor to "unlinked" here would hide it from + # `Contacts.list_needs_review/1` at the very moment + # `:remap_immich_candidates` moved both members' pending suggestions onto + # it, stranding those suggestions where no screen can reach them. + # + # `Kith.Contacts.Merge` still treats the Immich group as computed rather + # than picked (see @computed_fields there): `:clear` is not a value any + # member stores either. + test "every member at needs_review with no person id stays needs_review", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah", immich_status: "needs_review"}) + b = contact(ctx.account_id, %{first_name: "Sarah", immich_status: "needs_review"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.immich_person_id == :clear + assert res.fields.immich_person_url == :clear + assert res.fields.immich_last_synced_at == :clear + assert res.fields.immich_status == "needs_review" + end + + test "needs_review beats unlinked when no member is linked", ctx do + unlinked = contact(ctx.account_id, %{first_name: "Sarah", immich_status: "unlinked"}) + pending = contact(ctx.account_id, %{first_name: "Sarah", immich_status: "needs_review"}) + + res = MergeResolution.resolve([unlinked, pending], unlinked.id) + + assert res.fields.immich_status == "needs_review" + end + + test "all members unlinked stays unlinked", ctx do + a = contact(ctx.account_id, %{first_name: "Sarah"}) + b = contact(ctx.account_id, %{first_name: "Sarah"}) + + res = MergeResolution.resolve([a, b], a.id) + + assert res.fields.immich_status == "unlinked" + end + end +end diff --git a/test/kith/contacts_merge_test.exs b/test/kith/contacts_merge_test.exs index 4351bc9e..3094016b 100644 --- a/test/kith/contacts_merge_test.exs +++ b/test/kith/contacts_merge_test.exs @@ -1,10 +1,13 @@ defmodule Kith.Contacts.MergeTest do use Kith.DataCase + use Oban.Testing, repo: Kith.Repo alias Kith.Contacts alias Kith.ContactsFixtures alias Kith.AccountsFixtures + import Kith.RemindersFixtures + setup do ContactsFixtures.seed_reference_data!() user = AccountsFixtures.user_fixture() @@ -29,6 +32,45 @@ defmodule Kith.Contacts.MergeTest do %{user: user, account_id: account_id, contact_a: contact_a, contact_b: contact_b} end + # Enqueues a real notification job for `reminder` and records it the way + # Reminders.enqueue_jobs_for_reminder/2 would. + defp notification_job!(reminder) do + {:ok, job} = + Oban.insert( + Kith.Workers.ReminderNotificationWorker.new(%{ + reminder_id: reminder.id, + type: "on_day", + days_before: 0 + }) + ) + + Repo.update_all( + from(r in Kith.Reminders.Reminder, where: r.id == ^reminder.id), + set: [enqueued_oban_job_ids: [job.id]] + ) + + job + end + + # The 3-arity call the wizard actually makes — `field_choices` starts empty + # and gains one entry per click. apply_legacy_choices/4 resolves each field + # independently (a per-field reduce over `choices`), so it cannot reproduce + # the wizard's lost-gap-fill bug itself: that one lives in the LiveView's own + # `default_field_choices/0` / `effective_choice/4` and is + # covered by test/kith_web/live/contact_live/merge_test.exs. The tests + # below instead pin a narrower invariant: an unrelated explicit choice must + # never disturb the resolution of a field the user didn't touch. + defp wizard_merge(survivor_id, loser_id, clicked) do + Contacts.merge_contacts(survivor_id, loser_id, clicked) + end + + # A contact in a different account, used to force a changeset failure on the + # survivor that D4's coercion does not pre-empt. + defp foreign_contact do + other_account_id = AccountsFixtures.user_fixture().account_id + ContactsFixtures.contact_fixture(other_account_id, %{first_name: "Stranger"}) + end + describe "merge_contacts/3" do test "merges contacts with default field choices (keep survivor)", ctx do {:ok, _result} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) @@ -242,6 +284,238 @@ defmodule Kith.Contacts.MergeTest do end end + describe "merge_contacts/3 shim" do + test "still honours non_survivor field choices", ctx do + {:ok, survivor} = + Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id, %{ + "company" => "non_survivor" + }) + + assert survivor.company == "New Corp" + end + + test "fills a gap on the survivor from the loser", ctx do + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [middle_name: nil] + ) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [middle_name: "Jo"] + ) + + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert survivor.middle_name == "Jo" + end + + test "fills a gap on the survivor from the loser through the wizard path", ctx do + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [middle_name: nil] + ) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [middle_name: "Jo"] + ) + + # Pins wizard_merge/3's narrow invariant: the user clicked one unrelated + # field, so middle_name must still gap-fill even though `choices` isn't + # empty. + {:ok, survivor} = + wizard_merge(ctx.contact_a.id, ctx.contact_b.id, %{"company" => "non_survivor"}) + + assert survivor.middle_name == "Jo" + assert survivor.company == "New Corp" + end + + test "never overrides an existing survivor value by default", ctx do + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert survivor.company == "Old Corp" + end + + test "never overrides an existing survivor value through the wizard path", ctx do + # Pins wizard_merge/3's narrow invariant: company was never touched, so + # the survivor-value protection must still hold for it. + {:ok, survivor} = + wizard_merge(ctx.contact_a.id, ctx.contact_b.id, %{"occupation" => "survivor"}) + + assert survivor.company == "Old Corp" + end + + test "keeps the survivor's value even when the loser was updated later", ctx do + # The resolver's conflict default is most-held, then most-recently-updated. + # Make the loser unambiguously newer so only the survivor-value protection + # can keep "Old Corp". + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [updated_at: DateTime.add(DateTime.utc_now(:second), 3600, :second)] + ) + + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert survivor.company == "Old Corp" + end + + test "keeps the survivor's value even when the loser was updated later, through the wizard path", + ctx do + # Same conflict-default scenario as above, run with an unrelated field + # clicked. Pins wizard_merge/3's narrow invariant, not the wizard's + # lost-gap-fill bug — apply_legacy_choices/4 resolves each field + # independently, so that gap cannot occur here. + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [updated_at: DateTime.add(DateTime.utc_now(:second), 3600, :second)] + ) + + {:ok, survivor} = + wizard_merge(ctx.contact_a.id, ctx.contact_b.id, %{"occupation" => "survivor"}) + + assert survivor.company == "Old Corp" + end + + test "ignores an unrecognised field key instead of crashing", ctx do + {:ok, survivor} = + Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id, %{ + "definitely_not_a_field" => "non_survivor", + "company" => "non_survivor" + }) + + assert survivor.company == "New Corp" + end + + test "an explicit survivor choice still empties a field the survivor lacks", ctx do + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [middle_name: nil] + ) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [middle_name: "Jo"] + ) + + {:ok, survivor} = + Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id, %{ + "middle_name" => "survivor" + }) + + assert survivor.middle_name == nil + end + + test "clears first_met_through rather than pointing at the trashed loser", ctx do + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [first_met_through_id: ctx.contact_b.id] + ) + + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert survivor.first_met_through_id == nil + end + + test "clears first_met_through rather than pointing at the trashed loser, through the wizard path", + ctx do + # Same D4 self-reference clearing as above, run with an unrelated field + # clicked. Pins wizard_merge/3's narrow invariant, not the wizard's + # lost-gap-fill bug. + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [first_met_through_id: ctx.contact_b.id] + ) + + {:ok, survivor} = + wizard_merge(ctx.contact_a.id, ctx.contact_b.id, %{"occupation" => "survivor"}) + + assert survivor.first_met_through_id == nil + end + + test "an explicit \"survivor\" choice cannot keep a pointer at the loser", ctx do + # apply_legacy_choices/4 overrides the resolver's :clear with the + # survivor's raw value, which here is the loser's id — routing around + # MergeResolution.clear_self_reference/2 entirely. Only the engine's own + # D4 enforcement catches this. + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [first_met_through_id: ctx.contact_b.id] + ) + + {:ok, survivor} = + Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id, %{ + "first_met_through_id" => "survivor" + }) + + assert survivor.first_met_through_id == nil + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_a.id).first_met_through_id == nil + end + + test "restoring the survivor's birthdate restores its flag too", %{account_id: account_id} do + survivor = + ContactsFixtures.contact_fixture(account_id, %{ + first_name: "Alice", + birthdate: ~D[1985-04-12], + birthdate_year_unknown: false + }) + + loser = + ContactsFixtures.contact_fixture(account_id, %{ + first_name: "Alice", + birthdate: ~D[1900-04-12], + birthdate_year_unknown: true + }) + + fields = Contacts.legacy_resolution_fields(survivor, loser) + + assert fields.birthdate == ~D[1985-04-12] + assert fields.birthdate_year_unknown == false + end + + test "gap-filling the survivor's empty birthdate keeps the loser's flag", + %{account_id: account_id} do + survivor = ContactsFixtures.contact_fixture(account_id, %{first_name: "Alice"}) + + loser = + ContactsFixtures.contact_fixture(account_id, %{ + first_name: "Alice", + birthdate: ~D[1900-06-15], + birthdate_year_unknown: true + }) + + fields = Contacts.legacy_resolution_fields(survivor, loser) + + assert fields.birthdate == ~D[1900-06-15] + assert fields.birthdate_year_unknown == true + end + + test "an explicit birthdate choice carries that contact's flag", + %{account_id: account_id} do + survivor = + ContactsFixtures.contact_fixture(account_id, %{ + first_name: "Alice", + birthdate: ~D[1985-04-12], + birthdate_year_unknown: false + }) + + loser = + ContactsFixtures.contact_fixture(account_id, %{ + first_name: "Alice", + birthdate: ~D[1900-04-12], + birthdate_year_unknown: true + }) + + assert {:ok, merged} = + Contacts.merge_contacts(survivor.id, loser.id, %{"birthdate" => "non_survivor"}) + + assert merged.birthdate == ~D[1900-04-12] + assert merged.birthdate_year_unknown == true + end + + test "a choice naming a boolean flag directly is ignored, not a crash", + %{contact_a: contact_a, contact_b: contact_b} do + # `birthdate_year_unknown` is `null: false`; the wizard's "choose-field" + # event passes the raw client string with no allowlist, so this must not + # reach the merge as a NULL write. + assert {:ok, merged} = + Contacts.merge_contacts(contact_a.id, contact_b.id, %{ + "birthdate_year_unknown" => "non_survivor" + }) + + assert merged.birthdate_year_unknown == false + end + end + describe "merge_preview/2" do test "returns counts of sub-entities to be merged", ctx do ContactsFixtures.note_fixture(ctx.contact_b, ctx.user.id) @@ -270,4 +544,1730 @@ defmodule Kith.Contacts.MergeTest do assert preview.relationships_to_dedup >= 1 end end + + describe "merge_cluster/4 validation" do + setup ctx do + scope = Kith.Accounts.Scope.for_user(ctx.user) + %{scope: scope} + end + + defp resolution(fields \\ %{}), do: %{fields: fields, drop: %{}} + + test "merges three contacts into one survivor", ctx do + c = + Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Alice", + nickname: "Al" + }) + + {:ok, survivor} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id, c.id], + resolution(%{first_name: "Alice", nickname: "Al", company: "New Corp"}) + ) + + assert survivor.id == ctx.contact_a.id + assert survivor.nickname == "Al" + assert survivor.company == "New Corp" + + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id).deleted_at != nil + assert Repo.get!(Kith.Contacts.Contact, c.id).deleted_at != nil + end + + test "clears a field set to :clear", ctx do + {:ok, survivor} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution(%{occupation: :clear}) + ) + + assert survivor.occupation == nil + end + + test "rejects clearing a required field", ctx do + assert {:error, {:not_clearable, :first_name}} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution(%{first_name: :clear}) + ) + end + + test "rejects a value no member holds", ctx do + assert {:error, {:unknown_value, :company}} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution(%{company: "Never Corp"}) + ) + end + + test "rejects an empty loser list", ctx do + assert {:error, :no_losers} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [], resolution()) + end + + test "rejects the survivor appearing among the losers", ctx do + assert {:error, :survivor_in_losers} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_a.id], + resolution() + ) + end + + test "rejects a contact from another account", ctx do + other_user = Kith.AccountsFixtures.user_fixture() + + stranger = + Kith.ContactsFixtures.contact_fixture(other_user.account_id, %{first_name: "Zed"}) + + assert {:error, :different_accounts} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [stranger.id], + resolution() + ) + end + + test "rejects an already-trashed member", ctx do + {:ok, _} = + ctx.contact_b + |> Ecto.Changeset.change(%{deleted_at: DateTime.utc_now(:second)}) + |> Repo.update() + + assert {:error, :trashed} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution() + ) + end + + test "rejects a nonexistent loser id", ctx do + assert {:error, :not_found} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [-1], + resolution() + ) + end + + test "wraps a changeset failure as {:invalid_fields, changeset}", ctx do + # contact_b references a contact in a *different* account as + # first_met_through (written straight to the column, which is the only way + # such a pointer could exist). Resolving to that value passes + # held_by_member?/3 — a member holds it — but applying it to the survivor + # is rejected by Contact.update_changeset/2. A pointer at a merged member + # cannot serve as the injection here: the engine coerces that to nil + # before validation (design spec D4). + stranger = foreign_contact() + + Repo.update_all( + from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [first_met_through_id: stranger.id] + ) + + assert {:error, {:invalid_fields, %Ecto.Changeset{} = changeset}} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution(%{first_met_through_id: stranger.id}) + ) + + assert %{first_met_through_id: ["must be a contact in the same account"]} = + errors_on(changeset) + end + + test "clears a first_met_through_id pointing at a merged member (D4)", ctx do + # The survivor genuinely holds this value, so validate_fields/2 accepts it + # and apply_fields/2 writes it — and then :remap_inbound_first_met rewrites + # every pointer at a loser to the survivor, turning it into a + # self-reference. The engine coerces the value to nil instead. + Repo.update_all( + from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [first_met_through_id: ctx.contact_b.id] + ) + + assert {:ok, survivor} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution(%{first_met_through_id: ctx.contact_b.id}) + ) + + assert survivor.first_met_through_id == nil + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_a.id).first_met_through_id == nil + end + + test "merges a cluster whose every member is at immich needs_review", ctx do + # ImmichSyncWorker sets needs_review without an immich_person_id and scans + # both duplicates, so this is the ordinary post-sync state of a duplicate + # pair. The resolver clears the three nullable Immich columns — `:clear` + # is a value no member stores — so the merge must accept the group as + # computed rather than halting with {:unknown_value, :immich_person_id}. + # The status itself must survive at needs_review: the survivor has just + # absorbed both members' pending candidate rows and has to stay visible + # to `Contacts.list_needs_review/1`. + Repo.update_all( + from(c in Kith.Contacts.Contact, + where: c.id in ^[ctx.contact_a.id, ctx.contact_b.id] + ), + set: [immich_status: "needs_review", immich_person_id: nil] + ) + + a = Repo.get!(Kith.Contacts.Contact, ctx.contact_a.id) + b = Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id) + fields = Kith.Contacts.MergeResolution.resolve([a, b], a.id).fields + + assert fields.immich_person_id == :clear + assert fields.immich_status == "needs_review" + + assert {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, a.id, [b.id], %{fields: fields, drop: %{}}) + + assert survivor.immich_status == "needs_review" + assert is_nil(survivor.immich_person_id) + assert Repo.get!(Kith.Contacts.Contact, b.id).deleted_at != nil + end + + test "a rejected merge leaves the database unchanged", ctx do + before_a = Repo.get!(Kith.Contacts.Contact, ctx.contact_a.id) + before_b = Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id) + + assert {:error, {:unknown_value, :company}} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id], + resolution(%{company: "Never Corp"}) + ) + + after_a = Repo.get!(Kith.Contacts.Contact, ctx.contact_a.id) + after_b = Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id) + + assert after_a == before_a + assert after_b == before_b + assert after_a.deleted_at == nil + assert after_b.deleted_at == nil + end + + test "returns an error for a field key that is not a contact field", ctx do + assert {:error, {:unknown_field, :not_a_real_field}} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{not_a_real_field: "whatever"}, + drop: %{}, + unchecked_ids: [] + }) + end + + test "rejects an unknown field even when its value is :clear", ctx do + # :clear alone is not proof the field is known — an unrecognised field + # carrying :clear must not slip through the `value == :clear` branch + # before the known?/1 guard has a chance to reject it. + assert {:error, {:unknown_field, :not_a_real_field}} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{not_a_real_field: :clear}, + drop: %{}, + unchecked_ids: [] + }) + end + end + + describe "merge_cluster/4 remapping" do + setup ctx do + scope = Kith.Accounts.Scope.for_user(ctx.user) + %{scope: scope} + end + + test "import records follow the survivor", ctx do + import_job = Kith.ImportsFixtures.import_fixture(ctx.account_id, ctx.user.id) + + {:ok, record} = + %Kith.Imports.ImportRecord{} + |> Kith.Imports.ImportRecord.changeset(%{ + source: "monica_api", + source_entity_type: "contact", + source_entity_id: "42", + local_entity_type: "contact", + local_entity_id: ctx.contact_b.id, + account_id: ctx.account_id, + import_id: import_job.id + }) + |> Repo.insert() + + assert {:ok, _merged} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.reload!(record).local_entity_id == ctx.contact_a.id + end + + test "a partial resolution cannot leave the survivor met through itself", ctx do + # A field map that omits :first_met_through_id entirely — the shape the + # cluster screen can produce. clear_member_self_reference/2 is a no-op + # here, so the guard has to live in the remap step itself. + Repo.update_all( + from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [first_met_through_id: ctx.contact_b.id] + ) + + assert {:ok, merged} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{first_name: "Alice"}, + drop: %{} + }) + + refute merged.first_met_through_id == merged.id + assert merged.first_met_through_id == ctx.contact_b.id + end + + test "moves the six record types the old engine orphaned", ctx do + account_id = ctx.account_id + loser_id = ctx.contact_b.id + + Repo.insert!(%Kith.Contacts.Debt{ + account_id: account_id, + contact_id: loser_id, + creator_id: ctx.user.id, + title: "Loan", + amount: Decimal.new("25.00"), + direction: "owed_to_me", + status: "active" + }) + + Repo.insert!(%Kith.Contacts.Gift{ + account_id: account_id, + contact_id: loser_id, + creator_id: ctx.user.id, + name: "Book", + direction: "given", + status: "given" + }) + + Repo.insert!(%Kith.Contacts.Pet{ + account_id: account_id, + contact_id: loser_id, + name: "Mochi" + }) + + Repo.insert!(%Kith.Tasks.Task{ + account_id: account_id, + contact_id: loser_id, + creator_id: ctx.user.id, + title: "Call back" + }) + + Repo.insert!(%Kith.Conversations.Conversation{ + account_id: account_id, + contact_id: loser_id, + creator_id: ctx.user.id, + subject: "Catch up" + }) + + Repo.insert!(%Kith.Contacts.ImmichCandidate{ + account_id: account_id, + contact_id: loser_id, + immich_photo_id: "photo-1", + immich_server_url: "https://immich.example.com", + thumbnail_url: "https://immich.example.com/thumb/1", + suggested_at: DateTime.utc_now(:second) + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [loser_id], %{ + fields: %{}, + drop: %{} + }) + + for schema <- [ + Kith.Contacts.Debt, + Kith.Contacts.Gift, + Kith.Contacts.Pet, + Kith.Tasks.Task, + Kith.Conversations.Conversation, + Kith.Contacts.ImmichCandidate + ] do + assert Repo.aggregate( + from(r in schema, where: r.contact_id == ^survivor.id), + :count + ) == 1, + "#{inspect(schema)} was not remapped" + + assert Repo.aggregate( + from(r in schema, where: r.contact_id == ^loser_id), + :count + ) == 0, + "#{inspect(schema)} left behind on the loser" + end + end + + test "repoints inbound first_met_through references", ctx do + admirer = + Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Dana", + first_met_through_id: ctx.contact_b.id + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.get!(Kith.Contacts.Contact, admirer.id).first_met_through_id == survivor.id + end + + test "moves notes and addresses from every loser", ctx do + c = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Alice"}) + + Kith.ContactsFixtures.note_fixture(ctx.contact_b, ctx.user.id) + Kith.ContactsFixtures.note_fixture(c, ctx.user.id) + Kith.ContactsFixtures.address_fixture(c) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id, c.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.aggregate( + from(n in Kith.Contacts.Note, where: n.contact_id == ^survivor.id), + :count + ) == 2 + + assert Repo.aggregate( + from(a in Kith.Contacts.Address, where: a.contact_id == ^survivor.id), + :count + ) == 1 + end + + test "deduplicates colliding tags and moves unique ones (contact_tags has no schema)", ctx do + {:ok, shared_tag} = + Contacts.create_tag(ctx.account_id, %{"name" => "shared", "color" => "#FF0000"}) + + {:ok, unique_tag} = + Contacts.create_tag(ctx.account_id, %{"name" => "unique", "color" => "#00FF00"}) + + Contacts.tag_contact(ctx.contact_a, shared_tag) + Contacts.tag_contact(ctx.contact_b, shared_tag) + Contacts.tag_contact(ctx.contact_b, unique_tag) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + tag_ids = + Repo.all( + from(ct in "contact_tags", where: ct.contact_id == ^survivor.id, select: ct.tag_id) + ) + + assert Enum.sort(tag_ids) == Enum.sort([shared_tag.id, unique_tag.id]) + + assert Repo.aggregate( + from(ct in "contact_tags", where: ct.contact_id == ^ctx.contact_b.id), + :count + ) == 0 + end + + test "drops a loser photo colliding with the survivor's content_hash instead of raising", + ctx do + Repo.insert!(%Kith.Contacts.Photo{ + account_id: ctx.account_id, + contact_id: ctx.contact_a.id, + creator_id: ctx.user.id, + file_name: "a.jpg", + storage_key: "keys/a.jpg", + file_size: 100, + content_type: "image/jpeg", + content_hash: "same-hash" + }) + + Repo.insert!(%Kith.Contacts.Photo{ + account_id: ctx.account_id, + contact_id: ctx.contact_b.id, + creator_id: ctx.user.id, + file_name: "b.jpg", + storage_key: "keys/b.jpg", + file_size: 100, + content_type: "image/jpeg", + content_hash: "same-hash" + }) + + Repo.insert!(%Kith.Contacts.Photo{ + account_id: ctx.account_id, + contact_id: ctx.contact_b.id, + creator_id: ctx.user.id, + file_name: "c.jpg", + storage_key: "keys/c.jpg", + file_size: 200, + content_type: "image/jpeg", + content_hash: "different-hash" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + survivor_hashes = + Repo.all( + from(p in Kith.Contacts.Photo, + where: p.contact_id == ^survivor.id, + select: p.content_hash + ) + ) + + assert Enum.sort(survivor_hashes) == Enum.sort(["same-hash", "different-hash"]) + + assert Repo.aggregate( + from(p in Kith.Contacts.Photo, where: p.contact_id == ^ctx.contact_b.id), + :count + ) == 0 + end + + test "dedupes collisions between two losers, not just loser-vs-survivor", ctx do + now = DateTime.utc_now(:second) + + contact_c = + Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Carol"}) + + {:ok, tag} = + Contacts.create_tag(ctx.account_id, %{"name" => "duplicate", "color" => "#123456"}) + + # Survivor holds none of these — only the two losers collide with each other. + Contacts.tag_contact(ctx.contact_b, tag) + Contacts.tag_contact(contact_c, tag) + + {1, [%{id: activity_id}]} = + Repo.insert_all( + "activities", + [ + %{ + account_id: ctx.account_id, + title: "Group hang", + occurred_at: now, + inserted_at: now, + updated_at: now + } + ], + returning: [:id] + ) + + Repo.insert_all("activity_contacts", [ + %{activity_id: activity_id, contact_id: ctx.contact_b.id}, + %{activity_id: activity_id, contact_id: contact_c.id} + ]) + + Repo.insert!(%Kith.Contacts.Photo{ + account_id: ctx.account_id, + contact_id: ctx.contact_b.id, + creator_id: ctx.user.id, + file_name: "b.jpg", + storage_key: "keys/b.jpg", + file_size: 100, + content_type: "image/jpeg", + content_hash: "shared-hash" + }) + + Repo.insert!(%Kith.Contacts.Photo{ + account_id: ctx.account_id, + contact_id: contact_c.id, + creator_id: ctx.user.id, + file_name: "c.jpg", + storage_key: "keys/c.jpg", + file_size: 100, + content_type: "image/jpeg", + content_hash: "shared-hash" + }) + + Repo.insert!(%Kith.Contacts.ImmichCandidate{ + account_id: ctx.account_id, + contact_id: ctx.contact_b.id, + immich_photo_id: "shared-photo", + immich_server_url: "https://immich.example.com", + thumbnail_url: "https://immich.example.com/thumb/shared", + suggested_at: now + }) + + Repo.insert!(%Kith.Contacts.ImmichCandidate{ + account_id: ctx.account_id, + contact_id: contact_c.id, + immich_photo_id: "shared-photo", + immich_server_url: "https://immich.example.com", + thumbnail_url: "https://immich.example.com/thumb/shared", + suggested_at: now + }) + + assert {:ok, survivor} = + Contacts.merge_cluster( + ctx.scope, + ctx.contact_a.id, + [ctx.contact_b.id, contact_c.id], + %{fields: %{}, drop: %{}} + ) + + assert Repo.aggregate( + from(ct in "contact_tags", + where: ct.contact_id == ^survivor.id and ct.tag_id == ^tag.id + ), + :count + ) == 1 + + assert Repo.aggregate( + from(ac in "activity_contacts", + where: ac.activity_id == ^activity_id and ac.contact_id == ^survivor.id + ), + :count + ) == 1 + + assert Repo.aggregate( + from(p in Kith.Contacts.Photo, + where: p.contact_id == ^survivor.id and p.content_hash == "shared-hash" + ), + :count + ) == 1 + + assert Repo.aggregate( + from(ic in Kith.Contacts.ImmichCandidate, + where: ic.contact_id == ^survivor.id and ic.immich_photo_id == "shared-photo" + ), + :count + ) == 1 + end + + test "keeps the survivor's birthday reminder and drops colliding loser ones", ctx do + birthday_a = + Kith.RemindersFixtures.birthday_reminder_fixture( + ctx.account_id, + ctx.contact_a.id, + ctx.user.id + ) + + birthday_b = + Kith.RemindersFixtures.birthday_reminder_fixture( + ctx.account_id, + ctx.contact_b.id, + ctx.user.id + ) + + Kith.RemindersFixtures.reminder_instance_fixture(birthday_b) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + remaining = + Repo.all( + from(r in Kith.Reminders.Reminder, + where: r.type == "birthday" and r.contact_id == ^survivor.id + ) + ) + + assert [%{id: id}] = remaining + assert id == birthday_a.id + assert Repo.get(Kith.Reminders.Reminder, birthday_b.id) == nil + + assert Repo.aggregate( + from(ri in Kith.Reminders.ReminderInstance, + where: ri.reminder_id == ^birthday_b.id + ), + :count + ) == 0 + end + + test "cancels the Oban jobs of the birthday reminders it deletes", ctx do + birthday_a = + Kith.RemindersFixtures.birthday_reminder_fixture( + ctx.account_id, + ctx.contact_a.id, + ctx.user.id + ) + + birthday_b = + Kith.RemindersFixtures.birthday_reminder_fixture( + ctx.account_id, + ctx.contact_b.id, + ctx.user.id + ) + + kept_job = notification_job!(birthday_a) + doomed_job = notification_job!(birthday_b) + + {:ok, _survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + # birthday_b is the reminder the merge destroys, so its scheduled + # notification must go with it (design spec §2 step 7). Every other + # reminder just changes owner and keeps its job. + assert Repo.get!(Oban.Job, doomed_job.id).state == "cancelled" + assert Repo.get!(Oban.Job, kept_job.id).state == "available" + end + + test "when only losers have birthday reminders, keeps the lowest-id one", ctx do + c = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Carol"}) + + birthday_b = + Kith.RemindersFixtures.birthday_reminder_fixture( + ctx.account_id, + ctx.contact_b.id, + ctx.user.id + ) + + _birthday_c = + Kith.RemindersFixtures.birthday_reminder_fixture(ctx.account_id, c.id, ctx.user.id) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id, c.id], %{ + fields: %{}, + drop: %{} + }) + + remaining = + Repo.all( + from(r in Kith.Reminders.Reminder, + where: r.type == "birthday" and r.contact_id == ^survivor.id + ) + ) + + assert [%{id: id}] = remaining + assert id == birthday_b.id + end + + test "moves reminders, reminder_instances, documents, calls and life_events", ctx do + loser_id = ctx.contact_b.id + + reminder = + Kith.RemindersFixtures.recurring_reminder_fixture(ctx.account_id, loser_id, ctx.user.id) + + Kith.RemindersFixtures.reminder_instance_fixture(reminder) + Kith.ContactsFixtures.document_fixture(ctx.contact_b) + + Repo.insert!(%Kith.Activities.Call{ + account_id: ctx.account_id, + contact_id: loser_id, + occurred_at: DateTime.utc_now(:second) + }) + + [life_event_type_id] = + Repo.all(from(t in "life_event_types", select: t.id, limit: 1)) + + Repo.insert!(%Kith.Activities.LifeEvent{ + account_id: ctx.account_id, + contact_id: loser_id, + occurred_on: Date.utc_today(), + life_event_type_id: life_event_type_id + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [loser_id], %{ + fields: %{}, + drop: %{} + }) + + for schema <- [ + Kith.Reminders.Reminder, + Kith.Reminders.ReminderInstance, + Kith.Contacts.Document, + Kith.Activities.Call, + Kith.Activities.LifeEvent + ] do + assert Repo.aggregate( + from(r in schema, where: r.contact_id == ^survivor.id), + :count + ) == 1, + "#{inspect(schema)} was not remapped" + + assert Repo.aggregate( + from(r in schema, where: r.contact_id == ^loser_id), + :count + ) == 0, + "#{inspect(schema)} left behind on the loser" + end + end + end + + describe "merge_cluster/4 deduplication" do + setup ctx do + scope = Kith.Accounts.Scope.for_user(ctx.user) + + email_type = + Repo.one!( + from(t in Kith.Contacts.ContactFieldType, where: like(t.protocol, "mailto%"), limit: 1) + ) + + %{scope: scope, email_type: email_type} + end + + test "identical contact fields collapse to one", ctx do + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_a, ctx.email_type.id, %{ + "value" => "sarah@example.com" + }) + + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_b, ctx.email_type.id, %{ + "value" => "sarah@example.com" + }) + + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_b, ctx.email_type.id, %{ + "value" => "other@example.com" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + values = + from(f in Kith.Contacts.ContactField, + where: f.contact_id == ^survivor.id, + select: f.value + ) + |> Repo.all() + |> Enum.sort() + + assert values == ["other@example.com", "sarah@example.com"] + end + + test "contact fields differing only by case and whitespace collapse to one", ctx do + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_a, ctx.email_type.id, %{ + "value" => "Sarah@Example.com " + }) + + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_b, ctx.email_type.id, %{ + "value" => "sarah@example.com" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + count = + Repo.aggregate( + from(f in Kith.Contacts.ContactField, where: f.contact_id == ^survivor.id), + :count + ) + + assert count == 1 + end + + test "addresses differing only by case and whitespace collapse to one, distinct address kept", + ctx do + Kith.ContactsFixtures.address_fixture(ctx.contact_a, %{ + "line1" => " 123 Main St ", + "postal_code" => "62701" + }) + + Kith.ContactsFixtures.address_fixture(ctx.contact_b, %{ + "line1" => "123 MAIN ST", + "postal_code" => " 62701 " + }) + + Kith.ContactsFixtures.address_fixture(ctx.contact_b, %{ + "line1" => "456 Other Ave", + "postal_code" => "10001" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + lines = + from(a in Kith.Contacts.Address, where: a.contact_id == ^survivor.id, select: a.line1) + |> Repo.all() + |> Enum.sort() + + assert lines == [" 123 Main St ", "456 Other Ave"] + end + + test "two addresses with no line1 or postal_code are not treated as duplicates", ctx do + # Both normalize to the same empty key, but they carry different + # cities — collapsing them would silently destroy a distinct address. + Kith.ContactsFixtures.address_fixture(ctx.contact_a, %{ + "line1" => nil, + "postal_code" => nil, + "city" => "Paris" + }) + + Kith.ContactsFixtures.address_fixture(ctx.contact_b, %{ + "line1" => nil, + "postal_code" => nil, + "city" => "Tokyo" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + cities = + from(a in Kith.Contacts.Address, where: a.contact_id == ^survivor.id, select: a.city) + |> Repo.all() + |> Enum.sort() + + assert cities == ["Paris", "Tokyo"] + end + + test "a tag on both members is kept once", ctx do + {:ok, tag} = Contacts.create_tag(ctx.account_id, %{name: "Design"}) + Contacts.tag_contact(ctx.contact_a, tag) + Contacts.tag_contact(ctx.contact_b, tag) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + count = + from(ct in "contact_tags", where: ct.contact_id == ^survivor.id, select: count()) + |> Repo.one() + + assert count == 1 + end + + test "a relationship between two merged members is removed, not self-referential", ctx do + type = Repo.one!(from(t in Kith.Contacts.RelationshipType, limit: 1)) + Kith.ContactsFixtures.relationship_fixture(ctx.contact_a, ctx.contact_b, type.id) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.aggregate( + from(r in Kith.Contacts.Relationship, + where: r.contact_id == ^survivor.id and r.related_contact_id == ^survivor.id + ), + :count + ) == 0 + end + + test "a relationship between two losers is removed, not self-referential", ctx do + # Neither end is the survivor here — both contact_b and c are losers, + # so this only exercises the "both endpoints are cluster members" + # cleanup, not the loser-vs-survivor path already covered above. + c = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Carol"}) + type = Repo.one!(from(t in Kith.Contacts.RelationshipType, limit: 1)) + Kith.ContactsFixtures.relationship_fixture(ctx.contact_b, c, type.id) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id, c.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.aggregate( + from(r in Kith.Contacts.Relationship, + where: r.contact_id == ^survivor.id and r.related_contact_id == ^survivor.id + ), + :count + ) == 0 + end + + test "two losers holding the same forward relationship to a third contact collapse to one", + ctx do + # contact_b -> third and c -> third, same type, same direction. The + # survivor holds neither, so a plain survivor-vs-loser collision check + # would miss this and the blanket move would violate the unique index. + third = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Third"}) + c = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Carol"}) + type = Repo.one!(from(t in Kith.Contacts.RelationshipType, limit: 1)) + Kith.ContactsFixtures.relationship_fixture(ctx.contact_b, third, type.id) + Kith.ContactsFixtures.relationship_fixture(c, third, type.id) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id, c.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.aggregate( + from(r in Kith.Contacts.Relationship, + where: + r.contact_id == ^survivor.id and r.related_contact_id == ^third.id and + r.relationship_type_id == ^type.id + ), + :count + ) == 1 + end + + test "two losers holding the same reverse relationship from a third contact collapse to one", + ctx do + # third -> contact_b and third -> c, same type, same direction — the + # mirror of the previous case on the related_contact_id side. + third = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Third"}) + c = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Carol"}) + type = Repo.one!(from(t in Kith.Contacts.RelationshipType, limit: 1)) + Kith.ContactsFixtures.relationship_fixture(third, ctx.contact_b, type.id) + Kith.ContactsFixtures.relationship_fixture(third, c, type.id) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id, c.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.aggregate( + from(r in Kith.Contacts.Relationship, + where: + r.related_contact_id == ^survivor.id and r.contact_id == ^third.id and + r.relationship_type_id == ^type.id + ), + :count + ) == 1 + end + + test "a loser's forward relationship colliding with the survivor's own is dropped, not moved", + ctx do + # contact_a (survivor) -> third already exists; contact_b (loser) holds + # the identical (related_contact_id, type) pair. Exercises the + # `o.contact_id = $2` branch specifically, distinct from the two-loser + # collision tests above. + third = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Third"}) + type = Repo.one!(from(t in Kith.Contacts.RelationshipType, limit: 1)) + Kith.ContactsFixtures.relationship_fixture(ctx.contact_a, third, type.id) + Kith.ContactsFixtures.relationship_fixture(ctx.contact_b, third, type.id) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.aggregate( + from(r in Kith.Contacts.Relationship, + where: + r.contact_id == ^survivor.id and r.related_contact_id == ^third.id and + r.relationship_type_id == ^type.id + ), + :count + ) == 1 + end + + test "a loser's reverse relationship colliding with the survivor's own is dropped, not moved", + ctx do + # third -> contact_a (survivor) already exists; third -> contact_b + # (loser) holds the identical pair. Exercises the + # `o.related_contact_id = $2` branch specifically. + third = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Third"}) + type = Repo.one!(from(t in Kith.Contacts.RelationshipType, limit: 1)) + Kith.ContactsFixtures.relationship_fixture(third, ctx.contact_a, type.id) + Kith.ContactsFixtures.relationship_fixture(third, ctx.contact_b, type.id) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert Repo.aggregate( + from(r in Kith.Contacts.Relationship, + where: + r.related_contact_id == ^survivor.id and r.contact_id == ^third.id and + r.relationship_type_id == ^type.id + ), + :count + ) == 1 + end + end + + describe "merge_cluster/4 side effects" do + setup ctx do + scope = Kith.Accounts.Scope.for_user(ctx.user) + + email_type = + Repo.one!( + from(t in Kith.Contacts.ContactFieldType, where: like(t.protocol, "mailto%"), limit: 1) + ) + + %{scope: scope, email_type: email_type} + end + + test "drops a survivor-owned contact field outright and records it in the audit entry", ctx do + keep = + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_a, ctx.email_type.id, %{ + "value" => "keep@example.com" + }) + + drop = + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_a, ctx.email_type.id, %{ + "value" => "drop@example.com" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{contact_fields: [drop.id]} + }) + + Oban.drain_queue(queue: :default) + + values = + from(f in Kith.Contacts.ContactField, + where: f.contact_id == ^survivor.id, + select: f.value + ) + |> Repo.all() + + assert values == ["keep@example.com"] + assert Repo.get(Kith.Contacts.ContactField, keep.id) + refute Repo.get(Kith.Contacts.ContactField, drop.id) + + log = + Repo.one!( + from(l in Kith.AuditLogs.AuditLog, + where: l.event == "contact_merged", + order_by: [desc: l.id], + limit: 1 + ) + ) + + assert [dropped] = log.metadata["dropped"] + assert dropped["type"] == "contact_fields" + assert dropped["value"] == "drop@example.com" + assert dropped["owner_id"] == ctx.contact_a.id + end + + test "a loser-owned dropped contact field is not deleted; it rides to trash with its loser", + ctx do + drop = + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_b, ctx.email_type.id, %{ + "value" => "drop@example.com" + }) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{contact_fields: [drop.id]} + }) + + Oban.drain_queue(queue: :default) + + # Still there — untouched, still owned by the (now-trashed) loser, not + # hard-deleted and not remapped onto the survivor. + reloaded = Repo.get!(Kith.Contacts.ContactField, drop.id) + assert reloaded.contact_id == ctx.contact_b.id + refute reloaded.contact_id == survivor.id + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id).deleted_at != nil + + log = + Repo.one!( + from(l in Kith.AuditLogs.AuditLog, + where: l.event == "contact_merged", + order_by: [desc: l.id], + limit: 1 + ) + ) + + assert [dropped] = log.metadata["dropped"] + assert dropped["type"] == "contact_fields" + assert dropped["value"] == "drop@example.com" + assert dropped["owner_id"] == ctx.contact_b.id + end + + test "drops a survivor-owned address outright and records it in the audit entry", ctx do + address = Kith.ContactsFixtures.address_fixture(ctx.contact_a, %{"line1" => "1 Drop St"}) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{addresses: [address.id]} + }) + + Oban.drain_queue(queue: :default) + + refute Repo.get(Kith.Contacts.Address, address.id) + + assert Repo.aggregate( + from(a in Kith.Contacts.Address, where: a.contact_id == ^survivor.id), + :count + ) == 0 + + log = + Repo.one!( + from(l in Kith.AuditLogs.AuditLog, + where: l.event == "contact_merged", + order_by: [desc: l.id], + limit: 1 + ) + ) + + assert [dropped] = log.metadata["dropped"] + assert dropped["type"] == "addresses" + assert dropped["value"] == "1 Drop St" + assert dropped["owner_id"] == ctx.contact_a.id + end + + test "drops a survivor-owned tag outright and records it in the audit entry", ctx do + {:ok, tag} = + Contacts.create_tag(ctx.account_id, %{"name" => "Drop Me", "color" => "#ABCDEF"}) + + Contacts.tag_contact(ctx.contact_a, tag) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{tags: [tag.id]} + }) + + Oban.drain_queue(queue: :default) + + assert Repo.aggregate( + from(ct in "contact_tags", + where: ct.contact_id == ^survivor.id and ct.tag_id == ^tag.id + ), + :count + ) == 0 + + log = + Repo.one!( + from(l in Kith.AuditLogs.AuditLog, + where: l.event == "contact_merged", + order_by: [desc: l.id], + limit: 1 + ) + ) + + assert [dropped] = log.metadata["dropped"] + assert dropped["type"] == "tags" + assert dropped["value"] == "Drop Me" + assert dropped["owner_id"] == ctx.contact_a.id + end + + test "a loser-owned dropped tag is not moved to the survivor; it rides to trash with its loser", + ctx do + {:ok, tag} = + Contacts.create_tag(ctx.account_id, %{"name" => "Loser Tag", "color" => "#123456"}) + + Contacts.tag_contact(ctx.contact_b, tag) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{tags: [tag.id]} + }) + + Oban.drain_queue(queue: :default) + + assert Repo.aggregate( + from(ct in "contact_tags", + where: ct.contact_id == ^survivor.id and ct.tag_id == ^tag.id + ), + :count + ) == 0 + + assert Repo.aggregate( + from(ct in "contact_tags", + where: ct.contact_id == ^ctx.contact_b.id and ct.tag_id == ^tag.id + ), + :count + ) == 1 + + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id).deleted_at != nil + + log = + Repo.one!( + from(l in Kith.AuditLogs.AuditLog, + where: l.event == "contact_merged", + order_by: [desc: l.id], + limit: 1 + ) + ) + + assert [dropped] = log.metadata["dropped"] + assert dropped["type"] == "tags" + assert dropped["value"] == "Loser Tag" + assert dropped["owner_id"] == ctx.contact_b.id + end + + test "a dropped tag's audit entry names only the cluster's own owners, not every contact holding it", + ctx do + outsider = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Outsider"}) + + {:ok, tag} = + Contacts.create_tag(ctx.account_id, %{"name" => "Widely Used", "color" => "#654321"}) + + Contacts.tag_contact(ctx.contact_a, tag) + Contacts.tag_contact(ctx.contact_b, tag) + Contacts.tag_contact(outsider, tag) + + {:ok, _survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{tags: [tag.id]} + }) + + Oban.drain_queue(queue: :default) + + log = + Repo.one!( + from(l in Kith.AuditLogs.AuditLog, + where: l.event == "contact_merged", + order_by: [desc: l.id], + limit: 1 + ) + ) + + owner_ids = log.metadata["dropped"] |> Enum.map(& &1["owner_id"]) |> Enum.sort() + + assert owner_ids == Enum.sort([ctx.contact_a.id, ctx.contact_b.id]) + refute outsider.id in owner_ids + + # The outsider's own tag link is untouched — the merge never touched + # any contact outside its own cluster. + assert Repo.aggregate( + from(ct in "contact_tags", + where: ct.contact_id == ^outsider.id and ct.tag_id == ^tag.id + ), + :count + ) == 1 + end + + test "rejects a dropped id belonging to no member", ctx do + stranger = + Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Zed"}) + + field = + Kith.ContactsFixtures.contact_field_fixture(stranger, ctx.email_type.id, %{ + "value" => "zed@example.com" + }) + + assert {:error, {:unknown_drop, :contact_fields}} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{contact_fields: [field.id]} + }) + end + + test "rejects a dropped tag id belonging to no member", ctx do + {:ok, tag} = + Contacts.create_tag(ctx.account_id, %{"name" => "Orphan", "color" => "#000000"}) + + assert {:error, {:unknown_drop, :tags}} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{tags: [tag.id]} + }) + end + + test "last_talked_to takes the maximum across members", ctx do + older = ~U[2025-01-01 00:00:00Z] + newer = ~U[2026-06-01 00:00:00Z] + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_a.id), + set: [last_talked_to: older] + ) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [last_talked_to: newer] + ) + + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{}, + drop: %{} + }) + + assert DateTime.compare(survivor.last_talked_to, newer) == :eq + end + + test "recomputes the survivor's display_name synchronously, with no job enqueued", ctx do + {:ok, survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{last_name: "S."}, + drop: %{} + }) + + assert survivor.display_name == "Alice S." + + # DisplayNameRecomputeWorker is account-wide (recomputes every contact + # from `%{"account_id", "display_name_format"}`); enqueuing it with a + # `contact_id` arg can never match a perform/1 clause, so the merge + # must not enqueue it at all — `Contact.update_changeset/2` already + # recomputes display_name synchronously (ruling R15). + refute_enqueued(worker: Kith.Workers.DisplayNameRecomputeWorker) + end + + # This one fails at the FIRST Multi step, so it pins the error contract and + # the "nothing at all happened" outcome. It cannot prove the later steps roll + # back — nothing had run yet. See the late-failure test below for that. + test "a merge that fails inside the transaction rolls back every mutation", ctx do + note = Kith.ContactsFixtures.note_fixture(ctx.contact_b, ctx.user.id) + + drop = + Kith.ContactsFixtures.contact_field_fixture(ctx.contact_a, ctx.email_type.id, %{ + "value" => "drop@example.com" + }) + + # contact_b references a contact in another account, so resolving to that + # value passes validate_fields/2 (a member holds it, and it names no member + # so D4's coercion leaves it alone), but applying it to the survivor fails + # Contact.update_changeset/2 — inside the Multi, after `with` has already + # accepted the resolution. + stranger = foreign_contact() + + Repo.update_all( + from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [first_met_through_id: stranger.id] + ) + + assert {:error, {:invalid_fields, %Ecto.Changeset{}}} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{first_met_through_id: stranger.id}, + drop: %{contact_fields: [drop.id]} + }) + + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id).deleted_at == nil + assert Repo.get!(Kith.Contacts.Note, note.id).contact_id == ctx.contact_b.id + assert Repo.get!(Kith.Contacts.ContactField, drop.id) + + assert Repo.aggregate( + from(l in Kith.AuditLogs.AuditLog, where: l.event == "contact_merged"), + :count + ) == 0 + end + + test "a failure late in the Multi rolls back the remaps that already ran", ctx do + note = Kith.ContactsFixtures.note_fixture(ctx.contact_b, ctx.user.id) + + outsider = + Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Outsider"}) + + Repo.update_all( + from(c in Kith.Contacts.Contact, where: c.id == ^outsider.id), + set: [first_met_through_id: ctx.contact_b.id] + ) + + # No real data can make :soft_delete_losers fail, so the failure is + # injected as a constraint that makes the soft delete itself illegal. By + # then :survivor and every remap step above it have written — the point of + # the test is that all of them come back (design spec D7). + Repo.query!( + "ALTER TABLE contacts ADD CONSTRAINT contacts_never_deleted CHECK (deleted_at IS NULL)" + ) + + assert_raise Postgrex.Error, fn -> + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{last_name: "S."}, + drop: %{} + }) + end + + Repo.query!("ALTER TABLE contacts DROP CONSTRAINT contacts_never_deleted") + + # :survivor (the first step) + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_a.id).last_name == "Smith" + # :remap_owned + assert Repo.get!(Kith.Contacts.Note, note.id).contact_id == ctx.contact_b.id + # :remap_inbound_first_met + assert Repo.get!(Kith.Contacts.Contact, outsider.id).first_met_through_id == + ctx.contact_b.id + + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id).deleted_at == nil + + assert Repo.aggregate( + from(l in Kith.AuditLogs.AuditLog, where: l.event == "contact_merged"), + :count + ) == 0 + end + + test "a rejected merge changes nothing at all", ctx do + Kith.ContactsFixtures.note_fixture(ctx.contact_b, ctx.user.id) + + assert {:error, {:unknown_value, :company}} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id], %{ + fields: %{company: "Never Corp"}, + drop: %{} + }) + + assert Repo.get!(Kith.Contacts.Contact, ctx.contact_b.id).deleted_at == nil + + assert Repo.aggregate( + from(n in Kith.Contacts.Note, where: n.contact_id == ^ctx.contact_b.id), + :count + ) == 1 + + assert Repo.aggregate( + from(l in Kith.AuditLogs.AuditLog, where: l.event == "contact_merged"), + :count + ) == 0 + end + + test "writes one audit entry naming every loser", ctx do + c = Kith.ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Alice"}) + + {:ok, _survivor} = + Contacts.merge_cluster(ctx.scope, ctx.contact_a.id, [ctx.contact_b.id, c.id], %{ + fields: %{}, + drop: %{} + }) + + Oban.drain_queue(queue: :default) + + logs = + from(l in Kith.AuditLogs.AuditLog, where: l.event == "contact_merged") |> Repo.all() + + assert length(logs) == 1 + assert Enum.sort(hd(logs).metadata["loser_ids"]) == Enum.sort([ctx.contact_b.id, c.id]) + end + + test "the surviving birthday reminder matches the merged birthdate", ctx do + survivor = + ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Alice", + birthdate: ~D[1985-01-05] + }) + + loser = + ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Alice", + birthdate: ~D[1985-07-22] + }) + + {:ok, survivor_reminder} = Kith.Reminders.create_birthday_reminder(survivor, ctx.user.id) + {:ok, _loser_reminder} = Kith.Reminders.create_birthday_reminder(loser, ctx.user.id) + + # Force the loser's birthdate to win. + assert {:ok, merged} = + Contacts.merge_cluster(ctx.scope, survivor.id, [loser.id], %{ + fields: %{birthdate: ~D[1985-07-22], birthdate_year_unknown: false}, + drop: %{} + }) + + assert merged.birthdate == ~D[1985-07-22] + + reminder = Kith.Reminders.get_birthday_reminder(merged.id, ctx.account_id) + + assert reminder + assert reminder.id == survivor_reminder.id + assert reminder.next_reminder_date == Kith.TimeHelper.next_birthday_date(~D[1985-07-22]) + end + + test "clearing the birthdate leaves the surviving birthday reminder alone", ctx do + # A merge that clears a birthdate is not the same event as a user + # removing one, so the kept reminder survives untouched — the engine + # destroys birthday reminders only to resolve the unique-index collision. + survivor = + ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Alice", + birthdate: ~D[1985-01-05] + }) + + loser = ContactsFixtures.contact_fixture(ctx.account_id, %{first_name: "Alice"}) + + {:ok, reminder} = Kith.Reminders.create_birthday_reminder(survivor, ctx.user.id) + + assert {:ok, merged} = + Contacts.merge_cluster(ctx.scope, survivor.id, [loser.id], %{ + fields: %{birthdate: :clear, birthdate_year_unknown: false}, + drop: %{} + }) + + assert is_nil(merged.birthdate) + + kept = Kith.Reminders.get_birthday_reminder(merged.id, ctx.account_id) + + assert kept + assert kept.id == reminder.id + assert kept.next_reminder_date == reminder.next_reminder_date + end + + test "merging two needs_review contacts keeps the survivor reviewable", ctx do + survivor = + ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Alice", + immich_status: "needs_review" + }) + + loser = + ContactsFixtures.contact_fixture(ctx.account_id, %{ + first_name: "Alice", + immich_status: "needs_review" + }) + + fields = + [survivor, loser] + |> Kith.Contacts.MergeResolution.resolve(survivor.id) + |> Map.fetch!(:fields) + + assert {:ok, merged} = + Contacts.merge_cluster(ctx.scope, survivor.id, [loser.id], %{ + fields: fields, + drop: %{} + }) + + assert merged.immich_status == "needs_review" + + assert merged.id in Enum.map(Contacts.list_needs_review(ctx.account_id), & &1.id) + end + end + + describe "stay-in-touch reminders" do + setup ctx do + survivor_reminder = + stay_in_touch_reminder_fixture(ctx.account_id, ctx.contact_a.id, ctx.user.id, "monthly") + + loser_reminder = + stay_in_touch_reminder_fixture(ctx.account_id, ctx.contact_b.id, ctx.user.id, "weekly") + + Map.merge(ctx, %{ + survivor_reminder: survivor_reminder, + loser_reminder: loser_reminder + }) + end + + test "leaves the survivor with exactly one active stay-in-touch reminder", ctx do + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + count = + Repo.aggregate( + from(r in Kith.Reminders.Reminder, + where: r.contact_id == ^survivor.id and r.type == "stay_in_touch" and r.active == true + ), + :count + ) + + assert count == 1 + end + + test "keeps the survivor's own reminder, not the loser's", ctx do + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + kept = + Repo.one!( + from(r in Kith.Reminders.Reminder, + where: r.contact_id == ^survivor.id and r.type == "stay_in_touch" and r.active == true + ) + ) + + assert kept.id == ctx.survivor_reminder.id + assert kept.frequency == "monthly" + end + + test "cancels the Oban jobs of the reminder it discards", ctx do + job = notification_job!(ctx.loser_reminder) + + {:ok, _survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert Repo.get(Oban.Job, job.id).state in ["cancelled", "discarded"] + end + + # The regression this whole task exists for: a second active reminder makes + # Reminders.resolve_stay_in_touch_instance/1 raise Ecto.MultipleResultsError. + test "logging an interaction after the merge does not raise", ctx do + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert {:ok, _} = Kith.Reminders.resolve_stay_in_touch_instance(survivor.id) + end + end + + describe "me_contact_id" do + test "repoints a user whose me-contact is merged away", ctx do + {:ok, _user} = Kith.Accounts.link_me_contact(ctx.user, ctx.contact_b.id) + + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert Repo.get!(Kith.Accounts.User, ctx.user.id).me_contact_id == survivor.id + end + + test "leaves a user whose me-contact is the survivor untouched", ctx do + {:ok, _user} = Kith.Accounts.link_me_contact(ctx.user, ctx.contact_a.id) + + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert Repo.get!(Kith.Accounts.User, ctx.user.id).me_contact_id == survivor.id + end + + test "does not touch a user in another account", ctx do + other = Kith.AccountsFixtures.user_fixture() + other_contact = ContactsFixtures.contact_fixture(other.account_id) + {:ok, _} = Kith.Accounts.link_me_contact(other, other_contact.id) + + {:ok, _survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + assert Repo.get!(Kith.Accounts.User, other.id).me_contact_id == other_contact.id + end + + # The slow-burn failure this guards: the pointer survives the merge, then + # the purge worker hard-deletes the loser and the FK nilifies it away. + test "survives the 30-day purge of the merged-away contact", ctx do + {:ok, _user} = Kith.Accounts.link_me_contact(ctx.user, ctx.contact_b.id) + + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + Repo.query!("DELETE FROM contacts WHERE id = $1", [ctx.contact_b.id]) + + assert Repo.get!(Kith.Accounts.User, ctx.user.id).me_contact_id == survivor.id + end + end + + describe "inactive birthday reminders" do + setup ctx do + import Kith.RemindersFixtures + + reminder = + birthday_reminder_fixture(ctx.account_id, ctx.contact_a.id, ctx.user.id) + + Repo.update_all( + from(r in Kith.Reminders.Reminder, where: r.id == ^reminder.id), + set: [active: false, enqueued_oban_job_ids: []] + ) + + # A birthdate on the loser that the merge will gap-fill onto the + # survivor, forcing next_reminder_date to change. + Repo.update_all( + from(c in Kith.Contacts.Contact, where: c.id == ^ctx.contact_b.id), + set: [birthdate: ~D[1990-06-15]] + ) + + Map.put(ctx, :reminder, reminder) + end + + test "does not enqueue jobs for a deactivated reminder", ctx do + {:ok, _survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + kept = Repo.get!(Kith.Reminders.Reminder, ctx.reminder.id) + + assert kept.active == false + assert kept.enqueued_oban_job_ids == [] + end + + test "still tracks the merged birthdate on the date field", ctx do + {:ok, survivor} = Contacts.merge_contacts(ctx.contact_a.id, ctx.contact_b.id) + + kept = Repo.get!(Kith.Reminders.Reminder, ctx.reminder.id) + + assert survivor.birthdate == ~D[1990-06-15] + assert kept.next_reminder_date == Kith.TimeHelper.next_birthday_date(~D[1990-06-15]) + end + end end diff --git a/test/kith/duplicate_detection_test.exs b/test/kith/duplicate_detection_test.exs index e72891ea..6660c840 100644 --- a/test/kith/duplicate_detection_test.exs +++ b/test/kith/duplicate_detection_test.exs @@ -237,4 +237,208 @@ defmodule Kith.DuplicateDetectionTest do assert DuplicateDetection.pending_count(account1.id) == 0 end end + + describe "resolve_after_merge/4" do + setup do + Kith.ContactsFixtures.seed_reference_data!() + user = Kith.AccountsFixtures.user_fixture() + account_id = user.account_id + + names = [a: "Ann", b: "Bea", c: "Cal", d: "Dee", e: "Eve"] + + contacts = + Map.new(names, fn {key, name} -> + {key, Kith.ContactsFixtures.contact_fixture(account_id, %{first_name: name})} + end) + + %{user: user, account_id: account_id, contacts: contacts} + end + + defp pair!(account_id, one, two, status \\ "pending") do + {low, high} = if one.id < two.id, do: {one, two}, else: {two, one} + + Repo.insert!(%Kith.Contacts.DuplicateCandidate{ + account_id: account_id, + contact_id: low.id, + duplicate_contact_id: high.id, + score: 0.9, + status: status, + detected_at: DateTime.utc_now(:second) + }) + end + + defp status_of(account_id, one, two) do + {low, high} = if one.id < two.id, do: {one, two}, else: {two, one} + + Repo.one( + from(d in Kith.Contacts.DuplicateCandidate, + where: + d.account_id == ^account_id and d.contact_id == ^low.id and + d.duplicate_contact_id == ^high.id, + select: d.status + ) + ) + end + + test "pairs inside the merged set become merged", ctx do + %{a: a, b: b} = ctx.contacts + pair!(ctx.account_id, a, b) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], []) + + assert status_of(ctx.account_id, a, b) == "merged" + end + + test "pairs from a merged member to an unchecked member become dismissed", ctx do + %{a: a, b: b, d: d} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, a, d) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], [d.id]) + + assert status_of(ctx.account_id, a, d) == "dismissed" + end + + test "pairs between two unchecked members are untouched", ctx do + %{a: a, b: b, d: d, e: e} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, d, e) + + :ok = + Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], [d.id, e.id]) + + assert status_of(ctx.account_id, d, e) == "pending" + end + + test "a loser's dismissal is repointed onto the survivor", ctx do + %{a: a, b: b, d: d} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, b, d, "dismissed") + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], [d.id]) + + assert status_of(ctx.account_id, a, d) == "dismissed" + assert status_of(ctx.account_id, b, d) == nil + end + + # `d` is deliberately NOT unchecked: set_status/5 then leaves both a-d and + # b-d alone, so the two rows still hold *different* statuses by the time + # repoint/3 collapses b-d onto a-d and @status_rank has to choose. Passing + # `d` as unchecked would flatten both to "dismissed" first and the ranking + # would never be exercised. + test "repointing keeps the strongest status on collision", ctx do + %{a: a, b: b, d: d} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, a, d, "merged") + pair!(ctx.account_id, b, d, "pending") + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], []) + + assert status_of(ctx.account_id, a, d) == "merged" + end + + test "the winning row keeps its own score, reasons and detected_at", ctx do + %{a: a, b: b, d: d} = ctx.contacts + pair!(ctx.account_id, a, b) + + existing = pair!(ctx.account_id, a, d, "merged") + losing = pair!(ctx.account_id, b, d, "pending") + + detected = ~U[2025-06-01 00:00:00Z] + + Repo.update_all( + from(x in Kith.Contacts.DuplicateCandidate, where: x.id == ^existing.id), + set: [score: 0.42, reasons: ["existing"], detected_at: detected] + ) + + Repo.update_all( + from(x in Kith.Contacts.DuplicateCandidate, where: x.id == ^losing.id), + set: [score: 0.11, reasons: ["repointed"], detected_at: ~U[2026-06-01 00:00:00Z]] + ) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], []) + + {low, high} = if a.id < d.id, do: {a.id, d.id}, else: {d.id, a.id} + + row = + Repo.one!( + from(x in Kith.Contacts.DuplicateCandidate, + where: + x.account_id == ^ctx.account_id and x.contact_id == ^low and + x.duplicate_contact_id == ^high + ) + ) + + assert row.status == "merged" + assert row.score == 0.42 + assert row.reasons == ["existing"] + assert DateTime.compare(row.detected_at, detected) == :eq + end + + test "pairs in an unrelated cluster are untouched", ctx do + %{a: a, b: b, d: d, e: e} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, d, e) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], []) + + assert status_of(ctx.account_id, d, e) == "pending" + end + + test "another account's pending pair is untouched", ctx do + %{a: a, b: b} = ctx.contacts + + other_user = Kith.AccountsFixtures.user_fixture() + other_account_id = other_user.account_id + other_x = Kith.ContactsFixtures.contact_fixture(other_account_id, %{first_name: "Xan"}) + other_y = Kith.ContactsFixtures.contact_fixture(other_account_id, %{first_name: "Yaz"}) + pair!(other_account_id, other_x, other_y) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], []) + + assert status_of(other_account_id, other_x, other_y) == "pending" + end + + test "two losers repointing to the same third contact collide without raising", ctx do + %{a: a, b: b, c: c, d: d} = ctx.contacts + pair!(ctx.account_id, a, b) + pair!(ctx.account_id, a, c) + pair!(ctx.account_id, b, c) + pair!(ctx.account_id, b, d, "dismissed") + pair!(ctx.account_id, c, d) + + :ok = + Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id, c.id], [d.id]) + + assert status_of(ctx.account_id, a, d) == "dismissed" + assert status_of(ctx.account_id, b, d) == nil + assert status_of(ctx.account_id, c, d) == nil + assert status_of(ctx.account_id, b, c) == "merged" + end + + test "every merged row has at least one trashed endpoint", ctx do + %{a: a, b: b} = ctx.contacts + pair!(ctx.account_id, a, b) + + Repo.update_all(from(c in Kith.Contacts.Contact, where: c.id == ^b.id), + set: [deleted_at: DateTime.utc_now(:second)] + ) + + :ok = Kith.DuplicateDetection.resolve_after_merge(ctx.account_id, a.id, [b.id], []) + + merged_rows = + from(d in Kith.Contacts.DuplicateCandidate, + where: d.status == "merged", + join: c1 in Kith.Contacts.Contact, + on: c1.id == d.contact_id, + join: c2 in Kith.Contacts.Contact, + on: c2.id == d.duplicate_contact_id, + select: {c1.deleted_at, c2.deleted_at} + ) + |> Repo.all() + + assert merged_rows != [] + assert Enum.all?(merged_rows, fn {one, two} -> one != nil or two != nil end) + end + end end diff --git a/test/kith_web/controllers/api/contact_controller_test.exs b/test/kith_web/controllers/api/contact_controller_test.exs index 3bdee7a0..873e7ee8 100644 --- a/test/kith_web/controllers/api/contact_controller_test.exs +++ b/test/kith_web/controllers/api/contact_controller_test.exs @@ -1,6 +1,7 @@ defmodule KithWeb.API.ContactControllerTest do use KithWeb.ConnCase + import Ecto.Query import Kith.AccountsFixtures import Kith.ContactsFixtures @@ -173,6 +174,44 @@ defmodule KithWeb.API.ContactControllerTest do get_resp = conn |> get(~p"/api/contacts/#{non_survivor.id}") |> json_response(404) assert get_resp["status"] == 404 end + + test "keeps the survivor's value when the non-survivor is newer, and fills gaps", %{ + conn: conn + } do + user = user_fixture() + conn = authed_conn(conn, user) + + survivor = + contact_fixture(user.account_id, %{ + first_name: "Survivor", + last_name: "Smith", + middle_name: nil + }) + + non_survivor = + contact_fixture(user.account_id, %{ + first_name: "NonSurvivor", + last_name: "Jones", + middle_name: "Filled" + }) + + # The resolver breaks a 1-1 conflict by most-recently-updated, so make the + # non-survivor unambiguously newer: without the survivor-value protection + # this merge renames the survivor to "NonSurvivor". + Kith.Repo.update_all( + from(c in Kith.Contacts.Contact, where: c.id == ^non_survivor.id), + set: [updated_at: DateTime.add(DateTime.utc_now(:second), 3600, :second)] + ) + + body = %{"survivor_id" => survivor.id, "non_survivor_id" => non_survivor.id} + + resp = conn |> post(~p"/api/contacts/merge", body) |> json_response(200) + + assert resp["data"]["first_name"] == "Survivor" + assert resp["data"]["last_name"] == "Smith" + # A field the survivor had no value for is still gap-filled (design spec §6). + assert resp["data"]["middle_name"] == "Filled" + end end # ── Archive (TEST-10-17) ─────────────────────────────────────────── diff --git a/test/kith_web/live/contact_live/merge_test.exs b/test/kith_web/live/contact_live/merge_test.exs new file mode 100644 index 00000000..28d879c0 --- /dev/null +++ b/test/kith_web/live/contact_live/merge_test.exs @@ -0,0 +1,122 @@ +defmodule KithWeb.ContactLive.MergeTest do + use KithWeb.ConnCase, async: true + + import Phoenix.LiveViewTest + import Kith.ContactsFixtures + + setup :register_and_log_in_user + + setup %{user: user} do + seed_reference_data!() + + survivor = + contact_fixture(user.account_id, %{ + first_name: "Alice", + last_name: "Smith", + company: nil, + occupation: "Designer" + }) + + loser = + contact_fixture(user.account_id, %{ + first_name: "Alice", + last_name: "S.", + company: "Acme", + occupation: nil + }) + + %{survivor: survivor, loser: loser, account_id: user.account_id} + end + + # Opens the wizard already on step 2 with the loser selected. The `?with=` + # param routes through `maybe_preselect_contact/3`, which is the same path + # the duplicate-review screen links in with — and it avoids driving the + # step-1 search, whose results are empty until a "search" event fires. + defp open_on_step_two(conn, survivor, loser) do + {:ok, view, _html} = live(conn, ~p"/contacts/#{survivor.id}/merge?with=#{loser.id}") + view + end + + # Merges without touching a single field radio. + defp merge_without_choosing(conn, survivor, loser) do + view = open_on_step_two(conn, survivor, loser) + + render_click(view, "go-to-preview", %{}) + render_click(view, "execute-merge", %{}) + + view + end + + describe "merging without touching any field radio" do + test "fills a survivor gap from the loser instead of clearing it", + %{conn: conn, survivor: survivor, loser: loser, account_id: account_id} do + merge_without_choosing(conn, survivor, loser) + + merged = Kith.Contacts.get_contact!(account_id, survivor.id) + assert merged.company == "Acme" + end + + test "still protects a value the survivor already holds", + %{conn: conn, survivor: survivor, loser: loser, account_id: account_id} do + merge_without_choosing(conn, survivor, loser) + + merged = Kith.Contacts.get_contact!(account_id, survivor.id) + assert merged.occupation == "Designer" + end + end + + describe "field choice radios" do + test "an untouched field with a nil survivor value highlights the loser column", + %{conn: conn, survivor: survivor, loser: loser} do + view = open_on_step_two(conn, survivor, loser) + + survivor_button = + view + |> element(~s{button[phx-value-field="company"][phx-value-source="survivor"]}) + |> render() + + loser_button = + view + |> element(~s{button[phx-value-field="company"][phx-value-source="non_survivor"]}) + |> render() + + # `company` is nil on the survivor and "Acme" on the loser, so the + # loser's value is the one that survives — the highlight must be on + # that button, not the survivor's. + assert loser_button =~ "color-accent-subtle" + refute survivor_button =~ "color-accent-subtle" + end + + test "an untouched field the survivor holds keeps the highlight on the survivor", + %{conn: conn, survivor: survivor, loser: loser} do + view = open_on_step_two(conn, survivor, loser) + + survivor_button = + view + |> element(~s{button[phx-value-field="occupation"][phx-value-source="survivor"]}) + |> render() + + loser_button = + view + |> element(~s{button[phx-value-field="occupation"][phx-value-source="non_survivor"]}) + |> render() + + # `occupation` is "Designer" on the survivor and nil on the loser, so + # the survivor's value stands and its button keeps the highlight. + assert survivor_button =~ "color-accent-subtle" + refute loser_button =~ "color-accent-subtle" + end + + test "an explicit survivor choice still pins a nil, clearing the field", + %{conn: conn, survivor: survivor, loser: loser, account_id: account_id} do + view = open_on_step_two(conn, survivor, loser) + + render_click(view, "choose-field", %{"field" => "company", "source" => "survivor"}) + render_click(view, "go-to-preview", %{}) + render_click(view, "execute-merge", %{}) + + merged = Kith.Contacts.get_contact!(account_id, survivor.id) + assert merged.company == nil + end + end +end