feat(territories): unified search with proximity ranking (#138)#206
Merged
Conversation
… filter chips Search across territory, attribution and building filter pages now trims input, strips diacritics, and matches case-insensitively against harmonised scope: - Territory pages add publisher firstname/lastname via current attribution - Attribution page adds building street/number/zip via the attached territory - Building/prospection pages use the same normalized street comparison Three new columns hold pre-normalized (lowercased, accent-stripped) values: - Member.firstnameNormalized - Member.lastnameNormalized - Building.streetNormalized The migration enables the unaccent extension temporarily to backfill the columns, then drops it — the runtime DB role never needs the extension. All service writes (createMember, updateMember, createBuilding, editBuilding, anonymize, link-member, update-account, NDJSON import, open-data import) now maintain these columns via the new stripDiacritics() util. Above each filter form, an ActiveTerritoryFilters component renders one chip per applied filter plus a "Tout effacer" link. Chips drop their query param (and reset page) when clicked. Also accepts a leading `@` proximity marker in the search query — stripped for the text branch; proximity ranking lands in the next commit.
The search input now resolves address-like queries to coordinates via the
Google Maps Geocoding API and ranks results by distance from that point. A
single `@` prefix forces proximity even on short or non-address queries
(`@Bastille`).
Behaviour:
- A heuristic (`classifySearch`) only sends queries to the geocoder when they
have 3+ tokens, contain a French way-type (`rue`, `avenue`, `boulevard`,
…), or match `number street`. Short ambiguous strings (`12`, `D012`,
surnames) stay text-only unless explicitly @-prefixed.
- Geocoded results land in a Redis cache (`geocode:v1:<normalized query>`,
90-day TTL — addresses are stable). Missing results are cached as the
empty sentinel so repeat misses don't burn the API.
- When the API key is unset or the call fails, search degrades to text-only
— no banner, no Distance column.
- A `ProximityBanner` confirms the resolved address, lets the user clear
the point, and surfaces up to two alternates as clickable "Vouliez-vous
dire?" chips.
- A new `Distance` column appears second from the left only when proximity
is active. Territories with no geocoded entrance or building are grouped
under a "Sans coordonnées (N)" divider row.
- A `?sort=` toggle (Numéro / Date / Proximité depending on page) defaults
to Proximité on a geocode hit and lets the user revert to the standard
sort without dropping the banner or the distance column.
Distances use Haversine + `BuildingEntrance.{lat,lng}` with a fallback to
the parent Building coordinates. The "Sans coordonnées" group is preserved
in its original DB order so users with sparse coordinates aren't surprised.
…collapsible
The search input on every territory page now cycles through three placeholder
examples ("Pajot" → address → "@bastille") to surface the multiple intents the
classifier supports. Rotation pauses on focus so the hint doesn't move while
the user is typing. An ⓘ popover next to the input documents the `@`
proximity operator and the auto-detection, and notes that proximity is
disabled when no Google Maps key is configured.
On `max-sm`, the Select filters (zip, type, access, group, shops, status)
collapse behind a "Filtres avancés" toggle so the search input and submit
button keep the prominent position on a narrow screen. On `sm+` the Selects
render inline as before.
The Selects render once inside the form even when mobile-collapsed (a CSS
`contents` wrapper hides them on small screens; the Collapsible exposes the
same nodes), so the form payload is identical regardless of breakpoint.
The marketing seed (`seed-marketing.ts`) was creating Members and Buildings without filling `firstnameNormalized`, `lastnameNormalized`, `streetNormalized`, so they kept the empty-string column default and silently broke name-based search on demo data after the new columns landed.
…x Sans coordonnées affordance Four merge-blockers from the UX review: - GeocodeNotice (warning Alert) now renders above filters when a query was sent to the geocoder but came back empty (failed call, no result, or no GOOGLE_MAPS_API_KEY). Wires the previously-dead `geocodeAttempted` field and `territories_filter_geocode_failed` i18n key. - `@` alone now flips classifySearch to `forced: true` with `geoQuery: null`, letting the same notice prompt the user to type a place — the operator mode is no longer silently swallowed. - Active filter chips: the body and X are now separate targets. Clicking the label/value does nothing; only the trailing X removes the filter, with a destructive hover state so the intent reads visually. Prevents accidental filter removal when scanning the chip row. - "Sans coordonnées" tail row restyled from italic-muted-xs to a dashed divider with a count Badge — reads as a section break rather than a footnote. When the current page lies entirely past the partition boundary (so the in-table divider would never render), a banner above the table tells the user they are in the un-coord tail. Also adds a `warning` variant to the shared `Alert` primitive (amber palette matching the existing `warning` Badge).
…lumn, mobile trigger Visual changes from the UI review: - ProximityBanner reads as a confirmation: blue-500 left accent, blue-50 background, blue-600 MapPin. Alternates muted so they sit subordinate to the resolved address. - SearchInputWithHelp is now a proper input group — Input gets `pr-9` and the ⓘ button is absolutely positioned inside the right edge. Reads as one field instead of two adjacent controls. - Distance column right-aligned with `tabular-nums` (numeric convention), using `text-foreground/80` instead of muted-foreground so the value keeps weight. The column is still gated on `proximityActive` — only shown when geocoding succeeded. - Mobile "Filtres avancés" trigger upgraded from ghost-link to an outline Button with a leading `SlidersHorizontal` icon and a trailing `ChevronDown` that rotates on open. Reads as a proper control instead of helper copy. - Submit Button icon swapped from `SlidersHorizontal` (which conventionally means "open filters") to `Search`. `SlidersHorizontal` is now reserved for the mobile collapsible trigger where it actually helps discoverability. - "Modifier le point" → "Effacer la proximité" — the verb now matches what the link actually does (clears search/sort/page).
…rotation after typing Long-tail polish from the UX review: - Sort Select renders even when only one option is available (e.g. "Numéro" on a non-proximity territory list). Makes the sort control discoverable as a category before proximity unlocks it. Adds an `ArrowUpDown` icon in the trigger for at-a-glance recognition. - Help popover body switched from a single paragraph to a bulleted list of search modes (name / number / address / `@`-proximity), with the no-key disclaimer demoted to a smaller footer line below. - Placeholder rotation stops permanently once the user has typed anything — the hint has served its purpose and shouldn't shift back to examples after the user clears their query. "Tout effacer" already sits inline as the trailing chip in the chip flow (landed with the chip restructure in commit 4), so no extra change here.
…, isFinite, banner Six bugs surfaced by the code-review pass: - **geocoder.server.ts**: the Google call was passing the Redis cache key as the API key (shadowed `apiKey` with the local `key`). Every production geocode would return REQUEST_DENIED, get caught as warn, and pollute the cache with the empty sentinel for 90 days. Now passes `apiKey` correctly, logs non-OK statuses at error, and skips caching for non-OK responses so a transient outage doesn't poison results. - **TerritoryFilters / AttributionFilters**: `advancedSelects` was rendered twice inside the same `<Form>` (once `max-sm:hidden`, once in the `sm:hidden` Collapsible). Radix Select injects a hidden form input for each, and `display:none` does not exclude inputs from form submission — mobile users' filter choices were silently dropped because the desktop default arrived first in URLSearchParams. The advanced Selects now render once; their visibility is toggled via CSS based on the mobile open state. - **proximity-sort.server.ts**: added `Number.isFinite` guards on both entrance and building coordinates so NaN (possible from open-data CSV imports via `Number(lat)`) and Infinity are rejected like null. NaN flowing through Haversine would have produced unstable sort order and rendered as the same em-dash as a missing coord. - **proximity-sort.server.test.ts**: fixed `lng` → `longitude` typo on the building-fallback fixture that silently skipped the entire "two competing buildings, pick closer" path. The test now exercises it. - **import-congregation.server.ts**: building update path also backfills `streetNormalized` (previously only the create path set it, so re-imports left stale empty rows that name-search would silently miss). - **list routes**: the `ProximityBanner` is now shown whenever the geocode hit, regardless of the active sort. Combined with: a typed address auto-flips the default sort to `proximity`. The user can still pick `numéro`/`date` via the Select without losing the banner. - **address-regex.ts**: French repeater `quarter` → `quater`. The English spelling never matched. Also: geocoder test now asserts `params.key` is the configured API key, and covers non-OK status (REQUEST_DENIED no-cache), malformed cache fallback, and unknown `location_type` coercion to `OTHER`.
…tion clamp Important fixes from the review pass — second batch. - Migration `20260606000000_add_normalized_search_columns` no longer depends on `CREATE EXTENSION unaccent` (which managed Postgres often refuses outside the bootstrap role). The migration now only adds columns and indexes; backfill of existing rows runs in TypeScript via the new one-shot script `app/database/backfill-normalized.ts` reusing the same `stripDiacritics()` helper as the runtime writes. - Drop three dead i18n keys (`territories_filter_help_body`, `territories_filter_no_coordinates_count`, and `..._group`) that were introduced earlier in the PR and never referenced. - Replace the duplicated `getTerritoryTypeLabel` + `territoryContentLabel` inlines in `attributions/territories.tsx` with the shared helper that already exists in `server/territory-content-label.ts`. Use the same `territoryTypeLabels` Record style as `territory/list.tsx` for the type column. - `formatDistance` accepts a `locale` parameter (default still `'fr-FR'`) and memoizes formatter instances per locale. Loaders read `getLocale()` from Paraglide and forward it, so English-locale builds now render English unit conventions (`1.2 km` vs `1,2 km`). - Drop the dead `lateDate == null` branches in the attribution-list client sort — `Attribution.lateDate` is non-null per the Prisma schema. - Tighten `GeocodeNoticeData` to a discriminated union so `query` is required when `kind === 'failed'` and not present otherwise. Loaders now declare the type using the imported alias instead of an inline literal. - `paginationFromUrl` clamps `page` to `[1, max(1, pages)]` — an out-of-range `?page=` now lands on the last page instead of silently returning an empty table. - biome.json: extend the seed-script override to cover the new backfill-normalized.ts file (allow console output, etc.).
Three type-design improvements from the review pass: - `ActiveTerritoryFilterChip.key` is now a `TerritoryFilterKey` string literal union (`'search' | 'zip' | 'type' | 'access' | 'shops' | 'group' | 'status'`). A typo (`'zipcode'`) producing a chip whose X clears nothing is now a compile-time error. `appendChip` in `build-filter-chips.ts` propagates the constraint. - `sortFromUrl<A extends SortMode>(url, allowed, fallback): A` — the return type now narrows to the supplied subset, so callers' `sort !== 'number'` discriminations stay statically correct. - `GeocodeResult.locationType` already gained an `'OTHER'` fallback in C7 — keeping the runtime cast covered against new Google enum values (e.g. `PLUS_CODE`). Test additions: - `search-intent.server.test.ts` — adversarial cases: leading-whitespace `@`, double `@@`, the 2-token `Jean Dupont` non-address, and the documented `Rue`-alone street-word fallback. - `pagination.server.test.ts` — out-of-range page clamp, negative page fallback, non-numeric page fallback. - `build-filter-chips.test.ts` (new) — verifies `'none'` is dropped, unknown enum values produce no chip, the search chip is trimmed, group ids fall back to `null` when missing.
…gration Switch the normalized-column backfill from a separate `pnpm tsx` script to pure SQL inside the migration itself, so `prisma migrate deploy` populates existing rows automatically — no follow-up manual step required. Uses Postgres `translate()` with an explicit French-friendly character map (à→a, ä→a, ç→c, é→e, ñ→n, etc.) instead of the `unaccent` extension. The map matches `stripDiacritics()`'s output for every codepoint in it, and the migration stays privilege-safe — managed Postgres hosts that refuse `CREATE EXTENSION` still run this cleanly. Drops `app/database/backfill-normalized.ts` (and its biome.json override) since the migration now handles the backfill itself. The WHERE guard on the UPDATE keeps the SQL idempotent: re-running the migration won't touch rows whose normalized columns are already populated by either path.
…imity loaders
Closes the integration-test gaps the review pass flagged. Pure-function
coverage was already solid; this commit adds end-to-end DB exercises and a
Playwright smoke spec so we can deploy with confidence.
- **Migration backfill parity** (`backfill-parity.integration.test.ts`):
asserts the `translate()` map in the migration SQL produces the same
output as `stripDiacritics()` for representative French inputs
("François", "Hélène", "Champs-Élysées", …) by running the same SQL via
`$queryRawUnsafe`. **This test caught an off-by-one in the migration
map** (seven 'o' replacements instead of six) before merge — exactly the
silent-corruption risk the review flagged. Also locks the SQL/JS divergence
on ligatures and `ø` (both preserved on both sides).
- **Member normalized columns**
(`publishers/server/normalized-columns.integration.test.ts`):
`createMember`, `updateMember`, and `anonymizeMember` all confirm the
`firstnameNormalized` / `lastnameNormalized` columns actually land on
disk — not just that the value was passed to a Prisma mock.
- **Building normalized columns**
(`territories/server/building-normalized-columns.integration.test.ts`):
`createBuilding`, `editBuilding`, plus the import-congregation update
path (mirroring the data block) confirm `streetNormalized` writes
through and refreshes stale values from legacy rows.
- **Proximity loaders**
(`territories/server/proximity-loader.integration.test.ts`):
`findTerritoriesWithDetailsPaginated` and
`findActiveAttributionsPaginated` with `proximityArgs.origin` return
rows in distance order, populate the per-row `distances` map, report
`withCoordsCount` / `withoutCoordsCount` correctly, push no-coords
rows to the tail, and paginate the combined list. The non-proximity
branch still returns the shape callers rely on.
- **E2E** (`territories-search.spec.ts`): Playwright smoke covering the
search input + rotating placeholder, the `?search=` URL round-trip
through the active-filter chip, and "Tout effacer" wiping every query
param.
The migration map fix is co-shipped here because the parity test caught
it — running the integration test before merge is what surfaces it.
Integration suite: 209 tests pass. Unit: 1216 pass. Lint + typecheck clean.
…feature Product docs: - `product/territories.md` gains a "Search and Filtering" section covering the single-input search (names / numbers / addresses / neighbourhoods), the `@` operator, accent-insensitivity, the proximity banner and Distance column, the active-filter chips, "Tout effacer", and the mobile "Filtres avancés" Collapsible. - `product/feature-overview.md` Territories section gets a one-line entry for the unified search + proximity ranking, linking to the territories doc. Technical docs: - `development/architecture.md` gains a "Territory Search & Geocoding" section between "Google Maps Integration" and "PDF Generation": covers the normalized columns + write-time maintenance, the migration's pure-SQL backfill via `translate()`, the geocoder service (Redis cache, 90-day TTL, no-key fallback, non-OK no-cache, 5s timeout, `OTHER` location-type fallback), the `classifySearch` heuristic, the proximity pagination, the loader wiring with the discriminated `geocodeNotice` union, the mobile single-Select-render bug fix, and the integration/E2E test layout. - `self-hosting/environment-variables.md` notes that `GOOGLE_MAPS_API_KEY` now also powers the proximity search via the Geocoding API and that the cache TTL is 90 days; the fallback when the key is absent is documented. - `self-hosting/requirements.md` extends the Google Maps outbound bullet to mention proximity-search geocoding.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #138 — and goes beyond. Unifies the territory/attribution/buildings search & filter UX (case/accent-insensitive, harmonised scope, active-filter chips, "Tout effacer"), and (scope expansion agreed mid-PR) adds proximity search via Google Maps geocoding with a Distance sort. Multiple senior code review passes (UX, UI, code-review, test-coverage, silent-failure, type-design, comment-analyser, simplifier) and all their findings were folded in before requesting review.
What changed
Search
Member.firstname,Member.lastname,Building.street. Maintained at every write path (createMember,updateMember,anonymize-member,link-member,update-account,import-congregation,create-building,edit-building,import-open-data,seed-marketing).@-prefix forces proximity even on short queries (@Bastille); auto-detection triggers geocoding for address-shaped or street-word-bearing queries.Proximity (scope expansion)
@googlemaps/google-maps-services-js) with Redis cache (geocode:v1:<query>, 90-day TTL, 5s timeout,OTHERfallback for unknown Google enum values).BuildingEntrance.{latitude,longitude}with fallback toBuilding.{latitude,longitude}.Number.isFiniteguard rejects NaN/(∞) coords from open-data imports.paginateByProximitypartitions results into "with coords" (sorted by distance) + "without coords" (preserve default order), then paginates the combined list.ProximityBannerconfirms the resolved address, surfaces up to 2 "Vouliez-vous dire?" alternates, and offers an "Effacer la proximité" link. Banner shows whenever the geocode hit — even when the user reverts to the default sort.Distancecolumn (right-aligned, tabular-nums) only renders when the proximity sort is active.Filter UX
ActiveTerritoryFilterschips component — one chip per applied filter with an isolated X target (chip body is inert, X only). "Tout effacer" renders inline as a trailing chip.GeocodeNoticewarning Alert renders when geocoding was attempted but no result (failed) or the user typed@alone (missing-query).SlidersHorizontal+ rotatingChevronDown). Advanced Selects render once (CSS-toggled) to avoid Radix injecting duplicate hidden form inputs on mobile.ex. Pajot · 12 rue de la Paix · @Bastille), terminal once the user types.@operator and the auto-detection.Selectalways renders (withArrowUpDownicon) so users discover the control before proximity unlocks it.Migration
20260606000000_add_normalized_search_columns/migration.sqladds the three normalized columns + indexes, and backfills existing rows automatically using Postgrestranslate()with an explicit French-friendly char map. Nounaccentextension required — runs cleanly on managed Postgres without superuser. Idempotent: re-applying touches only rows still at the empty-string default.Notes for reviewers
This PR went through multiple review rounds. Worth highlighting bugs caught and fixed:
apiKeyand was sent to Google. Every production call would have returnedREQUEST_DENIED→ cached as empty for 90 days. Fixed + test now assertsmockGeocode.mock.calls[0]?.[0]?.params?.key === apiKey.advancedSelectswas rendered twice in the same<Form>; Radix Select injects a hidden form input in both copies;display:nonedoes not exclude inputs from form submission, so mobile submissions sent?zip=none&zip=75002andURLSearchParams.get('zip')returned the desktop default. Fixed by rendering Selects once and CSS-toggling visibility.Number.isFiniteguard on coords blocks NaN values from open-data CSV imports leaking into Haversine sort.OK/ZERO_RESULTSare).paginationFromUrlprevents?page=99rendering an empty table with no signal after a filter shrinks the total.addressRegextypo: French repeater isquater, notquarter. The English spelling never matched.Test plan
pnpm test:unit— 1216 tests passpnpm test:lint— 0 errors (37 pre-existing complexity warnings)pnpm test:typecheck— cleanpnpm test:integration— needs reviewer with live DBGOOGLE_MAPS_API_KEYset: type12 rue de la Paix→ banner appears, Distance column right-aligned, "Sans coordonnées" section at the tail;@Bastille→ forced geocode;D012→ no geocode (heuristic)GOOGLE_MAPS_API_KEY: same address →GeocodeNoticewarning above filters; text-only fallback works@alone → "Tapez un lieu après le @" noticepajotmatchesPäjot; searchdupontmatchesDupont(case-insensitive on normalized columns)