Skip to content

feat: unblock geo-enrichment (coords) + exact/estimate EPC with manual address override - #1

Merged
timothygithinji merged 3 commits into
mainfrom
feat/geo-coords-and-epc
May 27, 2026
Merged

feat: unblock geo-enrichment (coords) + exact/estimate EPC with manual address override#1
timothygithinji merged 3 commits into
mainfrom
feat/geo-coords-and-epc

Conversation

@timothygithinji

@timothygithinji timothygithinji commented May 27, 2026

Copy link
Copy Markdown
Owner

Why

A read of prod data showed the enrichment pipeline — gaff's differentiator — was mostly empty: only ~18% of clusters had crime/flood/amenities/council-tax, and EPC was 0/102. Root causes (both confirmed against prod, not guessed):

  1. Coordinates were stranded. Zoopla/OpenRent listings carry true per-property lat/lng in rawJson, but they were never promoted to the listings.lat/lng columns the enrichments read — so the geo tasks no-op'd for ~80% of clusters. (Geocoding from postcode doesn't help: scraped postcodes are outcodes only, e.g. "N11".)
  2. EPC had no way to pick the right cert. An outcode search returns the whole district's certificates with no coordinates to disambiguate, so the old code stamped one arbitrary cert on every cluster.

What

  • fix(enrich) — backfill stranded rawJson coords → listingsproperty_clusters, then re-fire the geo enrichments. (Already run against prod: geo-enrichment 18% → ~95%.)
  • feat(enrich-epc) — reverse-geocode the now-populated coords to a full unit postcode, exact-match a certificate by house number + street when the address has one, else a labelled postcode-level estimate (modal rating + range), else blank. Guards bedroom counts ("2 Bedroom Flat") from being mistaken for house numbers.
  • feat(epc)manual address override: property_clusters.user_address (migration 0014) + setClusterAddress server fn. The user pins the exact door (read off the photos vs Google Maps) → re-resolves EPC precisely. UI: a "Fix address for exact EPC" link → "Pin the exact address" dialog; cards show ~C est, listing detail shows EPC · area estimate · ~C · range C–E · N homes.

Verified

Driven in-app against prod data (logged in): geo-enrichment card populated, EPC renders ~D · area estimate · range C–E · 18 homes for a sample N14 listing, Fix-address dialog opens. Resolver across all clusters: 88 estimate / 1 exact / 1 none of 90. bun run build clean, 154 tests pass.

⚠️ Deploy notes

Done to prod DB already this session: migration 0014 applied, coords backfilled, EPC populated, geo-enrichment populated. Still needed: deploy this branch so the Worker serves the new code — until then the live site won't show the EPC display / dialog, and new scrapes won't self-promote coords or run the new enrich-epc. (The coords forward-path itself was already in main via 5c7fd543; it just needs deploying.)

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • New Features

    • Added ability to manually pin exact property address for improved EPC data accuracy.
    • EPC estimates now clearly marked with ~ indicator and show min/max rating ranges.
  • Improvements

    • Address editing available on both mobile and desktop listing views.
    • Better distinction between exact EPC certificates and postcode-level estimates.

Review Change Stack

…usters

Zoopla/OpenRent carry true per-property lat/lng in rawJson but they were
never promoted to the listings.lat/lng columns, leaving ~80% of clusters
coordinate-less and starving enrich-crime/flood/amenities/council-tax
(they no-op without coords). This one-off promotes rawJson coords ->
listings -> property_clusters and re-fires the geo enrichments. No
geocoding (the coords are ours; scraped postcodes are outcodes only).
…full postcode

Scraped postcodes are outcodes only and EPC certs carry no coordinates,
so the old code stamped one arbitrary district cert on every cluster.
Now: reverse-geocode the (newly-populated) cluster coords to a full unit
postcode, exact-match a cert by house number + street when the address
has one, else summarise the postcode into a labelled estimate (modal
rating + range), else blank. Guards bedroom counts ('2 Bedroom Flat')
from being read as house numbers. Backward-compatible epc blob.
Adds property_clusters.user_address (migration 0014) + setClusterAddress
server fn: the user pins the exact door (read off the photos vs Google
Maps), which re-fires enrich-epc to resolve a precise certificate. The
listing-detail PriceCard and mobile header expose a 'Fix address for
exact EPC' link -> a 'Pin the exact address' dialog. Review cards show
'~C est' and listing detail shows 'EPC area estimate ~C, range C-E, N
homes' so estimates never read as fake-precise.
@coderabbitai

coderabbitai Bot commented May 27, 2026

Copy link
Copy Markdown
ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: 5ea7d818-c961-45e7-83e5-facd3484f5d4

📥 Commits

Reviewing files that changed from the base of the PR and between e938563 and e7f29da.

📒 Files selected for processing (15)
  • db/schema.ts
  • drizzle/0014_cuddly_lily_hollister.sql
  • drizzle/meta/0014_snapshot.json
  • drizzle/meta/_journal.json
  • scripts/backfill-listing-coords.ts
  • src/components/listing-detail/desktop-listing-detail.tsx
  • src/components/review/info-row.tsx
  • src/components/review/review-card.tsx
  • src/lib/geocode.ts
  • src/routes/index.tsx
  • src/routes/listings/$clusterId.tsx
  • src/server/functions/listing-detail.ts
  • src/server/functions/review.ts
  • src/trigger/enrich-epc.test.ts
  • src/trigger/enrich-epc.ts

Walkthrough

This PR adds a postcode-and-address-driven EPC enrichment workflow, user address pinning capability, and estimate markers for postcode-level EPC data. Schema adds userAddress column; geocoding support via postcodes.io enables coordinate-to-postcode resolution; EPC enrichment refactors to prioritise exact address matching before falling back to postcode-level estimates; UI surfaces address editing and visually distinguishes estimates with ~ markers.

Changes

Core EPC Address Override & Enrichment

Layer / File(s) Summary
Schema migration: userAddress column
db/schema.ts, drizzle/0014_cuddly_lily_hollister.sql, drizzle/meta/_journal.json
Adds nullable userAddress text column to propertyClusters table for user-supplied address overrides, tracked in Drizzle migration.
Reverse postcode geocoding
src/lib/geocode.ts
Introduces reverseGeocodePostcode(lat, lng, options) helper that calls postcodes.io to snap cluster coordinates to full UK postcodes, enabling postcode resolution when exact address is unavailable.
Listing coordinate backfill script
scripts/backfill-listing-coords.ts
Executable script that promotes numeric coords from listings.rawJson, propagates to NULL cluster coords deterministically, and re-triggers four geo enrichment tasks in batches.
EPC enrichment: postcode-and-address-driven approach
src/trigger/enrich-epc.ts
Replaces coordinate-based certificate selection with: full postcode resolution (manual override → scraped postcode → reverse geocode), exact matching when house number + street word overlap is present, postcode-level estimate rollup when exact match unavailable, and provenance tracking (source: "exact" | "estimate" | "none"). Introduces pickExactCert, exactBlob, and estimateBlob helpers and rewrites enrichEpcTask run flow.
EPC enrichment test suite
src/trigger/enrich-epc.test.ts
Comprehensive tests for pickExactCert (address matching logic), exactBlob (10-year expiry, camelCase mapping), and estimateBlob (postcode modal aggregation, tie-breaking) replacing the prior normalisation tests.
Listing detail server function: address override & EPC metadata
src/server/functions/listing-detail.ts
Extended ListingDetailCluster to include userAddress: string | null and ListingDetailEpc to include source?: "exact" | "estimate", range?: { min; max }, and sampleSize?: number. Added setClusterAddress(clusterId, address) POST function that persists override, enforces household ownership, and triggers enrich-epc re-enrichment.
Review card: epcIsEstimate flag
src/server/functions/review.ts
Extended ReviewCard type with optional epcIsEstimate field computed from enrichment blob source metadata via new isEpcEstimate helper, enabling client-side distinction between exact certificates and postcode estimates.

UI: EPC Estimate Marker & Address Override

Layer / File(s) Summary
InfoRow: estimate EPC marker
src/components/review/info-row.tsx
Updated InfoRow Props to accept optional epcIsEstimate flag; EPC cell now conditionally renders ~ prefix and "est" unit when estimate, or plain rating otherwise.
ReviewCard: pass epcIsEstimate to InfoRow
src/components/review/review-card.tsx
Thread epcIsEstimate from review card data through to InfoRow component.
Hero page: EPC estimate display
src/routes/index.tsx
Updated buildHeroData to format hero EPC spec with ~{epcRating} when marked as estimate.
DesktopListingDetail: address edit button
src/components/listing-detail/desktop-listing-detail.tsx
Added optional onEditAddress callback prop; wired through InfoColumnPriceCard to render conditional "Edit pinned address" or "Fix address for exact EPC" button based on cluster.userAddress presence.
DesktopListingDetail: estimate EPC display
src/components/listing-detail/desktop-listing-detail.tsx
Extended epcRecordRow to display estimate records with ~ prefix and postcode-level metadata (range, sampleSize).
Listing page: address override dialog & mutation
src/routes/listings/$clusterId.tsx
Added address-entry dialog with input, save/clear controls, and setAddress mutation backing; integrated with desktop onEditAddress callback and mobile action button to allow users to pin exact addresses and trigger EPC re-enrichment.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes


Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

Comment @coderabbitai help to get the list of available commands and usage tips.

@timothygithinji
timothygithinji merged commit eac4e39 into main May 27, 2026
4 checks passed
@timothygithinji
timothygithinji deleted the feat/geo-coords-and-epc branch May 27, 2026 19:15
timothygithinji added a commit that referenced this pull request Jun 6, 2026
…islabelled stops

Two compounding bugs let listings far from any station into the queue:

1. classifyKind mapped Google's generic 'transit_station' type to 'rail'.
   In residential London that type is mostly bus-stop clusters, so bus
   stops were stamped kind='rail' at ~0.03mi with no walkMinutes — and the
   transport filter's straight-line heuristic then read 'rail ~0.6 min
   away', passing the 15-min rule. Treat a bare transit_station as
   unclassified (dropped) instead.

2. clusterPassesSearch judged tube/rail targets on that nearbyTransit data.
   Switch station kinds to the Google Routes walk times in stationRoutes —
   the same source the review card shows (e.g. Bush Hill Park 19 min) — so
   a cluster whose nearest real station is over the limit is now dropped.
   Bus/tram keep nearbyTransit; targets with no data stay pending.

Fix #1 needs a nearby-transit re-enrichment to clean existing rows; fix #2
takes effect immediately for Rightmove-sourced clusters (stationRoutes).
timothygithinji added a commit that referenced this pull request Jun 21, 2026
…islabelled stops

Two compounding bugs let listings far from any station into the queue:

1. classifyKind mapped Google's generic 'transit_station' type to 'rail'.
   In residential London that type is mostly bus-stop clusters, so bus
   stops were stamped kind='rail' at ~0.03mi with no walkMinutes — and the
   transport filter's straight-line heuristic then read 'rail ~0.6 min
   away', passing the 15-min rule. Treat a bare transit_station as
   unclassified (dropped) instead.

2. clusterPassesSearch judged tube/rail targets on that nearbyTransit data.
   Switch station kinds to the Google Routes walk times in stationRoutes —
   the same source the review card shows (e.g. Bush Hill Park 19 min) — so
   a cluster whose nearest real station is over the limit is now dropped.
   Bus/tram keep nearbyTransit; targets with no data stay pending.

Fix #1 needs a nearby-transit re-enrichment to clean existing rows; fix #2
takes effect immediately for Rightmove-sourced clusters (stationRoutes).
timothygithinji added a commit that referenced this pull request Jun 21, 2026
…islabelled stops

Two compounding bugs let listings far from any station into the queue:

1. classifyKind mapped Google's generic 'transit_station' type to 'rail'.
   In residential London that type is mostly bus-stop clusters, so bus
   stops were stamped kind='rail' at ~0.03mi with no walkMinutes — and the
   transport filter's straight-line heuristic then read 'rail ~0.6 min
   away', passing the 15-min rule. Treat a bare transit_station as
   unclassified (dropped) instead.

2. clusterPassesSearch judged tube/rail targets on that nearbyTransit data.
   Switch station kinds to the Google Routes walk times in stationRoutes —
   the same source the review card shows (e.g. Bush Hill Park 19 min) — so
   a cluster whose nearest real station is over the limit is now dropped.
   Bus/tram keep nearbyTransit; targets with no data stay pending.

Fix #1 needs a nearby-transit re-enrichment to clean existing rows; fix #2
takes effect immediately for Rightmove-sourced clusters (stationRoutes).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant