Skip to content

fix(epc): one property certified twice is not two properties - #65

Merged
paulieb89 merged 1 commit into
mainfrom
fix/epc-uprn-recertified-property
Sep 4, 2026
Merged

fix(epc): one property certified twice is not two properties#65
paulieb89 merged 1 commit into
mainfrom
fix/epc-uprn-recertified-property

Conversation

@paulieb89

Copy link
Copy Markdown
Owner

Reproduced live

EPCClient().search_by_postcode('NG11 9HD', address='27 Havenwood Rise')
-> EPCAmbiguousMatchError: 2 certificates share the address text
   '27 Havenwood Rise'; cannot select one

Both candidates carry UPRN 100031555077. One house, certified twice, whose
address text differs only by a comma. Properties are re-certified on every sale
and let, so this is the normal case — and every affected property was
unreachable by address, including through enrich_comps_with_epc, where it
silently left comps un-enriched.

Measured on real data, six postcodes

Postcode Distinct addresses Resolvable before After
NG11 9HD 30 12 30
M1 1AE 28 11 28
DE12 6LL 13 10 13
B5 4BX 76 76 76
Total 147 109 (74%) 147 (100%)

B5 4BX is the control — where no property holds more than one certificate,
nothing changes. This is not a blanket relaxation.

Both entry points were broken

The caller-supplied-UPRN branch refused identically (N certificates share UPRN X), so fixing only the address path would have left epc_lookup(uprn=...)
broken for the same properties.

The rule is deliberately narrow

All three must hold:

  • every candidate carries a non-empty UPRN and they all agree — UPRN is
    optional upstream and often absent, so two blanks are not agreement;
  • the candidates agree on canonical address text — a shared UPRN with
    different addresses is contradictory upstream data, not one property, and
    choosing there would be exactly the failure this module exists to prevent;
  • the registration dates order strictly — a tie has no "most recent", and
    resolving one by upstream row order is a defect the module already catalogues.
    Dates get a canonical ISO round-trip, not a string compare
    (ppd_source.validate_date_range carries the scar for that).

The design check

Every existing guard passes unmodified — including
test_duplicate_exact_addresses_remain_ambiguous (uprn=None),
test_duplicate_uprn_is_ambiguous and
test_ambiguous_uprn_does_not_pick_arbitrarily (both pair different addresses
under one UPRN).

That mattered. This module has been repaired four times, each round finding a
new way for partial evidence to look sufficient, and a fix needing those tests
relaxed would have been the fifth. Rule 4 adds no property evidence at all: it
applies only once identity is established to rules 1–3's standard, then chooses
within that one property's own certificate history.

Reported as uprn_latest_certificate at confidence 100 — identity is certain,
and the method name discloses that the newest of several certificates was taken.

Also

The ambiguous counter in enrichment.py was incremented and never read, so
how often selection refused — and therefore what any change to it is worth —
couldn't be measured from anything that function left behind. Now logged with a
per-method breakdown.

Expect epc_match_rate to rise and median_price_per_sqft to move, since
newly-matched comps bring floor areas into the median.

./scripts/validate.sh2140 passed, 28 skipped. Live A/B run against the
real EPC API on the deployed Machine, with production code untouched (patched
module loaded from /tmp).

Reproduced live against the real EPC API:

    EPCClient().search_by_postcode('NG11 9HD', address='27 Havenwood Rise')
    -> EPCAmbiguousMatchError: 2 certificates share the address text
       '27 Havenwood Rise'; cannot select one

Both candidates carry UPRN 100031555077. One house, certified twice, whose
address text differs only by a comma. Properties are re-certified on every sale
and let, so this is the normal case rather than an edge one, and every affected
property was unreachable by address -- including through enrich_comps_with_epc,
where it silently left comps un-enriched and depressed epc_match_rate.

Measured on real data across six postcodes, 147 distinct addresses:

    NG11 9HD   30 addresses   12 resolvable -> 30
    M1 1AE     28 addresses   11 resolvable -> 28
    DE12 6LL   13 addresses   10 resolvable -> 13
    B5 4BX     76 addresses   76 resolvable -> 76   (no duplicates; unchanged)
    TOTAL     147 addresses  109 (74%)     -> 147 (100%)

B5 4BX is the control: where no property holds more than one certificate,
nothing changes. This is not a blanket relaxation.

Both entry points were affected. The caller-supplied-UPRN branch refused
identically ("N certificates share UPRN X"), so fixing only the address path
would have left epc_lookup(uprn=...) broken for the same properties.

The rule is deliberately narrower than "same UPRN wins". All three must hold:

  * every candidate carries a non-empty UPRN and they all agree. UPRN is
    optional upstream and often absent, so two blanks are not agreement.
  * the candidates agree on canonical address text. A shared UPRN with
    DIFFERENT addresses is contradictory upstream data, not one property;
    choosing there would be precisely the failure this module exists to
    prevent -- attaching another property's floor area and every
    price-per-sqft derived from it.
  * the registration dates order strictly. A tie has no "most recent", and
    resolving one by upstream row order is a defect the module docstring
    already catalogues. Dates are parsed with a canonical ISO round trip,
    not compared as strings -- ppd_source.validate_date_range carries the
    scar for that.

Consequently every existing guard passes UNMODIFIED, including
test_duplicate_exact_addresses_remain_ambiguous (uprn=None),
test_duplicate_uprn_is_ambiguous and test_ambiguous_uprn_does_not_pick_arbitrarily
(both pair different addresses under one UPRN). That was the design check: this
module has been repaired four times, each round finding a new way for partial
evidence to look sufficient, and a fix that needed those tests relaxed would be
the fifth. Rule 4 adds no property evidence at all -- it applies only once
identity is established to rules 1-3's standard, then chooses within that one
property's own certificate history.

Reported as method "uprn_latest_certificate" at confidence 100: identity is
certain, and the method name discloses that the newest of several certificates
was taken.

Also wired up the `ambiguous` counter in enrichment.py, which was incremented
and never read -- so how often selection refused, and therefore what any change
to it is worth, could not be measured from anything that function left behind.
Now logged with a per-method breakdown.

Expect epc_match_rate to rise and median_price_per_sqft to move, since
newly-matched comps bring floor areas into the median.

./scripts/validate.sh: 2140 passed, 28 skipped.
@paulieb89
paulieb89 merged commit c9f7403 into main Sep 4, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant