Skip to content

Commit 8aede28

Browse files
authored
feat(overture): add Overture Maps POI integration (Places, off by default) (#73)
Adds optional Overture Maps Places as a first-party data source fused with OpenStreetMap, for commercial POI search and place-card enrichment. Ships OFF by default — the integrations self-disable when unconfigured and the search orchestrator short-circuits to the byte-identical Overpass-only path, so merging changes nothing about current runtime behavior. - Conflation backbone in @openmapx/core: tiered union-find matcher with name-independent corroboration (address keys, wikidata, phone/website signals). - data-manager Overture pipeline: DuckDB-over-S3 extract → PostGIS ingest → H3-blocked conflation → poi_conflation_link, plus monthly changelog deltas, cron, and a data overture-* CLI. - Two integrations, off by default: poi-overture (search fusion) and knowledge-overture (brand/multilingual-name/opening-hours/wikidata enrichment). - poi-search orchestrator: Overpass base + Overture augment, merged via the precomputed link table and a query-time union-find pass. Scope is Places only (CDLA-Permissive). Validated end-to-end on a full Berlin ingest; conflation measured at 0% phone-set contradiction among phone-bearing links.
1 parent 83c3972 commit 8aede28

73 files changed

Lines changed: 7796 additions & 223 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@openmapx/core": minor
3+
---
4+
5+
Add the OSM↔Overture POI conflation toolkit used by the Overture Places integration. Exposes the bipartite `conflate` matcher (`ConflationPoint`/`ConflationThresholds`/`ConflationResult`/`DEFAULT_CONFLATION_THRESHOLDS`), query-time `fusePoiResults`, the Overture category mappings (`overtureCategoryToOpenMapX`, `openMapXCategoryToOverture`, `OVERTURE_COMMERCIAL_CATEGORIES`, `openmapxCategoryToOvertureLeaves`), and the name-independent matching utilities in `geo-server` (`nameSimilarity`, `normalizeName`, `normalizeStreet`, `osmAddressKey`, `overtureAddressKey`, `normalizePhone`, `parsePhones`, `websiteDomain`). Matching corroborates on address keys, wikidata, and phone/website signals beyond fuzzy name similarity.

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ docs/superpowers/
7777
docs/plans/
7878
docs/legal/
7979
plans/
80+
.superpowers/
8081

8182
# CLI generated outputs
8283
infra/docker/docker-compose.generated.yml

apps/api/src/integration-host.ts

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -895,6 +895,26 @@ export function isIntegrationScheme(scheme: string): boolean {
895895
return integrations.has(scheme);
896896
}
897897

898+
/**
899+
* True when `scheme` matches an installed integration that is **not**
900+
* config-disabled (`config.enabled !== false`). The places route uses this
901+
* to decide whether a missing resolver should 404 (the integration owns the
902+
* scheme and is enabled, so the missing resolver is a boot failure) or fall
903+
* through to coord-fallback (the integration is config-disabled, so a stale
904+
* deep-link should degrade gracefully rather than hard-404).
905+
*
906+
* Keyed on `config.enabled`, not the runtime `enabled` flag: both a
907+
* config-disabled integration and an enabled-but-`setup()`-threw integration
908+
* have `enabled === false` at runtime, but only `config.enabled === false`
909+
* uniquely marks config-disabled. Using the runtime flag would let a broken
910+
* enabled integration fall through to coord-fallback, re-opening the
911+
* tag-substitution leak the 404 trap closes.
912+
*/
913+
export function isEnabledIntegrationScheme(scheme: string): boolean {
914+
const i = integrations.get(scheme);
915+
return i !== undefined && i.config?.enabled !== false;
916+
}
917+
898918
export function getIntegrationsByDomain(domain: string): LoadedIntegration[] {
899919
return Array.from(integrations.values()).filter(
900920
(i) => i.enabled && i.manifest.domains.includes(domain),

apps/api/src/routes/__tests__/places.test.ts

Lines changed: 49 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -48,9 +48,11 @@ vi.mock("@integrations/reviews/orchestrator", () => ({
4848
}));
4949

5050
const mockIsIntegrationScheme = vi.fn().mockReturnValue(false);
51+
const mockIsEnabledIntegrationScheme = vi.fn().mockReturnValue(false);
5152
vi.mock("../../integration-host.js", () => ({
5253
getAllIntegrations: vi.fn().mockReturnValue([]),
5354
isIntegrationScheme: (scheme: string) => mockIsIntegrationScheme(scheme),
55+
isEnabledIntegrationScheme: (scheme: string) => mockIsEnabledIntegrationScheme(scheme),
5456
}));
5557

5658
// Mock DB RIS service
@@ -348,14 +350,14 @@ describe("GET /places/:id", () => {
348350
expect(mockLookupByCoords).toHaveBeenCalled();
349351
});
350352

351-
it("returns 404 for an integration scheme whose resolver didn't register", async () => {
353+
it("returns 404 for an enabled integration scheme whose resolver didn't register", async () => {
352354
// Mirrors the failure mode that produced the original leak: a data-
353-
// source integration (here scooter-sharing) is installed — its manifest
354-
// is on disk so `isIntegrationScheme` returns true — but its `setup()`
355-
// threw at boot, so no resolver was registered. Without this gate the
356-
// route would fall through to lookupByCoords and substitute the
357-
// nearest OSM POI's tags onto the scooter.
358-
mockIsIntegrationScheme.mockImplementation((scheme) => scheme === "scooter-sharing");
355+
// source integration (here scooter-sharing) is installed and enabled —
356+
// `isEnabledIntegrationScheme` returns true — but its `setup()` threw at
357+
// boot, so no resolver was registered. Without this gate the route would
358+
// fall through to lookupByCoords and substitute the nearest OSM POI's
359+
// tags onto the scooter.
360+
mockIsEnabledIntegrationScheme.mockImplementation((scheme) => scheme === "scooter-sharing");
359361

360362
const res = await app.inject({
361363
method: "GET",
@@ -370,7 +372,46 @@ describe("GET /places/:id", () => {
370372
expect(mockLookupByNameAndCoords).not.toHaveBeenCalled();
371373
expect(mockLookupByCoords).not.toHaveBeenCalled();
372374

373-
mockIsIntegrationScheme.mockReturnValue(false);
375+
mockIsEnabledIntegrationScheme.mockReturnValue(false);
376+
});
377+
378+
it("coord-falls-back for a config-disabled integration scheme with lat/lng/name", async () => {
379+
// A config-disabled integration's scheme should NOT 404 — the request
380+
// should reach the coord-fallback so a shared link degrades gracefully.
381+
// `isEnabledIntegrationScheme` returns false (config-disabled); no resolver is registered.
382+
mockIsEnabledIntegrationScheme.mockReturnValue(false);
383+
mockLookupByNameAndCoords.mockResolvedValue(MOCK_PLACE);
384+
mockGetPlaceKnowledge.mockResolvedValue({ externalIds: {} });
385+
mockBuildReviewLinks.mockReturnValue([]);
386+
387+
const res = await app.inject({
388+
method: "GET",
389+
url: `/places/${encodeURIComponent("overture:some-gers-id")}?${qs({
390+
lat: "52.52",
391+
lng: "13.37",
392+
name: "Some Place",
393+
})}`,
394+
});
395+
396+
expect(res.statusCode).toBe(200);
397+
expect(mockLookupByNameAndCoords).toHaveBeenCalled();
398+
});
399+
400+
it("returns 404 for an enabled integration scheme with no resolver (no coords supplied)", async () => {
401+
// When no lat/lng are provided AND the scheme belongs to an enabled
402+
// integration whose resolver never registered, the route must 404.
403+
mockIsEnabledIntegrationScheme.mockImplementation((scheme) => scheme === "scooter-sharing");
404+
405+
const res = await app.inject({
406+
method: "GET",
407+
url: `/places/${encodeURIComponent("scooter-sharing:dott-456")}`,
408+
});
409+
410+
expect(res.statusCode).toBe(404);
411+
expect(mockLookupByNameAndCoords).not.toHaveBeenCalled();
412+
expect(mockLookupByCoords).not.toHaveBeenCalled();
413+
414+
mockIsEnabledIntegrationScheme.mockReturnValue(false);
374415
});
375416

376417
it("allows coord-fallback for a non-integration freeform scheme (stylePoi)", async () => {

apps/api/src/routes/places.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ import {
2323
type PlaceResolverContext,
2424
} from "@openmapx/place-ids";
2525
import type { FastifyPluginAsync } from "fastify";
26-
import { getAllIntegrations, isIntegrationScheme } from "../integration-host.js";
26+
import { getAllIntegrations, isEnabledIntegrationScheme } from "../integration-host.js";
2727
import { getPlaceKnowledge } from "../services/knowledge/index";
2828
import { buildReviewLinks } from "../services/review-links";
2929
import { TTL, withCache } from "../utils/cache.js";
@@ -235,7 +235,7 @@ export const placesRoute: FastifyPluginAsync = async (fastify) => {
235235
// never got to register its resolver. The manifest registry
236236
// tells us which is which: any scheme matching an installed
237237
// integration id is strict; everything else is freeform.
238-
if (isIntegrationScheme(parsedId.scheme)) {
238+
if (isEnabledIntegrationScheme(parsedId.scheme)) {
239239
fastify.log.warn(
240240
{ scheme: parsedId.scheme, rawId },
241241
"places: integration scheme has no resolver; refusing coord-fallback",

apps/api/src/services/knowledge/__tests__/index.test.ts

Lines changed: 59 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,14 +61,36 @@ function makePlace(osmTags?: Record<string, string>) {
6161
}
6262

6363
describe("getPlaceKnowledge", () => {
64-
it("returns {} immediately when place has no osmTags", async () => {
64+
it("returns {} immediately when place has neither osmTags nor coordinates", async () => {
6565
const { getPlaceKnowledge } = await import("../index.js");
66-
const result = await getPlaceKnowledge(makePlace(undefined));
66+
const placeWithoutBoth = {
67+
id: "overture:gers-abc",
68+
primaryScheme: "overture",
69+
ids: { overture: "gers-abc" },
70+
name: "Test",
71+
address: "",
72+
coordinates: undefined,
73+
osmTags: undefined,
74+
} as unknown as Parameters<typeof getPlaceKnowledge>[0];
75+
const result = await getPlaceKnowledge(placeWithoutBoth);
6776
expect(result).toEqual({});
6877
expect(wikidataLookup).not.toHaveBeenCalled();
6978
expect(wikipediaLookup).not.toHaveBeenCalled();
7079
});
7180

81+
it("calls sources with empty osmTags when coordinates are present but osmTags absent (Overture brandless)", async () => {
82+
wikidataLookup.mockResolvedValueOnce(null);
83+
wikipediaLookup.mockResolvedValueOnce(null);
84+
const { getPlaceKnowledge } = await import("../index.js");
85+
const result = await getPlaceKnowledge(makePlace(undefined));
86+
expect(result).toEqual({});
87+
expect(wikidataLookup).toHaveBeenCalledWith(
88+
{},
89+
undefined,
90+
expect.objectContaining({ coordinates: [13.4, 52.5] }),
91+
);
92+
});
93+
7294
it("scalar fields: first non-null description wins", async () => {
7395
wikidataLookup.mockResolvedValueOnce({ description: "Wikidata desc" });
7496
wikipediaLookup.mockResolvedValueOnce({ description: "Wikipedia desc" });
@@ -176,4 +198,39 @@ describe("getPlaceKnowledge", () => {
176198
expect(wikidataLookup).not.toHaveBeenCalled();
177199
expect(result.description).toBe("Wikipedia desc");
178200
});
201+
202+
it("optionality: enrichment output is unchanged when knowledge-overture is not registered", async () => {
203+
wikidataLookup.mockReset();
204+
wikipediaLookup.mockReset();
205+
wikidataLookup.mockResolvedValueOnce({ description: "Wikidata desc" });
206+
wikipediaLookup.mockResolvedValueOnce({ description: "Wikipedia desc" });
207+
208+
const { getPlaceKnowledge } = await import("../index.js");
209+
const result = await getPlaceKnowledge(makePlace({ wikidata: "Q42" }));
210+
211+
expect(result.description).toBe("Wikidata desc");
212+
expect(result.brand).toBeUndefined();
213+
expect(result.names).toBeUndefined();
214+
expect(result.structuredOpeningHours).toBeUndefined();
215+
});
216+
217+
it("merges brand, names, structuredOpeningHours from first non-null source", async () => {
218+
wikidataLookup.mockReset();
219+
wikipediaLookup.mockReset();
220+
wikidataLookup.mockResolvedValueOnce({
221+
brand: { name: "Starbucks", wikidata: "Q37158" },
222+
names: { de: "Starbucks" },
223+
structuredOpeningHours: "Mo-Fr 07:00-21:00",
224+
});
225+
wikipediaLookup.mockResolvedValueOnce({
226+
brand: { name: "Other Brand" },
227+
});
228+
229+
const { getPlaceKnowledge } = await import("../index.js");
230+
const result = await getPlaceKnowledge(makePlace({ wikidata: "Q42" }));
231+
232+
expect(result.brand).toEqual({ name: "Starbucks", wikidata: "Q37158" });
233+
expect(result.names).toEqual({ de: "Starbucks" });
234+
expect(result.structuredOpeningHours).toBe("Mo-Fr 07:00-21:00");
235+
});
179236
});

apps/api/src/services/knowledge/index.ts

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,15 +26,16 @@ function getKnowledgeSources(disallowedIntegrations: Set<string>): KnowledgeProv
2626
* Never throws — failures are silently dropped.
2727
*/
2828
export async function getPlaceKnowledge(place: Place, lang?: string): Promise<KnowledgeResult> {
29-
if (!place.osmTags) return {};
29+
if (!place.osmTags && !place.coordinates) return {};
3030

3131
const sources = getKnowledgeSources(await getGatedIntegrationIds());
3232

3333
const settled = await Promise.allSettled(
3434
sources.map((source) =>
35-
source.lookup(place.osmTags as Record<string, string>, lang, {
35+
source.lookup((place.osmTags ?? {}) as Record<string, string>, lang, {
3636
coordinates: place.coordinates,
3737
name: place.name,
38+
ids: place.ids,
3839
}),
3940
),
4041
);
@@ -52,6 +53,9 @@ export async function getPlaceKnowledge(place: Place, lang?: string): Promise<Kn
5253
facts,
5354
externalIds,
5455
airport,
56+
brand,
57+
names,
58+
structuredOpeningHours,
5559
} = result.value;
5660

5761
if (description && !merged.description) merged.description = description;
@@ -66,6 +70,11 @@ export async function getPlaceKnowledge(place: Place, lang?: string): Promise<Kn
6670
merged.externalIds = { ...externalIds, ...(merged.externalIds ?? {}) };
6771
}
6872
if (airport && !merged.airport) merged.airport = airport;
73+
if (brand && !merged.brand) merged.brand = brand;
74+
if (names && !merged.names) merged.names = names;
75+
if (structuredOpeningHours && !merged.structuredOpeningHours) {
76+
merged.structuredOpeningHours = structuredOpeningHours;
77+
}
6978
}
7079

7180
return merged;

0 commit comments

Comments
 (0)