feat: render_map_tool — single visualization primitive (server-side refs)#199
Open
mattpodwysocki wants to merge 25 commits into
Open
feat: render_map_tool — single visualization primitive (server-side refs)#199mattpodwysocki wants to merge 25 commits into
mattpodwysocki wants to merge 25 commits into
Conversation
Adds a single generic MCP App resource (ui://mapbox/map-app/index.html)
that any tool can point _meta.ui.resourceUri at. Tools emit a
MapAppPayload — a thin pass-through to Mapbox Style spec paint/layout
objects — and the shared iframe translates each layer/marker/legend
entry into the corresponding GL JS calls. No per-tool HTML template
required.
What's added:
- src/utils/mapAppPayload.ts — payload TypeScript types and a
polyline decoder (precision 5 + 6 fallback) so the iframe never has
to handle encoded geometry.
- src/resources/ui-apps/mapAppHtml.ts — generic HTML renderer that
reads _meta.ui.payload from the postMessage'd tool result OR a baked-in
initial-data block, then renders layers/markers/legend with proper
teardown on re-render.
- src/resources/ui-apps/MapAppUIResource.ts — BaseResource wrapper.
- Registered in resourceRegistry.ts.
DirectionsTool migration:
- Now builds a MapAppPayload (route line + start/end badge markers +
summary chip) rather than producing its own HTML.
- meta.ui.resourceUri points to the new generic resource.
- _meta.ui.payload carries the payload for the MCP Apps path.
- The same payload is baked into the inline MCP-UI rawHtml block via
renderMapAppHtml({ initialData }) — one source of truth.
- The old DirectionsAppUIResource stays registered for back-compat but
the tool no longer references it.
Tests:
- MapAppUIResource serves HTML with public token + workerDomains CSP.
- decodePolyline + decodePolylineWithFallback against the canonical
Google Polyline reference encoding.
- DirectionsTool test now asserts the generic resource URI and the
_meta.ui.payload shape (layers[0].id='route', markers=[start,end]).
733 tests passing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The MCP Apps host doesn't forward CallToolResult._meta through
ui/notifications/tool-result in practice, so the iframe was rendering
its "no payload" error. structuredContent is the guaranteed-delivery
field for tool result data, so attach _mapApp there instead.
- DirectionsResponseSchema now uses .passthrough() so the MCP SDK's
output-schema validation doesn't strip the _mapApp key.
- DirectionsTool returns structuredContent: { ...validatedData, _mapApp }
when a payload exists.
- _meta.ui.payload is kept as a belt-and-suspenders fallback for hosts
that DO forward _meta (matches the MCP Apps spec contract).
- Iframe reads structuredContent._mapApp first, falls back to
_meta.ui.payload.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Surfaces which keys are present on the result object so we can see whether structuredContent / _meta / content survived the host bridge. Temporary diagnostic until we confirm the delivery path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
DC -> Baltimore is >50KB so it went through the temp-resource branch, which built a stripped structuredContent that never included _mapApp. The iframe correctly reported "structuredContent keys: routes, waypoints" — _mapApp was simply absent. Build the MapAppPayload from the full validated response before the size check, attach it to both small and large response paths, and set _meta.ui.payload alongside on both as well. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…confirmed DC -> Baltimore renders correctly via structuredContent._mapApp. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Each of the following now emits a MapAppPayload via structuredContent. _mapApp (and _meta.ui.payload) instead of producing its own HTML — the shared MapAppUIResource renders all of them. - isochrone_tool: fill+line layer pairs per contour, origin marker - optimization_tool: trip line + numbered visit markers (start green, end red, middle blue), polyline decoded tool-side - search_and_geocode_tool + category_search_tool: shared buildSearchMapPayload helper, numbered orange pins, search-center reference marker - map_matching_tool: dashed orange raw trace + solid blue matched route, legend - ground_location_tool: origin pin + numbered POI pins - union/intersect/difference polygon-ops: shared buildPolygonOpsMapPayload with operation-keyed result color (green/purple/orange) and an Inputs+Result legend. Offline tools use MAPBOX_PUBLIC_TOKEN directly (no httpRequest in scope) for the inline MCP-UI path. Each tool's output schema now uses .passthrough() so the MCP SDK output validation doesn't strip _mapApp. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…D destination='last' The Mapbox V1 Optimization API returns "NotImplemented: This request is not supported" when roundtrip=false is paired with source='any' or destination='any'. The LLM was retrying with various combos (source: 'first', destination: 'any', etc) and all failed at the API. Convert this into a precise Zod schema refine before the network call: - roundtrip !== false || (source === 'first' && destination === 'last') - Surface the constraint in the source/destination/roundtrip field descriptions so the LLM picks valid combos on first try. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude Desktop and other MCP clients without form elicitation support hit this catch path on every search_and_geocode_tool call. Logging at 'warning' caused the host UI to flag the call as failed (red triangle) even though the tool returned isError: false with full results. This is expected fallback behavior, not a warning — drop to 'debug'. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… flag a tool call as failed Claude Desktop appears to mark tool calls as "Tool execution failed" in its UI when the tool emits any notifications/message after the request, even at debug level. The elicitation try/catch was hitting this on every call in clients without form-elicitation support. Drop the log entirely. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… schema descriptions
search_and_geocode_tool emitted three info-level notifications/message
on every call ("Starting search", "Fetching from URL", "Successfully
completed"). Claude Desktop's UI flags any tool emitting notifications
as visually failed even when the JSON-RPC response is isError: false.
No other tool in this repo emits these — drop them.
optimization_tool's source/destination/roundtrip descriptions said
"when roundtrip=false this MUST be 'first/last'" but the LLM still
picked source="any" destination="last" on the first try, hitting the
schema refine and requiring a retry. Restate the constraint up-front
in each field's description with explicit "REQUIRES" / "REJECTS"
language so the LLM picks valid args first time.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Claude Desktop's UI appears to require a non-empty layers[] for it to open the map-app iframe — with layers:[] (markers only) the card stays collapsed and shows a misleading "Failed to call tool" badge even when the JSON-RPC response is isError: false. Add a translucent orange circle feature for each result. The numbered markers still sit on top; the circle layer just gives Claude Desktop a layers[] entry to satisfy whatever check it's doing. Matches the optimization/isochrone/directions payload shape. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…alls
When an LLM chains two map-rendering tools in one chat (e.g.,
search_and_geocode_tool -> optimization_tool, or search ->
category_search), both result cards previously pointed at the SAME
ui://mapbox/map-app/index.html. Claude Desktop deduplicates iframes by
URI — only the latest tool's iframe renders, and the earlier card
collapses with a misleading "Failed to call tool" badge.
Give each tool its own URI under the same map-app namespace:
ui://mapbox/map-app/{flavor}/index.html
where flavor is one of: directions, isochrone, optimization, search,
category-search, map-matching, ground-location, polygon-ops.
All flavors share:
- the same renderMapAppHtml() module
- the same MapAppUIResource class (parameterized by flavor)
- the same MapAppPayload schema
Only the URI differs, so the host opens a fresh iframe per tool.
The code stays flat — N URI registrations is config, not duplication.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Refactors MCP App rendering from "every map-producing tool declares its own iframe" to "one render_map_tool, called terminally". Solves Claude Desktop's chain-position rendering quirk where intermediate tools in a chain show as failed because the host only fully renders the LAST tool's iframe. Architecture B: - render_map_tool is the ONLY tool that declares meta.ui.resourceUri. It accepts a MapAppPayload and renders it via a single shared MapAppUIResource (ui://mapbox/map-app/index.html). - Every other geo tool (directions, isochrone, optimization, search_and_geocode, category_search, map_matching, ground_location, union/intersect/difference) returns a ready-to-render `_mapApp` payload on its structuredContent. The LLM passes it through. - Tool descriptions teach the LLM: "Call render_map_tool with the _mapApp field after gathering geospatial data." - Composable: the LLM can merge payloads from multiple data tools and render them on one map. What's preserved from the prior generic-map-app work: - renderMapAppHtml — the shared iframe template - MapAppPayload — the wire format - All per-tool payload builders (buildDirectionsMapPayload, etc.) - Polyline decoding tool-side via decodePolyline/decodePolylineWithFallback What's removed: - Per-tool meta.ui.resourceUri declarations (10 tools) - Per-tool inline createUIResource emissions - DirectionsAppUIResource + directionsAppHtml (now redundant) - MAP_APP_FLAVORS scheme (workaround abandoned in favor of single URI) 737 tests passing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Before: each data tool returned its full MapAppPayload inline as
structuredContent._mapApp. The LLM had to copy that payload as input
to render_map_tool. For a directions response or isochrone with detailed
geometry, that's 5-50KB streamed token-by-token by the LLM — 20-30s of
latency, and the polygon-ops chain (geocode × 2 + isochrone × 2 + union
+ render) tripped Anthropic's API timeout.
After: each data tool stashes its MapAppPayload in temporaryResourceManager
(same store directions/isochrone already use for >50KB responses) and
returns a short ref:
structuredContent._mapApp = { ref: "mapbox://temp/map-payload-<uuid>" }
The LLM passes that ref to render_map_tool via payload_refs[]. The tool
dereferences server-side and renders. Multiple refs are merged into one
map — colliding layer IDs auto-suffix, summaries join with " · ",
legends concatenate.
The LLM emission shrinks from kilobytes of coordinates to a single URI
per dataset, eliminating the latency wall and the timeout risk on big
chains. Tools that need to compose payloads from raw GeoJSON can still
use inline `layers`/`markers`/`legend` fields on render_map_tool.
Touches all 10 data tools to swap inline payload for ref. render_map_tool
gains assemblePayload() which resolves refs, merges, and applies inline
overrides for summary/camera/legend.
3 new tests cover: ref resolution, multi-ref merge, layer-id collision
handling. Updated directions test to assert the ref shape and verify
round-trip via resolveMapPayloadRef.
739 tests passing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two LLM-facing fixes from the polygon-ops test screenshots:
1. **renderHint** in text output. Sonnet was hallucinating
`mapbox-isochrone-tool-result://0` instead of reading the actual
`_mapApp.ref` URI from structuredContent. The fix: every data tool
now appends a one-line render hint to its text output containing
the literal ref string and the exact tool-call shape:
📍 To show this on a live Mapbox GL JS map, call:
render_map_tool({ "payload_refs": ["mapbox://temp/map-payload-..."] })
That way the LLM literally sees the URI it needs to pass and the
exact call signature, instead of inferring either.
2. **Tighten polygon-op descriptions**. The union_tool failure showed
the LLM was misshapen the nested polygon coordinates. Update
PolygonSchema and each polygon-op tool description to include:
- The exact 3-level nesting (polygon → rings → coords)
- A concrete example value
- An explicit "if you have a Feature with type=Polygon, pass
feature.geometry.coordinates directly" instruction
- Guidance for chaining with isochrone_tool (use polygons=true, skip
MultiPolygons or unpack them)
Also bump ring schema to require min(4) coords (proper closed-ring
GeoJSON validity) and improve its description.
Category-search skips the hint when format=json_string (would break
caller's JSON.parse). Isochrone test updated to use .toContain because
the body now has a trailing hint.
739 tests passing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…tMessage timeouts) For a 238-mile route the merged MapAppPayload is ~300KB. Putting that inline in structuredContent didn't round-trip reliably through Claude Desktop's postMessage bridge — the iframe showed "Tool result did not contain a map payload" even though render_map_tool succeeded. Fix: render_map_tool now stashes the merged payload server-side via storeMapPayload and surfaces only the ref in structuredContent._mapApp. The iframe, on receiving the tool result, sees the ref, calls `resources/read` against TemporaryDataResource (which already serves `mapbox://temp/*`), gets the full payload, and renders. The flow stays the same from the LLM's perspective — it still passes refs into render_map_tool. The optimization is entirely on the host→iframe side: structuredContent stays small, the heavyweight payload travels through the resource-read channel that's designed for large blobs. Touches: RenderMapTool (store + return ref) and mapAppHtml (dereference ref before render, keep inline path as backwards-compat fallback). 739 tests passing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Even though every tool's output schema uses .passthrough() (which allows
additional properties in JSON Schema via additionalProperties: {}), some
clients appear to strictly validate tool results against the published
schema and silently flag responses with undeclared fields as failed in
their UI. Notable example: Claude Desktop showing "Tool execution
failed" on search_and_geocode_tool even when the JSON-RPC returned
isError: false — leading the LLM to fall back to a different MCP
server's geocoder even though the Mapbox response was valid.
Add a shared MapAppRefSchema and declare `_mapApp` (optional) on:
- SearchBoxResponseSchema (search_and_geocode_tool)
- CategorySearchResponseSchema (category_search_tool)
- DirectionsResponseSchema (directions_tool)
- IsochroneResponseSchema (isochrone_tool)
- OptimizationOutputSchema (optimization_tool)
- MapMatchingOutputSchema (map_matching_tool)
- GroundLocationOutputSchema (ground_location_tool)
- UnionOutputSchema, IntersectOutputSchema, DifferenceOutputSchema
Now the field is explicitly part of each schema's JSON Schema
representation, so even strict additionalProperties: false validators
will accept it. .passthrough() stays in place for forward compatibility
with API additions, but the load-bearing piece is the explicit
declaration.
739 tests passing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Same root cause as 3277cf5: strict client-side validators may strip undeclared fields from structuredContent even with .passthrough() on the Zod side. Without _mapApp surfacing on render_map_tool's result, the iframe can't read the ref and shows "Tool result did not contain a map payload." Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Underscore-prefixed keys at the top level of structuredContent appear to
be stripped by Claude Desktop when forwarding to the iframe via
postMessage — likely treated as "internal" fields (MCP reserves `_meta`
explicitly, other underscore keys aren't blessed).
After the previous fix declaring `_mapApp` on every output schema,
search/geocode stopped showing the failed badge but render_map_tool's
iframe still couldn't see the payload ref. The field reaches the server's
JSON-RPC layer cleanly (verified in logs) but never makes it into the
iframe's `structuredContent`.
Renaming to `mapboxRender` (camelCase, no leading underscore) sidesteps
the convention. Same shape: `{ ref: string }`.
Updated:
- MapAppRefSchema description (no longer ref to "_mapApp")
- All 10 data-tool output schemas
- RenderMapOutputSchema
- All 10 data-tool .ts files that assigned the field
- mapAppHtml.ts iframe extractor
- renderHint text generator
- Render tool's own description and storeMapPayload references
Tests updated to use new field name. 739 passing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ails Adds back the diagnostic so we can see what keys actually arrived at the iframe. If `mapboxRender` is missing despite being on the server response, that confirms host-side stripping. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…uredContent) Diagnostic dump from the iframe revealed Claude Desktop forwards only content and isError in ui/notifications/tool-result postMessages to MCP App iframes — structuredContent is dropped entirely. Workaround: render_map_tool adds a sentinel-tagged text item to content[] with the merged-payload ref. The iframe scans content text items for the sentinel and extracts the ref URI. The structuredContent path is kept as a fallback for spec-compliant hosts. 739 tests passing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ame stalls If the iframe is hanging without showing an error, the loading text now narrates which step it reached: tool-result received, ref extracted, fetching, parsing, rendering. Tells us whether resources/read is the hang point or something earlier. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Each item in content[] now shows its type plus a short preview (first 80 chars of text, or resource URI). Tells us whether the sentinel-tagged text item is being stripped or transformed by the host. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…content[]
Diagnostic revealed Claude Desktop replaces content[] entirely with
placeholder text ("Tool result too large for context, stored at
/mnt/user-data/...") when the response exceeds its size threshold. The
inline rawHtml resource was ~160KB (iframe HTML + script + initial-data
payload) — well over that threshold — and was getting our actual content
stripped along with it.
Claude Desktop already opens the MCP App iframe from meta.ui.resourceUri,
so the inline rawHtml was redundant for that host. Dropping it:
- Brings the render_map_tool response down to a few hundred bytes
(text + sentinel-tagged ref text).
- Lets the sentinel ref survive Claude Desktop's content trimming.
- The iframe extracts the ref and fetches the merged payload from
TemporaryDataResource via resources/read.
Trade-off: MCP-UI-only hosts (those that don't speak MCP Apps) no
longer get an inline rendered map. Acceptable for this iteration — the
demo target is Claude Desktop, and we can add an opt-in inline rawHtml
path later if needed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
For each map-emitting tool (isochrone, optimization, search_and_geocode, category_search, map_matching, ground_location, union, intersect, difference), add a test that: - Asserts structuredContent.mapboxRender.ref is set to a mapbox://temp/map-payload-... URI - Asserts the renderHint text in the response references the same ref - Resolves the ref via resolveMapPayloadRef and verifies the stored payload's layers/markers/legend match the tool's expected output shape (route line for directions/optimization, fill+line per contour for isochrone, dashed+solid for map matching, pin+numbered for search/ground-location, operation-keyed legend for polygon ops) Drops the MCP-UI test scaffolding from StaticMapImageTool and the config tests (we no longer support MCP-UI fallback — render_map_tool is the single visualization primitive). Removes @mcp-ui/server dependency, isMcpUiEnabled config, ENABLE_MCP_UI env var, and --disable-mcp-ui flag. 741 tests passing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced Jun 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Architecture B: data tools + one visualization primitive
Refactors MCP App rendering. Instead of every map-producing tool declaring its own iframe, there's now one tool —
render_map_tool— that takes aMapAppPayloadand renders it. Every other geo tool stores the payload server-side and returns a small ref onstructuredContent; the LLM passes refs through.Why: MCP App hosts (Claude Desktop today) only fully render the iframe for the last tool in a chained sequence. Intermediate tools render as failed cards even when their JSON-RPC returned
isError: false. Funneling all rendering through one terminal tool sidesteps this —render_map_toolis always last by design.The payload format
Thin pass-through to Mapbox Style spec — anything expressible in GL JS is expressible in the payload.
Flow
Data-tool calls are pure data exchanges. Only the terminal
render_map_toolopens an iframe. The iframe fetches the merged payload viaresources/read mapbox://temp/map-payload-<uuid>and draws it.Why server-side refs
Initial implementation passed the full payload inline in the tool result. Two problems surfaced:
content[]with"Tool result too large for context". The iframe shipped, but it had nothing to render.Server-side refs fix both: the payload lives on the server keyed by UUID, the tool result carries only the URI, and the iframe reads it back via standard
resources/read. Total roundtrip is a handful of KB.Why the sentinel text
Claude Desktop strips
structuredContententirely from the postMessage it forwards to MCP App iframes (we verified this with a diagnostic dump). It also deduplicates iframes by URI across a chained call sequence, only fully rendering the last one.So in addition to setting
structuredContent.mapboxRender.ref, each data tool also embeds the ref incontent[0].texttagged with a sentinel[[MAPBOX_RENDER_REF]] <uri>. The iframe extracts via either path —structuredContentfor spec-compliant hosts, sentinel regex for Claude Desktop.The
render_map_tooldescription tells the LLM to call it at the end of any chain with the collected refs.What landed
render_map_tool(src/tools/render-map-tool/). Takespayload_refs: string[], optional inline overrides, renders the merged result.mapAppHtml.ts. Renders anyMapAppPayload. Reads ref fromstructuredContentor content-sentinel, thenresources/read.MapAppUIResourceatui://mapbox/map-app/index.html. Targeted only byrender_map_tool'smeta.ui.resourceUri.storeMapPayload(payload)→mapbox://temp/map-payload-<uuid>with TTL.resolveMapPayloadRef(uri)reads back.mergeMapPayloads([...])for compositing.renderHint(ref)to text.What's removed
@mcp-ui/serverdependency and all inlinecreateUIResourceemissionsenableMcpUiconfig,ENABLE_MCP_UIenv var,--disable-mcp-uiflagmeta.ui.resourceUrifrom every data tool (onlyrender_map_tooldeclares it now)DirectionsAppUIResource+directionsAppHtml.tsMAP_APP_FLAVORSURI-disambiguation scheme (single URI now, funneled through render tool)Customer story
"Mapbox MCP gives you geo data primitives + one visualization primitive. Compose them like LEGO."
Other devs building MCP servers with their own logistics, telemetry, or sensor tools can plug into
render_map_tooldirectly — they emit aMapAppPayload, the Mapbox renderer draws it.Test plan
npm test)structuredContent.mapboxRender.refis set, render hint references the ref in text output, andresolveMapPayloadRef()returns the expected layers/markers/legend shape.🤖 Generated with Claude Code