feat(core): render the map as a globe when the style sets projection - #4533
feat(core): render the map as a globe when the style sets projection#4533johncarmack1984 wants to merge 35 commits into
Conversation
|
Downstream sky support PR #4534 is now ready for maintainer review. It is intentionally stacked on this globe work because the physical atmosphere depends on its globe camera matrices, depth pass, and projection-transition state, and it has been dogfooded in a production Expo React Native mobile app. Reviewers can isolate the sky work with the sky-only comparison: ClayWarren/maplibre-native@codex/globe-projection-4533...codex/sky-style-spec-4414 Sharing this as dependency and prioritization context; no timeline is implied. |
Bloaty Results 🐋Compared to main Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results/pr-4533-compared-to-main.txtCompared to d387090 (legacy) Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results/pr-4533-compared-to-legacy.txt |
|
Benchmark Results ⚡ Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/benchmark-results/pr-4533-compared-to-main.txt |
Bloaty Results (iOS) 🐋Compared to main Full report: https://maplibre-native.s3.eu-central-1.amazonaws.com/bloaty-results-ios/pr-4533-compared-to-main.txt |
6bddd11 to
126d38b
Compare
|
First CI run turned up one real failure, on the Device Farm C++ job: Fixed in phase 1: the test reads The branch had also fallen behind
|
126d38b to
7e7280f
Compare
|
For anyone coming from the newsletter: the 30k lines are mostly fixtures.
Where the decisions are:
6 and 9e are the two questions from #4525. The rest is that pattern applied again:
To try it:
|
7e7280f to
aa96607
Compare
|
I am wondering if you have tried merged the terrain 3d branch at #4190 into this at all. It seems some of the things left there, like LOD changes, you have already started to tackle as part of this globe PR I think the LOD changes in terrain might be a cause of at least one of the remaining test failures there. it might be good to reference the TERRAIN.md in that branch in https://github.com/maplibre/maplibre-native/blob/feature/terrain-3d/TERRAIN.md to see how we can combine these |
|
I'm a little worried about expanding scope too much on an this already gargantuan PR, but will look into both to see how they can be better integrated and planned for in unison Edit: oh I see you're heavily involved in the feedback on that branch, I'll see what I can do |
|
@acalcutt Read the LOD section of TERRAIN.md. The second divergence is the one this PR touches, and it goes a step further than the doc says: GL JS puts the Mercator cover on Pushed a commit that moves it out to Merging the branches: not into this PR. Dry-run merge of
GL JS composes that seam with prelude functions, and the globe prelude already defines The bigger gap is the cover loop. GL JS runs one |
5379b97 to
e3e2043
Compare
John Carmack picked this up on #4533 after Andrew pointed him at TERRAIN.md, and his reading of the LOD divergence is sharper than what was written here. Verified his claim against gl-js: the Mercator cover uses the variable-zoom function whenever terrain is set, not only at high pitch - mercator_covering_tiles_details_provider.ts returns `!!options.terrain || transform.pitch > clamp(78.5 - zfov/2, 0, 60)`. So with terrain on, gl-js picks a per-tile zoom from FOV, tile pitch and a tile-count budget on every frame at any pitch, where native's terrain cover uses one constant desired zoom. That is deeper than "native has no pitch term" and it applies to every terrain scene rather than only steep ones. The function is now reusable: globe commit e3e2043 lifts it into src/mln/util/tile_lod.hpp as TileZoomFunction, with elevationForTileCulling alongside it, unit-tested against gl-js's own function to 1e-9. It is still called only inside namespace globe, so wiring computeMeshCover to it touches no globe code. Notably elevationForTileCulling takes a centerElevation, which is exactly what RenderTerrain::getElevationForLatLng already returns. That supersedes any plan to port #5719 from TypeScript ourselves, and it is the next concrete step here. deltaZoom and the centre-clamped camera remain terrain-side. Also recorded: his merge dry run (170 shared files, 93 conflicts, 66 of them the one vertex position line where terrain wraps apply_drape_transform and globe wraps projectTile), the projectTileWithElevation / projectTileFor3D seam the globe prelude already provides, and that converging the two cover loops onto one details-provider is being left as a follow-up. His numbers, not independently checked here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
@johncarmack1984 Thanks for looking into my question. I agree that both PRs are huge and should not officially be merged, but testing it in a seperate branch might be helpful to figure out any remaining crossover. From the diagnosis is seems we have quite a few conflicts Just to note, the AI liked the plan outlined above and used that information to fill in a few gaps. It adds "the extraction is the right shape, and terrain will consume TileZoomFunction and elevationForTileCulling from main once they land — with the note that elevationForTileCulling's centerElevation is already available terrain-side as RenderTerrain::getElevationForLatLn" |
|
It had also suggested separating out the LOD changes to their own PR so they could be merged first and used in both our PRs, but I feel like these branches would be highly scrutinized as a whole so I am not sure how feasible that is |
Globe phase 8 of the design proposal (maplibre#4525): Symbols and collision. Oracle: The collision-*, text-* and icon-text-* globe tests. Consolidates from core/projection-property: - feat(core): render lines on the globe (Metal) - feat(core): render symbols on the globe (Metal) - feat(core): cover the globe with the tiles the camera can see - feat(core): hand the globe over to Mercator at high zoom - refactor(core): review pass over the globe branch - perf(core): project each line-label vertex once per symbol on the globe - fix(core): hide a line label with any vertex behind the globe - fix(core): project neighbour tile borders with the Mercator matrix on the globe - refactor(core): second review pass over the globe branch - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list
Globe phase 9a of the design proposal (maplibre#4525): Remaining layer families on Metal: lines. Oracle: The line globe tests. Consolidates from core/projection-property: - feat(core): render lines on the globe (Metal) - feat(core): render raster, hillshade and color-relief layers on the globe (Metal) - feat(core): render symbols on the globe (Metal) - refactor(core): review pass over the globe branch - fix(core): keep globe line widths in pixels at high latitude - test(core): port GL JS's checks into the globe unit tests - refactor(core): second review pass over the globe branch - test(core): port five more GL JS globe render tests - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list
…phase 9b) Globe phase 9b of the design proposal (maplibre#4525): Remaining layer families on Metal: circle, heatmap. Oracle: The circle and heatmap globe tests. Consolidates from core/projection-property: - feat(core): render circles and heatmaps on the globe (Metal) - feat(core): render raster, hillshade and color-relief layers on the globe (Metal) - feat(core): render symbols on the globe (Metal) - refactor(core): review pass over the globe branch - refactor(core): second review pass over the globe branch - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list
…lobe on Metal (globe phase 9c) Globe phase 9c of the design proposal (maplibre#4525): Remaining layer families on Metal: raster, hillshade, color-relief; the globe tile mesh. Oracle: raster-planet, raster-pole, hillshade, color-relief globe tests. Consolidates from core/projection-property: - feat(core): render raster, hillshade and color-relief layers on the globe (Metal) - feat(core): render fill-extrusions on the globe (Metal) - refactor(core): review pass over the globe branch - feat(core): let the globe zoom out below zero toward the poles - refactor(core): second review pass over the globe branch - test(core): port five more GL JS globe render tests - test(core): port the GL JS globe raster tests - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list - test(core): unignore the globe tests Linux OpenGL renders
… 9d) Globe phase 9d of the design proposal (maplibre#4525): Remaining layer families on Metal: fill-extrusion, with the depth prepass 3D layers need. Oracle: The fill-extrusion globe tests. Consolidates from core/projection-property: - feat(core): render fill-extrusions on the globe (Metal) - feat(core): render symbols on the globe (Metal) - refactor(core): review pass over the globe branch - perf(core): draw the globe depth prepass only when a 3D layer needs it - fix(core): coarsen a subdivided polygon that would overflow a segment instead of dropping the tile - refactor(core): second review pass over the globe branch - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list
… 9e) Globe phase 9e of the design proposal (maplibre#4525): The blend: the globe preset's transition to Mercator between zoom 11 and 12, keeping tiles across the hand-off. Oracle: zoom-transition; Map.GlobeHandOffKeepsTiles. Consolidates from core/projection-property: - feat(core): hand the globe over to Mercator at high zoom - refactor(core): review pass over the globe branch - perf(core): keep tiles across the globe's Mercator hand-off - test(core): port GL JS's checks into the globe unit tests - refactor(core): second review pass over the globe branch - refactor(core): name the projection types - test(core): port five more GL JS globe render tests - test(core): run the globe hand-off test on its own run loop - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list
…hase 9f) Globe phase 9f of the design proposal (maplibre#4525): Stencil clip masks drawn from the globe tile meshes, culled on the far side. Oracle: Every globe test with overlapping tiles; fill-planet-tiles on an iPhone. Consolidates from core/projection-property: - feat(core): clip layers to their tiles on the globe (Metal) - test(core): port five more GL JS globe render tests - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list - fix(core): cull the far side of the globe clip masks
Globe phase 10a of the design proposal (maplibre#4525): OpenGL globe prelude variant, clip masks, uniform-block tables, Linux OpenGL expectations. Oracle: The globe tests on the Linux OpenGL manifest. Consolidates from core/projection-property: - feat(core): render the globe on Vulkan, WebGPU and OpenGL - refactor(core): review pass over the globe branch - feat(core): clip layers to their tiles on the globe (OpenGL) - fix(core): skip uniform blocks a shader variant does not reference on OpenGL - refactor(core): second review pass over the globe branch - test(core): port five more GL JS globe render tests - fix(gl): set the stencil mode of a 3D layer group that has no clip tiles - test(core): render the globe tests on Linux OpenGL - test(core): keep only the tests OpenGL has not rendered in its ignore list - test(core): put the Linux OpenGL expectations where the runner looks - fix(gl): bind the projection block for every shader that projects through it - test(core): unignore the globe tests Linux OpenGL renders - fix(core): cull the far side of the globe clip masks - test(core): rerender the Linux OpenGL expectations without the mask bug - test(core): bundle the Linux OpenGL expectations for the Bazel render test - refactor(gl): drop the duplicate drawable include from the context - build(gl): forward-declare the GL passes before befriending them
Globe phase 10b of the design proposal (maplibre#4525): Vulkan globe prelude variant and clip mask. Oracle: The globe tests on the Vulkan manifests. Consolidates from core/projection-property: - feat(core): render the globe on Vulkan, WebGPU and OpenGL - refactor(core): review pass over the globe branch - refactor(core): second review pass over the globe branch - fix(core): cull the far side of the globe clip masks
Globe phase 10c of the design proposal (maplibre#4525): WebGPU globe prelude variant, clip mask, cull modes. Oracle: The globe tests on the WebGPU manifests (Dawn and wgpu). Consolidates from core/projection-property: - feat(core): render the globe on Vulkan, WebGPU and OpenGL - refactor(core): review pass over the globe branch - refactor(core): second review pass over the globe branch - perf(core): drop the globe's mesh caches when the style goes back to Mercator - fix(webgpu): honour the drawables' cull modes - fix(core): cull the far side of the globe clip masks
… to every shader group (globe phase 10d) Globe phase 10d of the design proposal (maplibre#4525): Phase 10, after the three backend ports: the shader-group lookup takes the projection variant as a required parameter ahead of the attribute name on the base class and on the Metal, OpenGL, Vulkan and WebGPU implementations, and every caller passes it explicitly Oracle: compiles on all four backends; render tests unchanged on every manifest Consolidates from core/projection-property: - refactor(core): second review pass over the globe branch - fix(core): draw the Vulkan globe clip mask with the whole projection block
Globe phase 9g of the design proposal (maplibre#4525): The location indicator layer family on the globe, rebuilt on projection change, quads without an image left undrawn. Oracle: LocationIndicator.{Mercator,Globe}PuckAtItsLocation. Consolidates from core/projection-property: - fix(core): place the location indicator on the globe - test(core): skip the location-indicator tests where the layer is not registered - fix(core): rebuild the location indicator when the projection changes - test(core): skip the globe puck test on OpenGL's own location indicator renderer - fix(core): keep a location indicator quad without an image out of the draw
Globe phase 9h of the design proposal (maplibre#4525): The debug layer's tile borders on the globe. Oracle: debug-tile-borders. Consolidates from core/projection-property: - fix(core): draw tile borders on the globe - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS antimeridian and seam tests - test(core): unignore the globe tests Linux OpenGL renders - test(core): rerender the Linux OpenGL expectations without the mask bug
Globe phase 9i of the design proposal (maplibre#4525): CustomLayerRenderParameters carry the projection; custom drawable layers project on the globe; the depth prepass runs for them. Oracle: custom; test/api/custom_layer.test.cpp. Consolidates from core/projection-property: - feat(core): hand custom layers the globe projection - test(core): port GL JS's checks into the globe unit tests - refactor(core): second review pass over the globe branch - fix(core): project custom drawable layers on the globe - fix(core): run the globe depth prepass for custom layers
Globe phase 11a of the design proposal (maplibre#4525): Platform SDK: the GLFW app. Oracle: The GLFW app by hand. Consolidates from core/projection-property: - feat(glfw): toggle the globe with F9 - refactor(core): second review pass over the globe branch - refactor(core): name the projection types
Globe phase 11b of the design proposal (maplibre#4525): Platform SDK: iOS example app and the iOS Metal expectations an iPhone rasterizes differently. Oracle: iOS render tests on device; the example app. Consolidates from core/projection-property: - test(core): iOS Metal expectations for the globe tests an iPhone rasterizes differently - feat(ios): add a globe style to the example app
Globe phase 11c of the design proposal (maplibre#4525): Platform SDK: Android test app. Oracle: compileOpenglDebugKotlin; the test app. Consolidates from core/projection-property: - feat(android): add a globe example to the test app
Globe phase 12a of the design proposal (maplibre#4525): Corrective tail: layer translation on the sphere. Oracle: The *-translate globe tests. Consolidates from core/projection-property: - refactor(core): second review pass over the globe branch - fix(core): apply *-translate on the globe - test(core): port the GL JS globe raster tests - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list - test(core): unignore the globe tests Linux OpenGL renders - test(core): rerender the Linux OpenGL expectations without the mask bug
…hase 12b) Globe phase 12b of the design proposal (maplibre#4525): Corrective tail: image sources. Oracle: The ported GL JS globe raster tests. Consolidates from core/projection-property: - feat(core): render raster, hillshade and color-relief layers on the globe (Metal) - refactor(core): second review pass over the globe branch - test(core): port five more GL JS globe render tests - fix(core): draw image sources on the globe by their own quad - test(core): port the GL JS globe raster tests - fix(core): draw tile borders on the globe - test(core): render the globe tests on Linux OpenGL - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list - test(core): unignore the globe tests Linux OpenGL renders
Globe phase 12c of the design proposal (maplibre#4525): Corrective tail: collision debug boxes. Oracle: The collision-box globe tests. Consolidates from core/projection-property: - fix(core): draw collision boxes on the globe
…globe phase 12d) Globe phase 12d of the design proposal (maplibre#4525): Corrective tail: camera behaviours. Oracle: The ported GL JS camera unit tests. Consolidates from core/projection-property: - fix(core): ease, fly and fit bounds on the globe the way GL JS does - fix(core): zoom around an anchor near the globe's horizon the way GL JS does
…be phase 12e) Globe phase 12e of the design proposal (maplibre#4525): Corrective tail: the antimeridian and seam tests, with the three residuals ignored for their stated reasons. Oracle: antimeridian-overdraw, fill-seams. Consolidates from core/projection-property: - test(core): port GL JS's checks into the globe unit tests - test(core): port five more GL JS globe render tests - fix(core): draw antimeridian-crossing geometry once on the globe - test(core): port the GL JS antimeridian and seam tests - test(core): port the GL JS globe text tests - test(core): unignore the globe tests Linux OpenGL renders - test(core): ignore the three antimeridian-overdraw residuals with their reasons
… (globe phase 12f) Globe phase 12f of the design proposal (maplibre#4525): Corrective tail: variable anchors and the ported text tests. Oracle: The text-* globe tests. Consolidates from core/projection-property: - test(core): port five more GL JS globe render tests - fix(core): place variable-anchor labels where the globe collides them - test(core): port the GL JS globe text tests - test(core): keep only the tests OpenGL has not rendered in its ignore list - test(core): unignore the globe tests Linux OpenGL renders
…d macOS (globe phase 12g) Globe phase 12g of the design proposal (maplibre#4525): Corrective tail: Map::isLocationOccluded and the SDKs' view-based annotations. Oracle: Transform.GlobeLocationOcclusion; MLNAnnotationViewTests. Consolidates from core/projection-property: - refactor(core): review pass over the globe branch - refactor(core): second review pass over the globe branch - fix(core): treat a coordinate behind the globe as offscreen on iOS and macOS
…timeridian (globe phase 12h) Globe phase 12h of the design proposal (maplibre#4525): Corrective tail: the tile cover's wrap across the antimeridian. Oracle: Transform.GlobeCenterLongitudeStaysContinuous; TileCover.GlobeCoverSurvivesTheAntimeridian. Consolidates from core/projection-property: - fix(core): keep the globe's center longitude continuous across the antimeridian
…over (globe phase 12i) `TileZoomFunction` and `elevationForTileCulling` move out of the globe cover's anonymous namespace into `util/tile_lod.hpp`, unchanged in behavior. They are GL JS `createCalculateTileZoomFunction` and `getElevationForTileCulling`, which GL JS applies to both projections: its Mercator cover switches to them whenever terrain is present or the pitch passes 78.5 - fov/2 degrees. `elevationForTileCulling` gains GL JS's center-elevation and content-elevation terms, both zero on the globe. Oracle: tile_lod.test.cpp, values from the GL JS functions on the same inputs; the globe cover tests are unchanged.
…nded source (globe phase 12j) Just below an integer zoom the globe cover already asks for the next zoom's tiles nearest the camera: GL JS's tile zoom function puts them about 0.07 zoom ahead of the center. A source with TileJSON bounds refused them, because `TilePyramid` capped its `TileRange` at the nominal zoom, `updateRenderables` skips a refused ideal tile with no parent fallback, and the parents those tiles replace stopped being retained. The fill and line layers dropped them while the symbol layers held them for fade, so the map blinked to the background with its labels still up, once per integer zoom crossed, on any bounded source. The demotiles TileJSON has bounds. GL JS checks bounds at the tile's own zoom (`TileBounds.contains`). `TileLodMode::Distance` already sizes the range to the source's maximum zoom for the same reason; the globe cover now does too. Mercator in the default LOD mode is unchanged, and the range costs what it did: two corner projections per update. Reported on Android Vulkan in the Globe activity (maplibre#4533); reproduced on macOS on Metal and Vulkan with mbgl-glfw easing the same style from zoom 5.2 to 6.5, blank between 5.93 and 6.0, none after. Oracle: render test projection/globe/source-bounds (the checkerboard just below zoom 7 from a bounded source, empty before); Map.GlobeBoundedSourceLoadsFinerTiles (no zoom 11 tile parsed before).
9b4ce9e to
fa131c9
Compare
Five things had gone stale or unrecorded: - The extraction is no longer "called only inside namespace globe" as far as this branch is concerned - it is cherry-picked onto feature/tile-lod and merged here. Records that the merge is temporary, must not reach a submitted branch, and is one revert to undo. - The 2026-09-07 plan is superseded. It assumed splitting tile_lod out would look like slipping it past review; the globe PR's author would in fact prefer that over vendoring, and is holding it back only so reviewers are not asked which PR to read first. - `centerElevation` matching `getElevationForLatLng` was asserted here without checking the units. Both sides exaggerate, so they compose - noted with the exact call, because getting it wrong would have been visible only at pitch near the horizon. - The tile zoom function does not decide whether it becomes a third LOD mode or replaces TileLodMode::Distance on Mercator. That is ours to answer before wiring the cover. - The bounded-source trap behind #4533's zoom-6 flicker applies to us: our DEM sources declare a maxzoom, and the function asks for tiles finer than the nominal zoom just below an integer zoom. Also records that deltaZoom was measured and does not cause the ancestor requests - identical miss lists with and without - and corrects an earlier "8 vs 11" note that was a truncated pipe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes #3161.
"projection": {"type": "globe"}renders the map as a globe on Metal, Vulkan, WebGPU and OpenGL, with GL JS's math and GL JS's render tests as the oracle. This PR is the twelve phases of #4525 as 32 commits, one per phase (9 and 10 are one commit per family/backend; 10d is the review's shader-group signature unification). Every commit builds and passes its own tests alone.The same commits are open as a stack of draft PRs on my fork, chained by base, so each phase can be read as one diff: https://github.com/johncarmack1984/maplibre-native/pulls — #3 to #34, the map is in #3. Rebase-merge would keep the phase commits on main. If you'd rather take phases one at a time, each fork PR can be re-targeted here in order.
Checked: on every commit, the Metal unit tests and the globe render group, plus a Linux OpenGL (Docker) run of the whole render suite, a Bazel build and an MSVC (Windows ARM64) build; the fork's CI on every phase ref, Windows judged against main's own baseline. At the tip: Metal 1362 passed / 79 ignored / 0 failed on the full suite, unit tests 915/915; Linux OpenGL in Docker 1364 passed with the same three environment failures main shows there; Dawn and wgpu 1373 / 86 / 0 on the branch before the instancing merge, and the globe group 59 / 4 / 0 at the tip. @ClayWarren has been running the branch offline in a production Expo/React Native app; the only gap they hit was atmosphere, which is the separate
skyproperty (#4414).AI Disclosure: Claude Fable 5 was used to orchestrate these changes under John's plan and direction. Claude Opus 4.6 was used to implement, with Fable and John ensuring the changes in every commit were written according to sound CS principles and internal repo conventions. CI has been run on every commit to ensure these changes create no regressions. In my review and testing, I found them to be well-written and to introduce no new issues. Open to feedback and happy to implement any changes deemed necessary. -John