Plans for 3D data consumption: viewing and tagging (parent issue)

[spwoodcock]

Context

• We are producing 3D data products from imagery: textured mesh & DEM/DSM.
• We have a good start for visualising the textured mesh in glTF format from work by @eurobob in Add vendored drone-mesh project as new 3D viewer #859
• However two things we need going forward, described below.

1. City-scale data viewer

a. Textured Mesh

• Currently the glTF approach involves loading the entire model at the start (this can be optimised using low/high res data split, but it's not enough at 100km2).
• From what I can see, the de-facto standard for this is 3D Tiles.
• We can either use our existing viewer, or other products for this, such as Plateau viewer.

b. Ortho + terrain

• This one is pretty standard: we should use MapLibre to drape the ortho imagery over the DEM.
• One optimisation we could probably do is converting the DEM into a terrainium encoded PMTiles.
• The question is, do we build this part in DroneTM, or in OAM (I think pushing to OAM and viewing there is more suitable 3D data ingestion & conversion openaerialmap#273)

2. 3D data tagging

We need a way to tag the 3D data, which the new viewer mentioned above has started using Gaussian splatting.

a. Multi-user collaborative project tagging

• Real-time multi-user tagging across the entire dataset, with an experience similar to Figma or collab editing on Google docs.
• All edits are reflected immediately, preventing duplication.
• Using local-first design, we could probably build a very nice web experience, but for a little more engineering effort.
• Would require 3D tiles approach above to load the whole project mesh.

b. Single user task tagging

• Using the task splitting model of Tasking Manager, where a user assigns themself a single task to work on and 'locks' the task, to prevent duplicated effort.
• The existing glTF can be loaded into the existing viewer and tagging completed by a single user, then saved and pushed somewhere.

Decision factors:

1. How good is the tagging experience on the textured glTF model Vs. a 3D mesh: usability.
2. Performance: does performance differ between the two approaches? Which one works better on low-spec devices and low internet bandwidth?
3. Others?

Plan

• Update this issue after learnings from FOSS4G Hiroshima, and takeaways from Project Plateau.
• Decide or not to implement 3D tiles into the viewer.
• Optimise the existing (currently unused after the new viewer was implemented) 3D tiles generation pipeline. Obj2Tiles is used. Does Plateau have more optimised alternatives?
• Hook up a system for the tagging be sent to.
• Hook up reading any existing tagging into the viewer (for collaboration / reduced duplicated effort)

[linear]

TECH-1276

[spwoodcock]

3DTiles standard (FOSS4G workshop notes):

• 3D tiles wrap glTF with a hierarchy. 3D tiles can reference multiple glTF and wrap up huge datasets. glTF can't reference other glTF.
• glTF 2.1 is being released to bridge the gap, and allow a glTF to reference other glTF - display all the models, without too much geospatial (used by non-geospatial companies).
• Gaussian splatting is coming to 3D tiles - baseline support and standard ratified on 28th Aug. Available in Cesium.
• Also supporting time series + voxels (volumetric data) - not as relevant to us.
• 3DTiles VectorTiles also possible to display 3D data on top.
• Sep 8th submit to OGC. Ratify 3DTiles spec in October.
• CityGML = data behind geometry with labels etc (in contrast to 3D tiles which are simply a rendering?)