diff --git a/docs/api-guide/engine/anari-rendering.md b/docs/api-guide/engine/anari-rendering.md index adc9298ff1..00a691c8bd 100644 --- a/docs/api-guide/engine/anari-rendering.md +++ b/docs/api-guide/engine/anari-rendering.md @@ -702,12 +702,13 @@ WebGPU/WebGL selection, a renderer selector for frame presentation, automatic HD available, orbit controls, validation feedback, and live instance, draw-call, and triangle statistics. -Use **GLTF ↓** or **USD ↓** to download the currently valid retained scene. Export bakes procedural -geometry, starfield distributions, and retained instances into a static snapshot; transfers mesh -positions, normals, UVs, vertex colors, materials, texture images, camera, and supported lights; -and emits standalone JSON glTF with embedded buffers/images or ASCII `.usda`. ANARI animation -declarations, optional renderer presets, bloom, fog, and renderer-only HDR controls remain -ANARI-specific and are not exported. +Use **GLTF ↓** or **USD ↓** to download the currently valid retained scene. Both paths bake +procedural geometry, starfield distributions, and retained instances; transfer mesh positions, +normals, UVs, vertex colors, materials, texture images, cameras, and supported lights; and emit +standalone JSON glTF with embedded resources or ASCII `.usda`. glTF additionally preserves +compatible source hierarchies, skeletal skins, morph targets, transform and morph animations, and +supported material animation pointers. ASCII USD export remains static. Renderer presets, bloom, +fog, and renderer-only HDR controls remain ANARI-specific. :::caution[Experimental playground format] The JSON format and its optional schema exports are experimental. They are not part of the ANARI C diff --git a/docs/api-reference/README.md b/docs/api-reference/README.md index af5e1f7c7a..896c847710 100644 --- a/docs/api-reference/README.md +++ b/docs/api-reference/README.md @@ -23,7 +23,7 @@ If you are looking for `Model`, start with [`@luma.gl/engine`][engine]. The `Mod | [`@luma.gl/text`][text] | Experimental / Private | `TextRenderer` facade and caller-owned GPU text data. | | [`@luma.gl/splats`][splats] | Experimental / Private | Gaussian splat rendering and caller-owned prepared GPU splat data. | | [`@luma.gl/experimental`][experimental] | Experimental / Private | Experimental v10 APIs, including WebGPU/WebGL WebXR and WebGL raw camera helpers. | -| [`@luma.gl/gltf`][gltf] | Optional | Standards-first glTF assets, physical materials, character animation, and lossless interchange. | +| [`@luma.gl/gltf`][gltf] | Optional | Standards-native glTF/GLB assets, physical materials, character animation, and source-faithful interchange. | | [`@luma.gl/test-utils`][test-utils] | Optional | Test setups, in particular support for rendering and comparing images. | \* At least one backend, either WebGL or WebGPU, must be installed to enable GPU resource creation. @@ -40,7 +40,7 @@ If you are looking for `Model`, start with [`@luma.gl/engine`][engine]. The `Mod - [`@luma.gl/text`][text] for `TextRenderer` and GPU text data; use [`@luma.gl/arrow`][arrow] for Arrow conversion. - [`@luma.gl/splats`][splats] for experimental Gaussian splat rendering and caller-owned prepared GPU data. - [`@luma.gl/experimental`][experimental] for v10 work-in-progress APIs, including experimental WebXR frame, view, and raw camera helpers. -- [`@luma.gl/gltf`][gltf] for standards-first glTF assets, physical materials, character animation, native extensions, and source-faithful `.gltf`/`.glb` interchange. +- [`@luma.gl/gltf`][gltf] for standards-native glTF/GLB scenegraphs, physical materials, automatic skeletal/morph animation, material variants, instancing, visibility, and source-faithful descriptor interchange. - [`@luma.gl/webgl`][webgl] and [`@luma.gl/webgpu`][webgpu] for backend adapters used by `@luma.gl/core`. - [`@luma.gl/webgl/constants`](/docs/api-reference/webgl/constants) when you need raw numeric WebGL enums. diff --git a/docs/api-reference/anari/anari-animation.md b/docs/api-reference/anari/anari-animation.md index cacdbd8757..61ee4a34aa 100644 --- a/docs/api-reference/anari/anari-animation.md +++ b/docs/api-reference/anari/anari-animation.md @@ -167,10 +167,13 @@ The showcase importer preserves morph `POSITION`, `NORMAL`, and `TANGENT` deltas node/mesh weights, and node-local geometry identities. Retained node weight animation updates the existing mesh vertex data without rebuilding its model. -Source `JOINTS_0` and `WEIGHTS_0` attributes are also preserved, and programmatic surfaces can -use `skin: {jointMatrices}`. The showcase importer does not currently create or animate that joint -palette automatically; complete imported skeletal playback requires application-provided palette -integration. See [ANARI arrays and geometry](/docs/api-reference/anari/anari-geometry). +Source `JOINTS_0` and `WEIGHTS_0` attributes, authored joint-node hierarchies, and optional inverse +bind matrices are preserved. The showcase importer automatically creates mesh-local retained joint +palettes and updates them after source skeletal clips advance; each changed surface is committed at +most once per frame. Applications can also supply an explicit `skin: {jointMatrices}` descriptor +when they own a programmatic palette. Skinning remains in the existing shared shader and generic +engine deformation helpers, not in a separate ANARI implementation. See +[ANARI arrays and geometry](/docs/api-reference/anari/anari-geometry). ## Validation diff --git a/docs/api-reference/anari/anari-geometry.md b/docs/api-reference/anari/anari-geometry.md index 6814054aa5..dcd1ce47a3 100644 --- a/docs/api-reference/anari/anari-geometry.md +++ b/docs/api-reference/anari/anari-geometry.md @@ -206,9 +206,11 @@ const surface = anariDevice.newSurface({ `jointMatrices` is a `Float32Array` or numeric array containing column-major joint matrices. The existing shared skinning module currently supports up to 64 joints. The glTF showcase importer -preserves source joint indices and converts normalized integer `WEIGHTS_0` values to floats, but -does not automatically create or animate the surface joint palette; applications must supply and -update that palette explicitly. +preserves source joint indices, converts normalized integer `WEIGHTS_0` values to floats, and +automatically creates mesh-local joint palettes from source joints and inverse bind matrices. +Imported skeletal clips refresh those palettes after each animation frame. Explicit +`skin: {jointMatrices}` remains available for programmatic surfaces whose application supplies and +updates its own palette. ### Morph targets diff --git a/docs/api-reference/gltf/README.md b/docs/api-reference/gltf/README.md index 49059ae5df..277f622b38 100644 --- a/docs/api-reference/gltf/README.md +++ b/docs/api-reference/gltf/README.md @@ -13,9 +13,10 @@ import {GLTFExample} from '@site/src/examples'; `@luma.gl/gltf` is a standards-first asset runtime for physically based materials, animated -characters, morph deformation, native animation pointers, and source-faithful `.gltf` / `.glb` -interchange across WebGPU and WebGL. It turns postprocessed glTF assets into ordinary luma.gl -scenegraphs and exports generic scene descriptors without depending on a particular renderer. +characters, automatic skeletal deformation, morph targets, material variants, GPU instancing, +recursive node visibility, and typed animation pointers across WebGPU and WebGL. It turns +postprocessed glTF assets into ordinary luma.gl scenegraphs and exports source-faithful `.gltf` / +`.glb` scene descriptors without taking ownership of a renderer. File loading, decompression, and glTF postprocessing belong to `@loaders.gl/gltf`; animation primitives, geometry, and shader modules remain in their existing luma.gl packages. @@ -43,7 +44,8 @@ const gltf = postProcessGLTF(asset); const scenegraphs = createScenegraphsFromGLTF(device, gltf, { useTangents: true, - useByteColors: false + useByteColors: false, + strictExtensions: true }); for (const scene of scenegraphs.scenes) { @@ -56,6 +58,9 @@ function renderFrame(timeMilliseconds: number): void { } requestAnimationFrame(renderFrame); + +// When the asset is no longer needed, release every model and owned GPU resource. +window.addEventListener('pagehide', () => scenegraphs.destroy(), {once: true}); ``` `GLTFAnimator.setTime()` takes an absolute clock value in **milliseconds**. The shared @@ -77,15 +82,20 @@ bundle contains: | --- | --- | | `scenes` | One `@luma.gl/engine` `GroupNode` root per source scene. | | `materials` | Shared engine materials in source glTF material order. | +| `variants` | Source-authored `KHR_materials_variants` controller with selection and reset. | +| `cameras` | Independent source camera definitions updated by supported animation pointers. | | `animator` | A `GLTFAnimator` backed by the shared engine animation mixer. | | `animations` | Decoded source clips, including supported animation-pointer channels. | +| `skins` | Automatically updated mesh-local skin palettes and source joint bindings. | | `lights` | World-space directional, point, and spot lights from `KHR_lights_punctual`. | | `extensionSupport` | A map describing support for extensions reported by the asset. | | `sceneBounds` | World-space bounds and camera-framing recommendations for each scene. | | `modelBounds` | Combined world-space bounds for the complete asset. | | `gltfNodeIdToNodeMap`, `gltfNodeIndexToNodeMap` | Source-node lookup tables for application integration. | | `gltfMeshIdToNodeMap` | Source-mesh lookup table. | +| `resourceCache` | Asset-owned or explicitly shared GPU texture and material-specialization cache. | | `gltf` | The original postprocessed glTF document. | +| `destroy()`, `dispose()` | Idempotent release of every scene-owned model, material, and GPU buffer. | Each bounds object contains `bounds`, `center`, `size`, `radius`, and `recommendedOrbitDistance`. @@ -100,6 +110,8 @@ type ParseGLTFOptions = { lights?: boolean; useTangents?: boolean; useByteColors?: boolean; + strictExtensions?: boolean; + resourceCache?: GLTFAssetResourceCache; }; ``` @@ -112,14 +124,52 @@ type ParseGLTFOptions = { tangent data. - `useByteColors: false` keeps authored punctual-light colors in the linear `[0, 1]` range. The default preserves luma.gl's legacy byte-style light-color convention. +- `strictExtensions: true` rejects required glTF extensions that do not have a complete supported + runtime path. Optional unsupported extensions remain visible in `extensionSupport`. +- `resourceCache` shares compatible asset textures and parsed material specializations across + independently owned scenegraphs. Omit it to give the returned scenegraph an automatic private + cache. + +Import the complete public option and return types directly from the package root: + +```ts +import type {GLTFScenegraphs, ParseGLTFOptions} from '@luma.gl/gltf'; +``` Parsed CPU geometry retains source semantics such as `POSITION`, `NORMAL`, `TANGENT`, `COLOR_0`, `TEXCOORD_0`, `TEXCOORD_1`, `JOINTS_0`, and `WEIGHTS_0`. Shader-facing attribute names are resolved only at model boundaries. +## Source-native extension controls + +The returned scenegraph preserves standards-native asset controls rather than converting them into +application-specific material or scene formats: + +```ts +for (const name of scenegraphs.variants.names) { + console.log(name); +} + +scenegraphs.variants.selectVariant('Midnight'); +scenegraphs.variants.resetVariant(); + +const camera = scenegraphs.cameras[0]; +const sourceNode = scenegraphs.gltfNodeIndexToNodeMap.get(0); +const sourceSkin = scenegraphs.skins.getBinding(0); +``` + +`EXT_mesh_gpu_instancing` creates one real instanced draw per source primitive; use +`getGLTFNodeInstancing()` to inspect its authored transforms and custom attributes. +`KHR_node_visibility` recursively suppresses hidden geometry and punctual lights. Supported +`KHR_animation_pointer` targets update node transforms, morph weights, visibility, material +factors, UV transforms, camera projections, and light properties. + +See [native glTF extensions](/docs/api-reference/gltf/gltf-native-extensions) for complete +selection examples, support guarantees, and structural-material limitations. + ## Materials, textures, and lights -The canonical PBR path preserves all 17 supported core and extension texture slots, authored +The canonical PBR path preserves all registered canonical material texture slots, authored sampler addressing/filtering, generated or supplied mipmaps, per-slot UV transforms, secondary UV coordinates, advanced material factors, alpha modes, and punctual lights. @@ -140,12 +190,54 @@ See [glTF animation and deformation](/docs/api-reference/gltf/gltf-animation), t [engine animation guide](/docs/api-guide/engine/animation), and [glTF extension support](/docs/api-reference/gltf/gltf-extensions) for details and limitations. +## Asset lifetime and shared GPU resources + +Every scenegraph owns its generated models, materials, geometry buffers, and instance buffers. +Generated source-image textures are cached by image identity, authored sampler settings, and +color/data interpretation. Identical images are uploaded once; differently wrapped, filtered, or +interpreted textures stay independent. + +```ts +const scenegraphs = createScenegraphsFromGLTF(device, gltf); + +// Safe to call multiple times; `dispose()` is an equivalent alias. +scenegraphs.destroy(); +scenegraphs.dispose(); +``` + +Scene destruction also releases hidden nodes and detached source-mesh templates. It never destroys +the application-owned `Device`, original source images, or caller-supplied image-based-lighting +environment textures. + +### Share textures between independent scenegraphs + +Create an explicit cache when independently animated scenegraphs should reuse immutable source +textures without sharing mutable materials, joint palettes, or scene nodes: + +```ts +import {createScenegraphsFromGLTF, GLTFAssetResourceCache} from '@luma.gl/gltf'; + +const resourceCache = new GLTFAssetResourceCache(device); +const first = createScenegraphsFromGLTF(device, gltf, {resourceCache}); +const second = createScenegraphsFromGLTF(device, gltf, {resourceCache}); + +console.log(resourceCache.textureCount, resourceCache.materialCount, resourceCache.referenceCount); + +first.destroy(); +second.destroy(); +resourceCache.destroy(); +``` + +A caller-supplied cache remains caller-owned after its final scenegraph is destroyed. If +`resourceCache.destroy()` is called while scenes still exist, destruction is safely deferred until +the last scene releases it. A cache can only be used with its original device. + ## Source-faithful asset interchange `exportGLTF()` serializes renderer-independent glTF scene descriptors as embedded JSON or binary GLB. Existing hierarchy, skins, inverse bind matrices, morph targets, animation clips, material pointers, variants, GPU instancing, cameras, punctual lights, sampler settings, and authored -physical materials remain available to the output asset. +physical materials remain available to the output asset when present in the supplied descriptor. ```ts import {exportGLTF, type GLTFExportScene} from '@luma.gl/gltf'; @@ -156,7 +248,9 @@ const binary: ArrayBuffer = exportGLTF(scene, {binary: true}); ``` See [glTF asset interchange](/docs/api-reference/gltf/gltf-interchange) for typed descriptors, -RGBA vertex colors, normalized joint weights, animation pointers, and resource ownership. +RGBA vertex colors, normalized joint weights, animation pointers, and resource ownership. Export +consumes a `GLTFExportScene` descriptor; it is not an automatic byte-for-byte repackager for an +arbitrary loaded asset or its original external-file layout. ## Package ownership @@ -164,5 +258,6 @@ RGBA vertex colors, normalized joint weights, animation pointers, and resource o - `@luma.gl/gltf` interprets glTF-specific scene, material, sampler, light, and animation data. - `@luma.gl/engine` owns generic scenegraph, animation, and morph-target primitives. - `@luma.gl/shadertools` owns the shared PBR, lighting, and skinning shader modules. +- `@luma.gl/experimental` owns optional high-level physical rendering and captured-scene refraction. - `@luma.gl/anari/gltf`, when explicitly imported, adapts decoded glTF data to retained ANARI objects without making the core ANARI entry point a glTF loader. diff --git a/docs/capabilities.mdx b/docs/capabilities.mdx index 459bf1a4ed..2585540e6e 100644 --- a/docs/capabilities.mdx +++ b/docs/capabilities.mdx @@ -411,11 +411,14 @@ Try [Lightstorm Megacity](/examples/showcase/lightstorm-megacity), | Authored normals and tangents | Evolving | WebGPU + WebGL2 | `@luma.gl/gltf` | Preserve supported vertex attributes; complete renderer-to-renderer fidelity varies. | | Punctual-light parsing | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Parse supported authored directional, point, and spot light definitions. | | Shared animation clips | Available | WebGPU + WebGL2 | `@luma.gl/engine` | Play, blend, crossfade, loop, and interpolate compatible imported tracks. | -| Selected animation pointers | Evolving | WebGPU + WebGL2 | `@luma.gl/gltf` | Update supported `KHR_animation_pointer` transforms, material factors, and UV properties. | +| Selected animation pointers | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Update supported `KHR_animation_pointer` transforms, morph weights, visibility, material factors, UV transforms, cameras, and punctual lights. | | Existing joint-driven skinning | Available | WebGPU + WebGL2 | `@luma.gl/shadertools` | Reuse established skin shaders and automatically bind mesh-local glTF joint palettes. | | Morph-target animation | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Animate POSITION, NORMAL, and TANGENT morph targets through shared glTF, engine, and retained-scene paths. | -| Imported GPU instancing | Opportunity | WebGPU + WebGL2 | `@luma.gl/gltf` | `EXT_mesh_gpu_instancing` is not yet translated into retained instance batches. | -| Imported node visibility | Opportunity | WebGPU + WebGL2 | `@luma.gl/gltf` | `KHR_node_visibility` does not yet have a supported runtime integration. | +| Imported GPU instancing | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | `EXT_mesh_gpu_instancing` creates one real instanced draw per canonical source primitive. | +| Imported node visibility | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | `KHR_node_visibility` recursively controls canonical scenegraph geometry, punctual lights, and supported STEP animation. | +| Source material variants | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Select and reset compatible `KHR_materials_variants` without replacing source scenegraph nodes. | +| Shared asset GPU resources | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Deduplicate compatible image uploads, reuse feature-specialized materials, and explicitly destroy owned scene resources. | +| Animated glTF and GLB interchange | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Export source-faithful scene descriptors with supported animation, skins, morph targets, materials, lights, and extensions. | The ownership boundaries are intentional: `@luma.gl/gltf` interprets assets, `@luma.gl/shadertools` owns shared shading, `@luma.gl/engine` owns generic animation, @@ -438,7 +441,7 @@ See the [glTF extension support matrix](/docs/api-reference/gltf/gltf-extensions | Retained animation playback | Experimental | WebGPU + WebGL2 | `@luma.gl/anari` | Play supported transform, material, and texture-coordinate animations. | | Editable JSON scenes | Experimental | WebGPU + WebGL2 | `@luma.gl/anari` | Inspect and edit supported retained scene descriptions interactively. | | Experimental OpenUSD import | Experimental | WebGPU + WebGL2 | `@luma.gl/anari` | The ANARI Playground imports supported ASCII USD and ASCII-root USDZ examples. | -| Scene-level skeletal animation | Opportunity | WebGPU + WebGL2 | `@luma.gl/anari` | Existing joint skinning has not yet been integrated into the retained ANARI renderer. | +| Scene-level skeletal animation | Experimental | WebGPU + WebGL2 | `@luma.gl/anari` | Imported glTF joint hierarchies automatically update retained mesh-local palettes through the existing shared skinning module. | This is an ANARI-inspired proof of concept, not a claim of complete standards conformance. Explore the [ANARI Playground](/examples/experimental/anari-playground). diff --git a/docs/whats-new.md b/docs/whats-new.md index b2e7b99282..6cb7c21661 100644 --- a/docs/whats-new.md +++ b/docs/whats-new.md @@ -254,7 +254,8 @@ Release Date: March 31, 2026 - **`linear` texture filtering** - default texture filtering is now `linear` instead of `nearest` for improved texture rendering. - **PBR material extensions** - the stock `pbrMaterial` shader now implements `KHR_materials_specular`, `KHR_materials_ior`, `KHR_materials_transmission`, `KHR_materials_volume`, `KHR_materials_clearcoat`, `KHR_materials_sheen`, `KHR_materials_iridescence`, and `KHR_materials_anisotropy`, using the parsed glTF extension uniforms and textures. - **Emissive materials** - the stock PBR shader now applies `KHR_materials_emissive_strength`, and core `emissiveFactor` values are preserved even when no emissive texture is present. -- **`KHR_animation_pointer`** - `createScenegraphsFromGLTF()` and `GLTFAnimator` now animate supported node TRS targets, selected material factors, and animated `KHR_texture_transform` offset/rotation/scale targets on stock PBR material texture slots. Structural targets such as animated `texCoord`, morph weights, cameras, and material mode switches remain unsupported. +- **`KHR_animation_pointer`** - `createScenegraphsFromGLTF()` and `GLTFAnimator` animate supported node transforms, morph weights, recursive visibility, material factors, `KHR_texture_transform` offset/rotation/scale, camera projections, and punctual-light properties. Structural targets such as animated `texCoord` and material mode switches remain unsupported. +- **Asset-owned GPU resources** - Source image uploads and feature-compatible material parsing are shared through `GLTFAssetResourceCache`; scenegraphs explicitly release owned models, materials, hidden nodes, and instancing buffers with idempotent `destroy()` / `dispose()` methods while preserving caller-owned image-based-lighting resources. - **Extension support docs** - the [`glTF Extension Support`](/docs/api-reference/gltf/gltf-extensions) table now documents the current built-in vs parsed-only extension coverage for `@luma.gl/gltf`. - **Extension support metadata** - `createScenegraphsFromGLTF()` now exposes an `extensionSupport` map so applications can inspect which extensions a model uses and whether `@luma.gl/gltf` supports them. diff --git a/modules/anari/README.md b/modules/anari/README.md index 10c9654f7b..98d717ee99 100644 --- a/modules/anari/README.md +++ b/modules/anari/README.md @@ -179,10 +179,12 @@ and real point lights following orbiting satellites. A Monaco editor provides sy schema-aware completion, property descriptions, and exact error indicators. Live edits preserve the last valid scene when JSON, parameter values, or retained object references are invalid. -The **GLTF ↓** and **USD ↓** actions export the currently valid scene as a static interchange -snapshot. Procedural meshes, starfields, and retained instances are baked into glTF 2.0 or ASCII -USD meshes with materials, textures, camera, and supported lights. ANARI animations and optional -renderer preset parameters for bloom, fog, and HDR presentation stay in the editable ANARI JSON. +The **GLTF ↓** and **USD ↓** actions export the currently valid scene to glTF 2.0 or ASCII USD. +Procedural meshes, starfields, and retained instances are baked into portable geometry with +materials, textures, cameras, and supported lights. glTF export additionally preserves supported +source hierarchies, skin bindings, morph targets, compatible animation clips, and material pointers. +USD export remains a static snapshot; renderer presets, bloom, fog, and HDR presentation controls +stay ANARI-specific. The optional, experimental `@luma.gl/anari/schemas` entry point exports Zod schemas and the generated draft-07 JSON Schema without adding Zod to imports of the core ANARI rendering API: @@ -238,7 +240,8 @@ preserved. The showcase automatically creates mesh-local surface joint palettes imported skeletal clips play; each changed retained surface is committed at most once per frame. Programmatic surfaces can also supply an explicit `skin: {jointMatrices}` descriptor when the application owns the palette. -Animated glTF export remains unsupported; interchange export is currently static. +glTF interchange exports compatible transform, morph-weight, and material animation channels; +unsupported renderer-only effects and ASCII USD exports remain static. The format loader lives under `examples/showcase/anari/usd-loader` and follows the loaders.gl loader contract so it can eventually move into a dedicated `@loaders.gl/usd` module. It currently diff --git a/modules/anari/test/gltf-review-light-import.node.spec.ts b/modules/anari/test/gltf-review-light-import.node.spec.ts index ddab90bef7..8a3830bf32 100644 --- a/modules/anari/test/gltf-review-light-import.node.spec.ts +++ b/modules/anari/test/gltf-review-light-import.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {readFile} from 'node:fs/promises'; import {parse} from '@loaders.gl/core'; diff --git a/modules/anari/test/scene-interchange.node.spec.ts b/modules/anari/test/scene-interchange.node.spec.ts index aa75b5acb0..ea930df639 100644 --- a/modules/anari/test/scene-interchange.node.spec.ts +++ b/modules/anari/test/scene-interchange.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {readFile} from 'node:fs/promises'; import {parse} from '@loaders.gl/core'; diff --git a/modules/engine/test/animation/morph-review-correctness.node.spec.ts b/modules/engine/test/animation/morph-review-correctness.node.spec.ts index f381d1e755..457e717f0e 100644 --- a/modules/engine/test/animation/morph-review-correctness.node.spec.ts +++ b/modules/engine/test/animation/morph-review-correctness.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import { decodeMorphTargetAttribute, diff --git a/modules/engine/test/scenegraph/scenegraph-visibility.node.spec.ts b/modules/engine/test/scenegraph/scenegraph-visibility.node.spec.ts index ec0ca9473c..f25dcd85b7 100644 --- a/modules/engine/test/scenegraph/scenegraph-visibility.node.spec.ts +++ b/modules/engine/test/scenegraph/scenegraph-visibility.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {GroupNode, ScenegraphNode} from '@luma.gl/engine'; import {describe, expect, test} from 'vitest'; diff --git a/modules/experimental/src/lugraph/lu-graph-breadth-first-search-internals.ts b/modules/experimental/src/lugraph/lu-graph-breadth-first-search-internals.ts index 9c262785a9..184c7e3115 100644 --- a/modules/experimental/src/lugraph/lu-graph-breadth-first-search-internals.ts +++ b/modules/experimental/src/lugraph/lu-graph-breadth-first-search-internals.ts @@ -1,6 +1,7 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors +// SPDX-FileComment: Independently implemented for WebGPU; inspired by NVIDIA RAPIDS cuGraph. import {type Binding} from '@luma.gl/core'; import {Computation} from '@luma.gl/engine'; diff --git a/modules/experimental/src/lugraph/lu-graph-breadth-first-search.ts b/modules/experimental/src/lugraph/lu-graph-breadth-first-search.ts index 0ef0ea378c..b9b5c84184 100644 --- a/modules/experimental/src/lugraph/lu-graph-breadth-first-search.ts +++ b/modules/experimental/src/lugraph/lu-graph-breadth-first-search.ts @@ -1,6 +1,7 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors +// SPDX-FileComment: Independently implemented for WebGPU; inspired by NVIDIA RAPIDS cuGraph. import type {Buffer} from '@luma.gl/core'; import {DynamicBuffer} from '@luma.gl/engine'; diff --git a/modules/experimental/test/engine/scene-review-deformation.node.spec.ts b/modules/experimental/test/engine/scene-review-deformation.node.spec.ts index 3d02fa213f..2336a517f2 100644 --- a/modules/experimental/test/engine/scene-review-deformation.node.spec.ts +++ b/modules/experimental/test/engine/scene-review-deformation.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {Geometry} from '@luma.gl/engine'; import {SceneRenderer, type SceneRenderOptions, type SceneSurface} from '@luma.gl/experimental'; diff --git a/modules/experimental/test/lugraph/lu-graph-breadth-first-search.node.spec.ts b/modules/experimental/test/lugraph/lu-graph-breadth-first-search.node.spec.ts index 5d2141f8b3..df9e135e0f 100644 --- a/modules/experimental/test/lugraph/lu-graph-breadth-first-search.node.spec.ts +++ b/modules/experimental/test/lugraph/lu-graph-breadth-first-search.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {Buffer} from '@luma.gl/core'; import {DynamicBuffer} from '@luma.gl/engine'; diff --git a/modules/experimental/test/lugraph/lu-graph-breadth-first-search.spec.ts b/modules/experimental/test/lugraph/lu-graph-breadth-first-search.spec.ts index 6b4d647e82..b0938b323b 100644 --- a/modules/experimental/test/lugraph/lu-graph-breadth-first-search.spec.ts +++ b/modules/experimental/test/lugraph/lu-graph-breadth-first-search.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {Buffer, type Device} from '@luma.gl/core'; import {GPUCommandGraph} from '@luma.gl/experimental'; diff --git a/modules/gltf/README.md b/modules/gltf/README.md index 92e56a9c20..95e1a8ce42 100644 --- a/modules/gltf/README.md +++ b/modules/gltf/README.md @@ -8,8 +8,9 @@ # @luma.gl/gltf -Standards-first glTF assets with physically based materials, animated skeletons, morph deformation, -native animation pointers, and source-faithful `.gltf` / `.glb` interchange on WebGPU and WebGL. +Standards-first glTF assets with physically based materials, automatic skeletal and morph +deformation, material variants, GPU instancing, recursive visibility, typed animation pointers, +and source-faithful descriptor-based `.gltf` / `.glb` interchange on WebGPU and WebGL. ```ts import {load} from '@loaders.gl/core'; @@ -19,23 +20,45 @@ import {createScenegraphsFromGLTF, exportGLTF} from '@luma.gl/gltf'; const asset = await load('/models/model.glb', GLTFLoader); const gltf = postProcessGLTF(asset); -const {scenes, animator, lights, modelBounds} = createScenegraphsFromGLTF(device, gltf, { +const scenegraphs = createScenegraphsFromGLTF(device, gltf, { useTangents: true, - useByteColors: false + useByteColors: false, + strictExtensions: true }); +const {scenes, animator, lights, cameras, variants, skins, modelBounds} = scenegraphs; animator.setTime(performance.now()); +if (variants.names.includes('Midnight')) { + variants.selectVariant('Midnight'); +} + const exportScene = {name: 'Exported scene', nodes: [{name: 'Root'}]}; const gltfDocument = exportGLTF(exportScene); const binaryAsset = exportGLTF(exportScene, {binary: true}); + +// Release generated models, materials, source-image textures, and instance buffers. +scenegraphs.destroy(); ``` -The module preserves all 17 supported core/PBR-extension texture slots, sampler filters and +The module preserves all registered canonical material texture slots, sampler filters and mipmaps, independent UV transforms and `TEXCOORD_1`, authored punctual lights, node transforms, -material/texture animation pointers, skin attributes, and animated morph targets. Generic +material/texture/camera/light/visibility animation pointers, automatic mesh-local joint palettes, +and animated morph targets. Generic `exportGLTF()` descriptors also preserve hierarchy, skins, inverse bind matrices, morph targets, -animation clips, variants, GPU instancing, cameras, lights, and source-faithful accessors. +animation clips, variants, GPU instancing, cameras, lights, and source-faithful accessors. Export +operates on supplied descriptors; it does not recreate an arbitrary source asset's original +external-file packaging. + +Repeated source images are cached by authored sampler settings and color/data interpretation. +Scenegraphs own their generated models, materials, buffers, and private texture caches. Supply +`new GLTFAssetResourceCache(device)` as `resourceCache` to share image uploads between independent +scenegraphs; explicitly destroy caller-owned caches after releasing their scenegraphs. Borrowed +image-based-lighting environments and the application device are never destroyed by an asset. + +Standalone glTF scenegraph drawing uses its established transmission fallback. The optional shared +`@luma.gl/experimental` scene renderer and retained ANARI facade provide captured-scene physical +refraction; no renderer or additional BRDF is implemented in `@luma.gl/gltf`. `@loaders.gl/gltf` owns asset loading and decompression. Generic animation, scenegraph, and morph primitives remain in `@luma.gl/engine`; shared PBR, lighting, and skinning shaders remain in @@ -43,6 +66,7 @@ primitives remain in `@luma.gl/engine`; shared PBR, lighting, and skinning shade - [glTF API overview](https://luma.gl/docs/api-reference/gltf) - [Materials, textures, and lighting](https://luma.gl/docs/api-reference/gltf/gltf-materials) +- [Native extensions, material variants, visibility, and instancing](https://luma.gl/docs/api-reference/gltf/gltf-native-extensions) - [Animation and deformation](https://luma.gl/docs/api-reference/gltf/gltf-animation) - [Asset interchange and GLB export](https://luma.gl/docs/api-reference/gltf/gltf-interchange) - [glTF extension support](https://luma.gl/docs/api-reference/gltf/gltf-extensions) diff --git a/modules/gltf/package.json b/modules/gltf/package.json index 57df7bc771..c249bb2fb8 100644 --- a/modules/gltf/package.json +++ b/modules/gltf/package.json @@ -1,7 +1,7 @@ { "name": "@luma.gl/gltf", "version": "9.4.0-alpha.4", - "description": "Standards-first glTF rendering, physical materials, animation, and lossless asset interchange", + "description": "Standards-native glTF scenegraphs, physical materials, animation, and source-faithful asset interchange", "type": "module", "license": "MIT", "repository": { diff --git a/modules/gltf/src/export/gltf-exporter.ts b/modules/gltf/src/export/gltf-exporter.ts index 03a08e60c3..81dcf48655 100644 --- a/modules/gltf/src/export/gltf-exporter.ts +++ b/modules/gltf/src/export/gltf-exporter.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors type GLTFExportTypedArray = | Int8Array diff --git a/modules/gltf/src/gltf/create-scenegraph-from-gltf.ts b/modules/gltf/src/gltf/create-scenegraph-from-gltf.ts index 1efda3f322..f26cdd489b 100644 --- a/modules/gltf/src/gltf/create-scenegraph-from-gltf.ts +++ b/modules/gltf/src/gltf/create-scenegraph-from-gltf.ts @@ -2,8 +2,8 @@ // SPDX-License-Identifier: MIT // SPDX-FileCopyrightText: Copyright (c) vis.gl contributors -import {Device} from '@luma.gl/core'; -import {GroupNode, Material} from '@luma.gl/engine'; +import {assert, Device} from '@luma.gl/core'; +import {GroupNode, Material, ModelNode} from '@luma.gl/engine'; import {GLTFPostprocessed} from '@loaders.gl/gltf'; import {Light} from '@luma.gl/shadertools'; import {parseGLTF, type ParseGLTFOptions} from '../parsers/parse-gltf'; @@ -18,6 +18,7 @@ import { type GLTFExtensionSupport } from './gltf-extension-support'; import {GLTFMaterialVariants} from './gltf-material-variants'; +import {GLTFAssetResourceCache} from './gltf-resource-cache'; export type GLTFScenegraphBounds = { /** World-space axis-aligned bounds for the scene or model. */ @@ -65,8 +66,17 @@ export type GLTFScenegraphs = { /** Automatically updated source skin bindings and reusable mesh-local joint palettes. */ skins: GLTFSkinController; + /** Asset-owned or caller-supplied cache for shared textures and parsed materials. */ + resourceCache: GLTFAssetResourceCache; + /** Original post-processed glTF document. */ gltf: GLTFPostprocessed; + + /** Releases all owned models, materials, buffers, and asset-managed textures exactly once. */ + destroy(): void; + + /** Alias for destroy(). */ + dispose(): void; }; /** Converts a post-processed glTF asset into luma.gl scenegraph nodes and animation helpers. */ @@ -79,8 +89,14 @@ export function createScenegraphsFromGLTF( assertSupportedGLTFExtensions(gltf); } + const ownsResourceCache = !options?.resourceCache; + const resourceCache = options?.resourceCache || new GLTFAssetResourceCache(device); + // A GPU resource cache is valid only on the device that created its textures. + assert(resourceCache.device === device); + resourceCache.retain(); + const {scenes, materials, gltfMeshIdToNodeMap, gltfNodeIdToNodeMap, gltfNodeIndexToNodeMap} = - parseGLTF(device, gltf, options); + parseGLTF(device, gltf, {...options, resourceCache}); const animations = parseGLTFAnimations(gltf); const sourceLights = @@ -127,6 +143,47 @@ export function createScenegraphsFromGLTF( const skins = new GLTFSkinController({gltf, scenes, gltfNodeIndexToNodeMap}); animator.setUpdateHandler(() => skins.update()); + let destroyed = false; + const destroy = (): void => { + if (destroyed) { + return; + } + destroyed = true; + + const sourceGroups = new Set([ + ...scenes, + ...gltfMeshIdToNodeMap.values(), + ...gltfNodeIdToNodeMap.values() + ]); + const modelNodes = new Set(); + const ownedMaterials = new Set(materials); + for (const group of sourceGroups) { + group.preorderTraversal(node => { + if (node instanceof ModelNode) { + modelNodes.add(node); + if (node.model?.material) { + ownedMaterials.add(node.model.material); + } + } + }); + } + + for (const modelNode of modelNodes) { + modelNode.destroy(); + } + for (const group of sourceGroups) { + group.destroy(); + } + for (const material of ownedMaterials) { + material.destroy(); + } + + if (ownsResourceCache) { + resourceCache.destroy(); + } + resourceCache.release(); + }; + return { scenes, materials, @@ -142,7 +199,10 @@ export function createScenegraphsFromGLTF( gltfNodeIdToNodeMap, gltfNodeIndexToNodeMap, skins, - gltf + resourceCache, + gltf, + destroy, + dispose: destroy }; } diff --git a/modules/gltf/src/gltf/gltf-instancing.ts b/modules/gltf/src/gltf/gltf-instancing.ts index 10de26883c..4752907e11 100644 --- a/modules/gltf/src/gltf/gltf-instancing.ts +++ b/modules/gltf/src/gltf/gltf-instancing.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import type {GLTFNodePostprocessed, GLTFPostprocessed} from '@loaders.gl/gltf'; import {Matrix4} from '@math.gl/core'; diff --git a/modules/gltf/src/gltf/gltf-material-variants.ts b/modules/gltf/src/gltf/gltf-material-variants.ts index f7e73ee64d..ad1c39cfdc 100644 --- a/modules/gltf/src/gltf/gltf-material-variants.ts +++ b/modules/gltf/src/gltf/gltf-material-variants.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import type {RenderPipelineParameters} from '@luma.gl/core'; import {GroupNode, Material, ModelNode} from '@luma.gl/engine'; diff --git a/modules/gltf/src/gltf/gltf-resource-cache.ts b/modules/gltf/src/gltf/gltf-resource-cache.ts new file mode 100644 index 0000000000..b7ce496a10 --- /dev/null +++ b/modules/gltf/src/gltf/gltf-resource-cache.ts @@ -0,0 +1,174 @@ +// luma.gl +// SPDX-License-Identifier: MIT +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors + +import {assert, type Device, type SamplerProps, type Texture} from '@luma.gl/core'; +import type {ParsedPBRMaterial} from '../pbr/pbr-material'; + +/** Texture interpretation used to safely reuse one source image across compatible materials. */ +export type GLTFResourceTextureOptions = { + sampler: SamplerProps; + colorSpace: 'srgb' | 'linear'; + role: 'color' | 'data'; +}; + +/** + * Owns reusable GPU textures and immutable material specializations for glTF assets on one device. + * + * Scenegraphs retain the cache while they reference its resources. Explicit destruction is deferred + * until every retained scenegraph has been destroyed, so shared scene instances remain independent. + */ +export class GLTFAssetResourceCache { + readonly device: Device; + + private readonly textures = new Map>(); + private readonly materialSpecializations = new Map>(); + private readonly objectIdentifiers = new Map(); + private retainedScenegraphs = 0; + private destructionRequested = false; + private isDestroyed = false; + + constructor(device: Device) { + this.device = device; + } + + /** Number of GPU texture objects currently owned by this cache. */ + get textureCount(): number { + let textureCount = 0; + for (const textures of this.textures.values()) { + textureCount += textures.size; + } + return textureCount; + } + + /** Number of source-material and shader-layout combinations currently cached. */ + get materialCount(): number { + let materialCount = 0; + for (const specializations of this.materialSpecializations.values()) { + materialCount += specializations.size; + } + return materialCount; + } + + /** Number of live scenegraphs currently borrowing these shared resources. */ + get referenceCount(): number { + return this.retainedScenegraphs; + } + + /** Whether all owned resources have been released. */ + get destroyed(): boolean { + return this.isDestroyed; + } + + /** @internal Keeps owned resources alive for one scenegraph. */ + retain(): void { + // A destroyed cache cannot be reused to build another scenegraph. + assert(!this.isDestroyed && !this.destructionRequested); + this.retainedScenegraphs++; + } + + /** @internal Releases one scenegraph without destroying caller-owned shared resources. */ + release(): void { + if (this.retainedScenegraphs === 0) { + return; + } + this.retainedScenegraphs--; + if (this.retainedScenegraphs === 0 && this.destructionRequested) { + this.destroyResources(); + } + } + + /** + * Destroys owned textures once all scenegraphs have released them. + * Caller-owned environments and image objects are never destroyed. + */ + destroy(): void { + if (this.isDestroyed) { + return; + } + this.destructionRequested = true; + if (this.retainedScenegraphs === 0) { + this.destroyResources(); + } + } + + /** Alias for destroy(), useful with application-level disposable resource collections. */ + dispose(): void { + this.destroy(); + } + + /** @internal Returns an existing texture only when image, sampling, and interpretation match. */ + getOrCreateTexture( + image: object, + options: GLTFResourceTextureOptions, + createTexture: () => Texture + ): Texture { + // A destroyed cache cannot own additional GPU resources. + assert(!this.isDestroyed); + + let imageTextures = this.textures.get(image); + if (!imageTextures) { + imageTextures = new Map(); + this.textures.set(image, imageTextures); + } + + const samplerEntries = Object.entries(options.sampler).sort(([first], [second]) => + first.localeCompare(second) + ); + const textureKey = JSON.stringify([options.colorSpace, options.role, samplerEntries]); + let texture = imageTextures.get(textureKey); + if (!texture) { + texture = createTexture(); + imageTextures.set(textureKey, texture); + } + return texture; + } + + /** @internal Reuses a fully parsed source material only for the same shader feature layout. */ + getOrCreateMaterial( + sourceMaterial: object, + specialization: string, + createMaterial: () => ParsedPBRMaterial + ): ParsedPBRMaterial { + // A destroyed cache cannot expose material bindings to released textures. + assert(!this.isDestroyed); + + let sourceSpecializations = this.materialSpecializations.get(sourceMaterial); + if (!sourceSpecializations) { + sourceSpecializations = new Map(); + this.materialSpecializations.set(sourceMaterial, sourceSpecializations); + } + + let parsedMaterial = sourceSpecializations.get(specialization); + if (!parsedMaterial) { + parsedMaterial = createMaterial(); + sourceSpecializations.set(specialization, parsedMaterial); + } + return parsedMaterial; + } + + /** @internal Distinguishes caller-owned environments when specializing shared materials. */ + getObjectIdentifier(object: object): number { + let identifier = this.objectIdentifiers.get(object); + if (identifier === undefined) { + identifier = this.objectIdentifiers.size + 1; + this.objectIdentifiers.set(object, identifier); + } + return identifier; + } + + private destroyResources(): void { + if (this.isDestroyed) { + return; + } + for (const imageTextures of this.textures.values()) { + for (const texture of imageTextures.values()) { + texture.destroy(); + } + } + this.textures.clear(); + this.materialSpecializations.clear(); + this.objectIdentifiers.clear(); + this.isDestroyed = true; + } +} diff --git a/modules/gltf/src/index.ts b/modules/gltf/src/index.ts index 4468cc0c9d..dc0570ae37 100644 --- a/modules/gltf/src/index.ts +++ b/modules/gltf/src/index.ts @@ -18,6 +18,7 @@ export { type GLTFScenegraphBounds, type GLTFScenegraphs } from './gltf/create-scenegraph-from-gltf'; +export type {ParseGLTFOptions} from './parsers/parse-gltf'; export { GLTFAnimationClip, type GLTFAnimationClipProps, @@ -38,6 +39,10 @@ export { } from './gltf/gltf-skin'; export {parseGLTFAnimations} from './parsers/parse-gltf-animations'; export {type ParseGLTFLightsOptions, parseGLTFLights} from './parsers/parse-gltf-lights'; +export { + GLTFAssetResourceCache, + type GLTFResourceTextureOptions +} from './gltf/gltf-resource-cache'; export { exportGLTF, type GLTFExportAccessor, diff --git a/modules/gltf/src/parsers/parse-gltf.ts b/modules/gltf/src/parsers/parse-gltf.ts index 93c14fec3c..79e7478550 100644 --- a/modules/gltf/src/parsers/parse-gltf.ts +++ b/modules/gltf/src/parsers/parse-gltf.ts @@ -25,7 +25,9 @@ import {createGLTFMaterial, createGLTFModel} from '../gltf/create-gltf-model'; import {getGLTFNodeInstancing, type GLTFGPUInstancing} from '../gltf/gltf-instancing'; import type {GLTFPrimitiveMaterialVariants} from '../gltf/gltf-material-variants'; import {type GLTFMorphTargetState, setGLTFMorphWeights} from '../gltf/morph-targets'; +import type {GLTFAssetResourceCache} from '../gltf/gltf-resource-cache'; import {type PBREnvironment} from '../pbr/pbr-environment'; +import type {ParsedPBRMaterial} from '../pbr/pbr-material'; import {convertGLDrawModeToTopology} from '../webgl-to-webgpu/convert-webgl-topology'; import {parsePBRMaterial} from './parse-pbr-material'; @@ -46,6 +48,8 @@ export type ParseGLTFOptions = { useByteColors?: boolean; /** Reject documents whose required extensions have no complete runtime implementation. */ strictExtensions?: boolean; + /** Reuses caller-owned textures and material specializations across compatible scenegraphs. */ + resourceCache?: GLTFAssetResourceCache; }; const defaultOptions: Required = { @@ -55,7 +59,8 @@ const defaultOptions: Required = { lights: true, useTangents: false, useByteColors: true, - strictExtensions: false + strictExtensions: false, + resourceCache: undefined! }; /** @@ -80,18 +85,17 @@ export function parseGLTF( } { const combinedOptions = {...defaultOptions, ...options}; const materialFactory = new MaterialFactory(device, {modules: [pbrMaterial]}); + const sourceMaterialAttributes = getSourceMaterialAttributes(gltf); const materials = (gltf.materials || []).map((gltfMaterial, materialIndex) => createGLTFMaterial(device, { id: getGLTFMaterialId(gltfMaterial, materialIndex), - parsedPPBRMaterial: parsePBRMaterial( + parsedPPBRMaterial: getParsedGLTFMaterial( device, - gltfMaterial as any, - {}, - { - ...combinedOptions, - gltf, - validateAttributes: false - } + gltfMaterial, + sourceMaterialAttributes.get(gltfMaterial.id) || {}, + gltf, + combinedOptions, + false ), materialFactory }) @@ -206,6 +210,69 @@ export function parseGLTF( return {scenes, materials, gltfMeshIdToNodeMap, gltfNodeIdToNodeMap, gltfNodeIndexToNodeMap}; } +/** Collects every primitive layout used by each source material and authored variant. */ +function getSourceMaterialAttributes(gltf: GLTFPostprocessed): Map> { + const attributesByMaterial = new Map>(); + + for (const mesh of gltf.meshes) { + for (const primitive of mesh.primitives || []) { + const sourceMaterials: GLTFMaterialPostprocessed[] = primitive.material + ? [primitive.material] + : []; + for (const mapping of primitive.extensions?.['KHR_materials_variants']?.mappings || []) { + const sourceMaterial = + typeof mapping.material === 'number' + ? gltf.materials[mapping.material] + : mapping.material; + if (sourceMaterial) { + sourceMaterials.push(sourceMaterial); + } + } + + for (const material of sourceMaterials) { + attributesByMaterial.set(material.id, { + ...attributesByMaterial.get(material.id), + ...primitive.attributes + }); + } + } + } + + return attributesByMaterial; +} + +/** Reuses parsed material state only when source identity and shader specialization agree. */ +function getParsedGLTFMaterial( + device: Device, + sourceMaterial: GLTFMaterialPostprocessed | undefined, + attributes: Record, + gltf: GLTFPostprocessed, + options: Required, + validateAttributes = true +): ParsedPBRMaterial { + const parseMaterial = () => + parsePBRMaterial(device, sourceMaterial as any, attributes, { + ...options, + gltf, + validateAttributes + }); + const resourceCache = options.resourceCache; + if (!resourceCache || !sourceMaterial) { + return parseMaterial(); + } + + const environment = options.imageBasedLightingEnvironment; + const specialization = JSON.stringify({ + attributes: Object.keys(attributes).sort(), + imageBasedLightingEnvironment: environment ? resourceCache.getObjectIdentifier(environment) : 0, + lights: options.lights, + pbrDebug: options.pbrDebug, + useTangents: options.useTangents + }); + + return resourceCache.getOrCreateMaterial(sourceMaterial, specialization, parseMaterial); +} + /** Creates a `GroupNode` for one glTF node transform. */ function createNodeForGLTFNode( device: Device, @@ -284,10 +351,13 @@ function createNodeForGLTFPrimitive({ const geometry = createGeometry(id, gltfPrimitive, topology); - const parsedPPBRMaterial = parsePBRMaterial(device, gltfPrimitive.material, geometry.attributes, { - ...options, - gltf - }); + const parsedPPBRMaterial = getParsedGLTFMaterial( + device, + gltfPrimitive.material, + geometry.attributes, + gltf, + options + ); const modelNode = createGLTFModel(device, { id, @@ -320,10 +390,13 @@ function createNodeForGLTFPrimitive({ if (!material) { continue; } - const variantMaterial = parsePBRMaterial(device, sourceMaterial as any, geometry.attributes, { - ...options, - gltf - }); + const variantMaterial = getParsedGLTFMaterial( + device, + sourceMaterial, + geometry.attributes, + gltf, + options + ); for (const variantIndex of mapping.variants || []) { mappings.set(variantIndex, { material, diff --git a/modules/gltf/src/parsers/parse-pbr-material.ts b/modules/gltf/src/parsers/parse-pbr-material.ts index 1799ee7838..988c36cc42 100644 --- a/modules/gltf/src/parsers/parse-pbr-material.ts +++ b/modules/gltf/src/parsers/parse-pbr-material.ts @@ -18,6 +18,7 @@ import { } from '../pbr/texture-transform'; import {convertSampler} from '../webgl-to-webgpu/convert-webgl-sampler'; import {GLEnum} from '../webgl-to-webgpu/gltf-webgl-constants'; +import type {GLTFAssetResourceCache} from '../gltf/gltf-resource-cache'; // TODO - synchronize the GLTF... types with loaders.gl // TODO - remove the glParameters, use only parameters @@ -191,6 +192,8 @@ export type ParsePBRMaterialOptions = { gltf?: GLTFPostprocessed; /** run primitive-attribute diagnostics such as missing TEXCOORD_0 / NORMAL */ validateAttributes?: boolean; + /** Optional asset-scoped owner that reuses compatible source image textures. */ + resourceCache?: GLTFAssetResourceCache; }; /** @@ -216,7 +219,8 @@ export function parsePBRMaterial( }, parameters: {}, glParameters: {}, - generatedTextures: [] + generatedTextures: [], + ...(options.resourceCache ? {resourceCache: options.resourceCache} : {}) }; // TODO - always available @@ -948,15 +952,28 @@ function addTexture( } satisfies SamplerProps }; - const texture = createGLTFTexture(device, image, baseOptions); + const textureTransformSlotDefinition = textureTransformSlot + ? getTextureTransformSlotDefinition(textureTransformSlot) + : undefined; + const colorSpace = textureTransformSlotDefinition?.colorSpace || 'linear'; + const texture = parsedMaterial.resourceCache + ? parsedMaterial.resourceCache.getOrCreateTexture( + image, + { + sampler: baseOptions.sampler, + colorSpace, + role: colorSpace === 'srgb' ? 'color' : 'data' + }, + () => createGLTFTexture(device, image, baseOptions) + ) + : createGLTFTexture(device, image, baseOptions); parsedMaterial.bindings[uniformName] = texture; if (define) parsedMaterial.defines[define] = true; if (enabledUniformName) { parsedMaterial.uniforms[enabledUniformName] = true; } - if (textureTransformSlot) { - const textureTransformSlotDefinition = getTextureTransformSlotDefinition(textureTransformSlot); + if (textureTransformSlotDefinition) { (parsedMaterial.uniforms as Record)[textureTransformSlotDefinition.uvSetUniform] = textureCoordinateSet; (parsedMaterial.uniforms as Record)[ diff --git a/modules/gltf/src/pbr/pbr-material.ts b/modules/gltf/src/pbr/pbr-material.ts index 28d750cf46..df1a349d06 100644 --- a/modules/gltf/src/pbr/pbr-material.ts +++ b/modules/gltf/src/pbr/pbr-material.ts @@ -9,6 +9,7 @@ import { PBRMaterialUniforms, PBRProjectionProps } from '@luma.gl/shadertools'; +import type {GLTFAssetResourceCache} from '../gltf/gltf-resource-cache'; type ParsedPBRMaterialUniforms = Partial & { clearcoatRoughnessMapEnabled?: boolean; @@ -29,4 +30,6 @@ export type ParsedPBRMaterial = { readonly glParameters: Record; /** List of all generated textures, makes it easy to destroy them later */ readonly generatedTextures: Texture[]; + /** Optional asset-scoped owner for shared textures and material specializations. */ + readonly resourceCache?: GLTFAssetResourceCache; }; diff --git a/modules/gltf/test/export/gltf-exporter.node.spec.ts b/modules/gltf/test/export/gltf-exporter.node.spec.ts index c4c7f9a208..d2b3b5f5c6 100644 --- a/modules/gltf/test/export/gltf-exporter.node.spec.ts +++ b/modules/gltf/test/export/gltf-exporter.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {parse} from '@loaders.gl/core'; import {GLTFLoader, postProcessGLTF} from '@loaders.gl/gltf'; diff --git a/modules/gltf/test/gltf/gltf-native-extensions.node.spec.ts b/modules/gltf/test/gltf/gltf-native-extensions.node.spec.ts index 5fc7c51c4f..c50bbfb03d 100644 --- a/modules/gltf/test/gltf/gltf-native-extensions.node.spec.ts +++ b/modules/gltf/test/gltf/gltf-native-extensions.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {readFile} from 'node:fs/promises'; import {parse} from '@loaders.gl/core'; diff --git a/modules/gltf/test/gltf/gltf-native-extensions.spec.ts b/modules/gltf/test/gltf/gltf-native-extensions.spec.ts index 42eb2169f2..3b79e43f4f 100644 --- a/modules/gltf/test/gltf/gltf-native-extensions.spec.ts +++ b/modules/gltf/test/gltf/gltf-native-extensions.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {load} from '@loaders.gl/core'; import {GLTFLoader, postProcessGLTF} from '@loaders.gl/gltf'; diff --git a/modules/gltf/test/gltf/gltf-resource-lifecycle.node.spec.ts b/modules/gltf/test/gltf/gltf-resource-lifecycle.node.spec.ts new file mode 100644 index 0000000000..ba28538656 --- /dev/null +++ b/modules/gltf/test/gltf/gltf-resource-lifecycle.node.spec.ts @@ -0,0 +1,220 @@ +// luma.gl +// SPDX-License-Identifier: MIT +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors + +import {readFile} from 'node:fs/promises'; +import {parse} from '@loaders.gl/core'; +import {GLTFLoader, postProcessGLTF, type GLTFPostprocessed} from '@loaders.gl/gltf'; +import {type Buffer, type Texture} from '@luma.gl/core'; +import {ModelNode} from '@luma.gl/engine'; +import { + createScenegraphsFromGLTF, + GLTFAssetResourceCache, + type ParseGLTFOptions, + type PBREnvironment +} from '@luma.gl/gltf'; +import {NullDevice} from '@luma.gl/test-utils'; +import {describe, expect, test} from 'vitest'; + +async function loadAsset(name: 'SimpleInstancing' | 'CubeVisibility'): Promise { + const source = await readFile(new URL(`../data/${name}.glb`, import.meta.url)); + return postProcessGLTF(await parse(source, GLTFLoader, {gltf: {loadImages: false}})); +} + +function makeSourceImage(): Record { + return { + compressed: true, + mipmaps: true, + data: [ + { + data: new Uint8Array(16), + width: 4, + height: 4, + textureFormat: 'bc7-rgba-unorm' + } + ] + }; +} + +function makeSourceTexture( + image: Record, + sampler: Record = {} +): any { + return { + id: 'source-image', + texture: {source: {image}, sampler: {parameters: sampler}} + }; +} + +function getActiveResourceCount(device: NullDevice, resource: 'Buffers' | 'Textures'): number { + return device.statsManager.getStats('Resource Counts').get(`${resource} Active`).count; +} + +describe('glTF asset resource ownership', () => { + test('reuses image textures across source materials, primitive layouts, and variants', async () => { + const device = new NullDevice({}); + const source = await loadAsset('CubeVisibility'); + const image = makeSourceImage(); + + for (const sourceMaterial of source.materials) { + sourceMaterial.pbrMetallicRoughness = { + ...sourceMaterial.pbrMetallicRoughness, + baseColorTexture: makeSourceTexture(image) + }; + } + source.extensions = { + ...source.extensions, + KHR_materials_variants: {variants: [{name: 'Shared'}]} + } as GLTFPostprocessed['extensions']; + source.meshes[0].primitives[0].extensions = { + KHR_materials_variants: {mappings: [{material: 1, variants: [0]}]} + }; + + const initialTextureCount = getActiveResourceCount(device, 'Textures'); + const scenegraphs = createScenegraphsFromGLTF(device, source); + const textures = scenegraphs.materials.map( + material => material.getBindings().pbr_baseColorSampler + ); + + expect(scenegraphs.resourceCache.textureCount).toBe(1); + expect(scenegraphs.resourceCache.materialCount).toBe(3); + expect(getActiveResourceCount(device, 'Textures')).toBe(initialTextureCount + 1); + expect(textures[0]).toBe(textures[1]); + expect(textures[1]).toBe(textures[2]); + scenegraphs.variants.selectVariant('Shared'); + + scenegraphs.destroy(); + scenegraphs.destroy(); + scenegraphs.dispose(); + + expect((textures[0] as Texture).destroyed).toBe(true); + expect(scenegraphs.resourceCache.destroyed).toBe(true); + expect(getActiveResourceCount(device, 'Textures')).toBe(initialTextureCount); + device.destroy(); + }); + + test('separates authored sampler settings and source color interpretation', async () => { + const device = new NullDevice({}); + const source = await loadAsset('CubeVisibility'); + const image = makeSourceImage(); + + source.materials[0].pbrMetallicRoughness = { + ...source.materials[0].pbrMetallicRoughness, + baseColorTexture: makeSourceTexture(image) + }; + source.materials[1].pbrMetallicRoughness = { + ...source.materials[1].pbrMetallicRoughness, + baseColorTexture: makeSourceTexture(image, {10242: 33071}) + }; + source.materials[2].normalTexture = makeSourceTexture(image); + + const scenegraphs = createScenegraphsFromGLTF(device, source); + const repeatColor = scenegraphs.materials[0].getBindings().pbr_baseColorSampler; + const clampedColor = scenegraphs.materials[1].getBindings().pbr_baseColorSampler; + const linearNormal = scenegraphs.materials[2].getBindings().pbr_normalSampler; + + expect(scenegraphs.resourceCache.textureCount).toBe(3); + expect(repeatColor).not.toBe(clampedColor); + expect(repeatColor).not.toBe(linearNormal); + expect((clampedColor as Texture).sampler.props.addressModeU).toBe('clamp-to-edge'); + + scenegraphs.destroy(); + expect((repeatColor as Texture).destroyed).toBe(true); + expect((clampedColor as Texture).destroyed).toBe(true); + expect((linearNormal as Texture).destroyed).toBe(true); + device.destroy(); + }); + + test('defers caller-owned cache destruction while independently owned scenegraphs remain', async () => { + const device = new NullDevice({}); + const source = await loadAsset('CubeVisibility'); + source.materials[0].pbrMetallicRoughness = { + ...source.materials[0].pbrMetallicRoughness, + baseColorTexture: makeSourceTexture(makeSourceImage()) + }; + + const resourceCache = new GLTFAssetResourceCache(device); + const options: ParseGLTFOptions = {resourceCache}; + const firstScene = createScenegraphsFromGLTF(device, source, options); + const secondScene = createScenegraphsFromGLTF(device, source, options); + const firstTexture = firstScene.materials[0].getBindings().pbr_baseColorSampler as Texture; + const secondTexture = secondScene.materials[0].getBindings().pbr_baseColorSampler as Texture; + + expect(firstScene.materials[0]).not.toBe(secondScene.materials[0]); + expect(firstTexture).toBe(secondTexture); + expect(resourceCache.referenceCount).toBe(2); + expect(resourceCache.materialCount).toBe(source.materials.length); + + firstScene.dispose(); + expect(resourceCache.referenceCount).toBe(1); + expect(firstTexture.destroyed).toBe(false); + + resourceCache.destroy(); + expect(resourceCache.destroyed).toBe(false); + expect(secondTexture.destroyed).toBe(false); + + secondScene.destroy(); + expect(resourceCache.referenceCount).toBe(0); + expect(resourceCache.destroyed).toBe(true); + expect(secondTexture.destroyed).toBe(true); + resourceCache.dispose(); + device.destroy(); + }); + + test('destroys hidden and detached instanced models without consuming borrowed IBL textures', async () => { + const device = new NullDevice({}); + const source = await loadAsset('SimpleInstancing'); + const borrowedEnvironmentTexture = device.createTexture({width: 1, height: 1}); + const imageBasedLightingEnvironment = { + diffuseEnvSampler: {texture: borrowedEnvironmentTexture}, + specularEnvSampler: {texture: borrowedEnvironmentTexture}, + brdfLutTexture: {texture: borrowedEnvironmentTexture} + } as PBREnvironment; + const initialBufferCount = getActiveResourceCount(device, 'Buffers'); + const scenegraphs = createScenegraphsFromGLTF(device, source, { + imageBasedLightingEnvironment + }); + + const instanceBuffers: Buffer[] = []; + scenegraphs.scenes[0].preorderTraversal(node => { + if (node instanceof ModelNode) { + instanceBuffers.push(...(node.managedResources as Buffer[])); + } + }); + expect(instanceBuffers).toHaveLength(4); + expect(instanceBuffers.every(buffer => !buffer.destroyed)).toBe(true); + + scenegraphs.scenes[0].display = false; + scenegraphs.destroy(); + + expect(instanceBuffers.every(buffer => buffer.destroyed)).toBe(true); + expect(getActiveResourceCount(device, 'Buffers')).toBe(initialBufferCount); + expect(borrowedEnvironmentTexture.destroyed).toBe(false); + borrowedEnvironmentTexture.destroy(); + device.destroy(); + }); + + test('keeps an externally supplied cache available after its last scene is destroyed', async () => { + const device = new NullDevice({}); + const source = await loadAsset('CubeVisibility'); + source.materials[0].pbrMetallicRoughness = { + ...source.materials[0].pbrMetallicRoughness, + baseColorTexture: makeSourceTexture(makeSourceImage()) + }; + const resourceCache = new GLTFAssetResourceCache(device); + + const firstScene = createScenegraphsFromGLTF(device, source, {resourceCache}); + const texture = firstScene.materials[0].getBindings().pbr_baseColorSampler as Texture; + firstScene.destroy(); + expect(resourceCache.referenceCount).toBe(0); + expect(resourceCache.destroyed).toBe(false); + expect(texture.destroyed).toBe(false); + + const secondScene = createScenegraphsFromGLTF(device, source, {resourceCache}); + expect(secondScene.materials[0].getBindings().pbr_baseColorSampler).toBe(texture); + secondScene.destroy(); + resourceCache.destroy(); + expect(texture.destroyed).toBe(true); + device.destroy(); + }); +}); diff --git a/modules/gltf/test/gltf/gltf-resource-lifecycle.spec.ts b/modules/gltf/test/gltf/gltf-resource-lifecycle.spec.ts new file mode 100644 index 0000000000..75ddef9641 --- /dev/null +++ b/modules/gltf/test/gltf/gltf-resource-lifecycle.spec.ts @@ -0,0 +1,108 @@ +// luma.gl +// SPDX-License-Identifier: MIT +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors + +import {load} from '@loaders.gl/core'; +import {GLTFLoader, postProcessGLTF} from '@loaders.gl/gltf'; +import {type Buffer, type Device, type Texture} from '@luma.gl/core'; +import {ModelNode} from '@luma.gl/engine'; +import {createScenegraphsFromGLTF, GLTFAssetResourceCache, parsePBRMaterial} from '@luma.gl/gltf'; +import {getTestDevices} from '@luma.gl/test-utils'; +import test from 'test/utils/vitest-tape'; + +function getActiveResourceCount(device: Device, resource: 'Buffers' | 'Textures'): number { + return device.statsManager.getStats('Resource Counts').get(`${resource} Active`).count; +} + +test('glTF reuses and releases real image textures on every available GPU backend', async testCase => { + for (const device of await getTestDevices()) { + const image = new ImageData(new Uint8ClampedArray([255, 96, 32, 255]), 1, 1); + const sourceMaterial: Parameters[1] = { + pbrMetallicRoughness: { + baseColorTexture: { + id: 'shared-gpu-image', + texture: {source: {image}, sampler: {parameters: {}}} + } + } + }; + const initialTextureCount = getActiveResourceCount(device, 'Textures'); + const resourceCache = new GLTFAssetResourceCache(device); + + const first = parsePBRMaterial( + device, + sourceMaterial, + {NORMAL: {}, TEXCOORD_0: {}}, + {resourceCache} + ); + const second = parsePBRMaterial( + device, + sourceMaterial, + {NORMAL: {}, TEXCOORD_0: {}}, + {resourceCache} + ); + const texture = first.bindings.pbr_baseColorSampler as Texture; + + testCase.equal( + second.bindings.pbr_baseColorSampler, + texture, + `${device.type} reuses the same real image upload` + ); + testCase.equal(resourceCache.textureCount, 1, `${device.type} owns one unique texture`); + testCase.equal( + getActiveResourceCount(device, 'Textures'), + initialTextureCount + 1, + `${device.type} allocates one GPU texture` + ); + + resourceCache.destroy(); + resourceCache.destroy(); + + testCase.ok(texture.destroyed, `${device.type} destroys its owned GPU texture`); + testCase.equal( + getActiveResourceCount(device, 'Textures'), + initialTextureCount, + `${device.type} returns its texture count to baseline` + ); + } + + testCase.end(); +}); + +test('glTF repeatedly releases instancing buffers and hidden models on real GPU backends', async testCase => { + const source = postProcessGLTF( + await load(new URL('../data/SimpleInstancing.glb', import.meta.url).href, GLTFLoader, { + gltf: {loadImages: false} + }) + ); + + for (const device of await getTestDevices()) { + const initialBufferCount = getActiveResourceCount(device, 'Buffers'); + + for (let iteration = 0; iteration < 3; iteration++) { + const scenegraphs = createScenegraphsFromGLTF(device, source, {strictExtensions: true}); + const instanceBuffers: Buffer[] = []; + scenegraphs.scenes[0].preorderTraversal(node => { + if (node instanceof ModelNode) { + instanceBuffers.push(...(node.managedResources as Buffer[])); + } + }); + + testCase.equal(instanceBuffers.length, 4, `${device.type} creates four instance columns`); + scenegraphs.scenes[0].display = false; + scenegraphs.destroy(); + scenegraphs.dispose(); + + testCase.ok( + instanceBuffers.every(buffer => buffer.destroyed), + `${device.type} releases every instance buffer after load ${iteration + 1}` + ); + testCase.equal( + getActiveResourceCount(device, 'Buffers'), + initialBufferCount, + `${device.type} restores its buffer baseline after load ${iteration + 1}` + ); + } + } + + testCase.end(); +}); diff --git a/modules/gltf/test/gltf/gltf-review-correctness.node.spec.ts b/modules/gltf/test/gltf/gltf-review-correctness.node.spec.ts index 62fa6397ba..57f1de11b2 100644 --- a/modules/gltf/test/gltf/gltf-review-correctness.node.spec.ts +++ b/modules/gltf/test/gltf/gltf-review-correctness.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {readFile} from 'node:fs/promises'; import {parse} from '@loaders.gl/core'; diff --git a/modules/gltf/test/index.ts b/modules/gltf/test/index.ts index 1f618e9ac9..fb33eec27b 100644 --- a/modules/gltf/test/index.ts +++ b/modules/gltf/test/index.ts @@ -14,3 +14,4 @@ import './parsers/parse-pbr-compressed-texture.spec'; import './parsers/parse-pbr-material.spec'; import './parsers/parse-pbr-sampler.spec'; import './gltf/gltf-native-extensions.spec'; +import './gltf/gltf-resource-lifecycle.spec'; diff --git a/test/examples/gltf-showcase-identity.node.spec.ts b/test/examples/gltf-showcase-identity.node.spec.ts index 47f59d52fc..ada59e83e7 100644 --- a/test/examples/gltf-showcase-identity.node.spec.ts +++ b/test/examples/gltf-showcase-identity.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {readFileSync} from 'node:fs'; import path from 'node:path'; diff --git a/test/examples/renderer-review-correctness.node.spec.ts b/test/examples/renderer-review-correctness.node.spec.ts index fb807a45b0..24355cc8df 100644 --- a/test/examples/renderer-review-correctness.node.spec.ts +++ b/test/examples/renderer-review-correctness.node.spec.ts @@ -1,6 +1,6 @@ // luma.gl // SPDX-License-Identifier: MIT -// Copyright (c) vis.gl contributors +// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors import {readFileSync} from 'node:fs'; import path from 'node:path';