Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions docs/api-reference/experimental/deferred-scene-renderer.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,15 @@ const options: SceneRenderOptions = {
console.log('Deferred-compatible scene:', supportsDeferredScene(options));

const statistics = renderer.render(options);
device.submit();

renderer.destroy();
```

`supportsDeferredScene(options)` reports the current scene's compatibility before rendering.
Calling it is optional: `DeferredSceneRenderer.render(options)` performs the same check and
selects forward rendering automatically when required.
selects forward rendering automatically when required. Rendering records the frame; submit the
device's command queue before presenting or destroying renderer-owned resources.

## Supported deferred scenes

Expand All @@ -72,7 +74,9 @@ present:
- A `BLEND` material or base-color alpha that infers blending.
- Any supplied environment-lighting texture.
- Transmission, nonzero thickness, clearcoat, sheen, iridescence, or anisotropy.
- A nondefault index of refraction, specular intensity, or specular color.
- A nondefault index of refraction, specular intensity, specular color, or authored specular map.
- Spot lights, whose direction and cone angles require forward shading.
- More than one directional light, which exceeds the deferred lighting pass's single-light layout.
- An unlit material.
- `debugNormals` or `debugDepth` output.

Expand Down
4 changes: 4 additions & 0 deletions docs/api-reference/experimental/pbr-environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ renderer.render({
camera,
environment
});
device.submit();

renderer.destroy();
environment.destroy();
Expand Down Expand Up @@ -69,6 +70,7 @@ const environment = preparePBREnvironment(device, {
});

renderer.render({...options, environment});
device.submit();

environment.destroy();
```
Expand Down Expand Up @@ -123,6 +125,7 @@ environment.intensity = 0.8;
environment.rotation = Math.PI / 2;

renderer.render({...options, environment});
device.submit();
```

## Color encoding and HDR
Expand Down Expand Up @@ -200,6 +203,7 @@ renderer.render({
brdfLUTTexture: loadedEnvironment.brdfLutTexture.texture
}
});
device.submit();
```

Choose `loadPBREnvironment()` for existing prefiltered assets. Choose `PBREnvironmentGenerator`
Expand Down
11 changes: 11 additions & 0 deletions docs/api-reference/experimental/scene-renderer.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ const options: SceneRenderOptions = {

const renderer = new SceneRenderer(device);
const statistics = renderer.render(options);
device.submit();

// {surfaceCount: 1, instanceCount: 2, drawCount: 1, triangleCount: 2}
console.log(statistics);
Expand All @@ -76,6 +77,10 @@ Every matrix in `transforms` places the same geometry/material pair in world spa
uploads the matrices to instanced vertex attributes and keeps all placements for one surface in
one draw. A surface with no transforms is not drawn.

`render()` records its render pass but does not submit the device's command queue. Call
`device.submit()` after encoding the frame, and always before destroying the renderer or any
borrowed resources. Applications combining multiple passes may submit once after the final pass.

## Scene descriptors

### `SceneSurface`
Expand Down Expand Up @@ -111,11 +116,13 @@ const deformingSurface: SceneSurface = {
};

renderer.render({...options, surfaces: [deformingSurface]});
device.submit();

deformingSurface.skin = {jointMatrices: nextJointMatrices};
deformingSurface.morphWeights = [0.7];

renderer.render({...options, surfaces: [deformingSurface]});
device.submit();
```

The existing shadertools `skin` module uploads the supplied joint palette, and the existing
Expand Down Expand Up @@ -214,6 +221,7 @@ renderer.render({
toneMapMode: PBR_TONE_MAP_MODE.KHRONOS_PBR_NEUTRAL,
outputColorSpace: 'srgb'
});
device.submit();
```

An `rgba16float` framebuffer defaults to linear, untonemapped output, preserving radiance above
Expand All @@ -238,6 +246,7 @@ const environment: SceneEnvironment = {
};

renderer.render({...options, environment});
device.submit();
```

All three textures must be present before IBL is enabled. An incomplete environment leaves IBL
Expand All @@ -260,6 +269,7 @@ const generatedEnvironment = generator.prepare({
});

renderer.render({...options, environment: generatedEnvironment});
device.submit();
```

[`PBREnvironmentGenerator`](/docs/api-reference/experimental/pbr-environment) integrates all six
Expand Down Expand Up @@ -324,6 +334,7 @@ renderer.render({
surfaces: [opaqueBackgroundSurface, glassSurface],
transmission: true
});
device.submit();
```

Physical transmission is distinct from alpha blending. Keep a genuinely opaque glTF transmission
Expand Down
9 changes: 5 additions & 4 deletions docs/capabilities.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -404,15 +404,16 @@ Try [Lightstorm Megacity](/examples/showcase/lightstorm-megacity),
| Metallic-roughness materials | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Apply supported PBR textures, material factors, normals, and emissive surfaces. |
| Specular and index of refraction | Evolving | WebGPU + WebGL2 | `@luma.gl/gltf` | Support documented glTF specular and index-of-refraction material extensions. |
| Clearcoat and sheen | Evolving | WebGPU + WebGL2 | `@luma.gl/gltf` | Apply documented clearcoat and sheen material lobes in compatible renderers. |
| Iridescence and anisotropy | Evolving | WebGPU + WebGL2 | `@luma.gl/gltf` | Use supported material approximations for thin-film color and directional highlights. |
| Transmission and volume | Evolving | WebGPU + WebGL2 | `@luma.gl/gltf` | Transmission remains approximate; full scene-color refraction is not implemented. |
| Iridescence and anisotropy | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Render thin-film interference and directional anisotropic highlights through the shared PBR shader. |
| Transmission and volume | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Shared scene renderers refract captured scene color; the standalone glTF rendering fallback remains approximate. |
| Chromatic dispersion | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Separate visible wavelengths when physical transmission uses authored `KHR_materials_dispersion`. |
| Authored UV transforms | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Preserve supported texture offset, rotation, and scale semantics. |
| Authored normals and tangents | Evolving | WebGPU + WebGL2 | `@luma.gl/gltf` | Preserve supported vertex attributes; complete renderer-to-renderer fidelity varies. |
| Punctual-light parsing | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Parse supported authored directional, point, and spot light definitions. |
| Shared animation clips | Available | WebGPU + WebGL2 | `@luma.gl/engine` | Play, blend, crossfade, loop, and interpolate compatible imported tracks. |
| Selected animation pointers | Evolving | WebGPU + WebGL2 | `@luma.gl/gltf` | Update supported `KHR_animation_pointer` transforms, material factors, and UV properties. |
| Existing joint-driven skinning | Evolving | WebGPU + WebGL2 | `@luma.gl/shadertools` | Reuse established skin shaders; higher joint counts and multiple skins remain incomplete. |
| Morph-target animation | Opportunity | WebGPU + WebGL2 | `@luma.gl/gltf` | Morph-weight playback is not wired through the shared rendering and animation paths. |
| Existing joint-driven skinning | Available | WebGPU + WebGL2 | `@luma.gl/shadertools` | Reuse established skin shaders and automatically bind mesh-local glTF joint palettes. |
| Morph-target animation | Available | WebGPU + WebGL2 | `@luma.gl/gltf` | Animate POSITION, NORMAL, and TANGENT morph targets through shared glTF, engine, and retained-scene paths. |
| Imported GPU instancing | Opportunity | WebGPU + WebGL2 | `@luma.gl/gltf` | `EXT_mesh_gpu_instancing` is not yet translated into retained instance batches. |
| Imported node visibility | Opportunity | WebGPU + WebGL2 | `@luma.gl/gltf` | `KHR_node_visibility` does not yet have a supported runtime integration. |

Expand Down
6 changes: 5 additions & 1 deletion examples/showcase/anari/gltf-to-anari.ts
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,11 @@ function makeImportedLights(
gltf: GLTFPostprocessed,
state: GLTFTranslationState
): JSONLightDeclaration[] {
return parseGLTFLights(gltf, {useByteColors: false}).flatMap(light => {
const activeNodeIdentifiers = new Set(Object.keys(state.nodeIdentifiers));
return parseGLTFLights(gltf, {
nodeIdentifiers: activeNodeIdentifiers,
useByteColors: false
}).flatMap(light => {
if (light.type === 'ambient') {
return [];
}
Expand Down
66 changes: 66 additions & 0 deletions modules/anari/test/gltf-review-light-import.node.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
// luma.gl
// SPDX-License-Identifier: MIT
// Copyright (c) vis.gl contributors

import {readFile} from 'node:fs/promises';
import {parse} from '@loaders.gl/core';
import {GLTFLoader, postProcessGLTF} from '@loaders.gl/gltf';
import {ANARISceneSchema} from '@luma.gl/anari/schemas';
import {parseGLTFLights} from '@luma.gl/gltf';
import {describe, expect, test} from 'vitest';
import {makeANARIJSONSceneFromGLTF} from '../../../examples/showcase/anari/gltf-to-anari';

describe('selected-scene glTF punctual-light ownership', () => {
test('filters inactive-scene lights without losing selected-scene hierarchy', async () => {
const assetData = await readFile(new URL('../../../test/data/box.glb', import.meta.url));
const source = postProcessGLTF(await parse(assetData, GLTFLoader, {gltf: {loadImages: false}}));
const selectedRoot = source.scene?.nodes?.[0] || source.scenes[0]?.nodes?.[0];
expect(selectedRoot).toBeDefined();
if (!selectedRoot) {
return;
}

const documentWithLights = source as typeof source & {
lights?: Array<Record<string, unknown>>;
};
documentWithLights.lights = [
{type: 'point', color: [0.2, 0.4, 0.6], intensity: 3},
{type: 'spot', color: [1, 0, 0], intensity: 17}
];

const selectedLightNode = {
id: 'selected-scene-light',
translation: [1, 2, 3],
extensions: {KHR_lights_punctual: {light: 0}}
} as (typeof source.nodes)[number];
const inactiveLightNode = {
id: 'inactive-scene-light',
translation: [9, 8, 7],
extensions: {KHR_lights_punctual: {light: 1}}
} as (typeof source.nodes)[number];
selectedRoot.children = [...(selectedRoot.children || []), selectedLightNode];
source.nodes.push(selectedLightNode, inactiveLightNode);
source.scenes.push({
id: 'inactive-light-scene',
nodes: [inactiveLightNode]
} as (typeof source.scenes)[number]);

expect(parseGLTFLights(source, {useByteColors: false})).toHaveLength(2);
expect(
parseGLTFLights(source, {
nodeIdentifiers: new Set([selectedLightNode.id]),
useByteColors: false
})
).toHaveLength(1);

const retainedScene = await makeANARIJSONSceneFromGLTF(source, 'SELECTED LIGHT SCENE');
const authoredLights = (retainedScene.lights || []).filter(light =>
light['@@id'].startsWith('source-')
);
expect(authoredLights).toHaveLength(1);
expect(authoredLights[0]['@@type']).toBe('point');
expect(authoredLights[0].color).toEqual([0.2, 0.4, 0.6]);
expect(authoredLights[0].intensity).toBe(3);
expect(ANARISceneSchema.safeParse(retainedScene).success).toBe(true);
});
});
63 changes: 59 additions & 4 deletions modules/engine/src/animation/morph-targets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,29 @@ export type MorphTargetAttributes = {
TANGENT?: Float32Array;
};

/** Decodes one immutable vertex attribute into its shader-facing floating-point values. */
export function decodeMorphTargetAttribute(attribute: GeometryAttribute): Float32Array {
const values = attribute.value;
if (values instanceof Float32Array) {
return values;
}

const decoded = new Float32Array(values.length);
const maximum = getNormalizedAttributeMaximum(values);
const signed =
values instanceof Int8Array || values instanceof Int16Array || values instanceof Int32Array;
for (let componentIndex = 0; componentIndex < values.length; componentIndex++) {
const value = Number(values[componentIndex]);
decoded[componentIndex] =
attribute['normalized'] && maximum
? signed
? Math.max(value / maximum, -1)
: value / maximum
: value;
}
return decoded;
}

/** Applies weighted morph deltas without modifying the immutable source vertex attributes. */
export function applyMorphTargets(
baseAttributes: Readonly<MorphTargetAttributes>,
Expand Down Expand Up @@ -72,9 +95,9 @@ export function updateMorphTargetBuffers(
): void {
const baseAttributes: MorphTargetAttributes = {};
for (const attributeName of ['POSITION', 'NORMAL', 'TANGENT'] as const) {
const values = geometry.attributes[attributeName]?.value;
if (values instanceof Float32Array) {
baseAttributes[attributeName] = values;
const attribute = geometry.attributes[attributeName];
if (attribute) {
baseAttributes[attributeName] = decodeMorphTargetAttribute(attribute);
}
}

Expand All @@ -89,7 +112,7 @@ export function updateMorphTargetBuffers(
const values = morphedAttributes[attributeName];
const source = attributes[attributeName];
if (values && source) {
attributes[attributeName] = {...source, value: values};
attributes[attributeName] = {...source, value: encodeMorphTargetAttribute(source, values)};
}
}

Expand Down Expand Up @@ -124,6 +147,38 @@ export function updateMorphTargetBuffers(
}
}

function encodeMorphTargetAttribute(
attribute: GeometryAttribute,
values: Float32Array
): GeometryAttribute['value'] {
if (attribute.value instanceof Float32Array) {
return values;
}

const encoded = attribute.value.slice();
const maximum = getNormalizedAttributeMaximum(encoded);
const signed =
encoded instanceof Int8Array || encoded instanceof Int16Array || encoded instanceof Int32Array;
for (let componentIndex = 0; componentIndex < values.length; componentIndex++) {
const value = values[componentIndex];
encoded[componentIndex] =
attribute['normalized'] && maximum
? Math.round(Math.max(signed ? -1 : 0, Math.min(1, value)) * maximum)
: value;
}
return encoded;
}

function getNormalizedAttributeMaximum(values: GeometryAttribute['value']): number {
if (values instanceof Int8Array) return 127;
if (values instanceof Uint8Array || values instanceof Uint8ClampedArray) return 255;
if (values instanceof Int16Array) return 32767;
if (values instanceof Uint16Array) return 65535;
if (values instanceof Int32Array) return 2147483647;
if (values instanceof Uint32Array) return 4294967295;
return 0;
}

function normalizeMorphDirections(values: Float32Array, componentCount: number): void {
for (let offset = 0; offset < values.length; offset += componentCount) {
const length = Math.hypot(values[offset], values[offset + 1], values[offset + 2]);
Expand Down
6 changes: 5 additions & 1 deletion modules/engine/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,11 @@ export {AnimationClip} from './animation/animation-clip';
export type {AnimationActionProps, AnimationLoopMode} from './animation/animation-mixer';
export {AnimationAction, AnimationMixer} from './animation/animation-mixer';
export type {MorphTargetAttributes} from './animation/morph-targets';
export {applyMorphTargets, updateMorphTargetBuffers} from './animation/morph-targets';
export {
applyMorphTargets,
decodeMorphTargetAttribute,
updateMorphTargetBuffers
} from './animation/morph-targets';
export type {SkinJointMatricesProps} from './animation/skin';
export {updateSkinJointMatrices} from './animation/skin';
export {Timeline} from './animation/timeline';
Expand Down
Loading
Loading