docs(experimental): demonstrate table-first luVS benchmarks

docs/api-reference/experimental/luvs.md

@@ -1,4 +1,5 @@
 import {ExperimentalDocsTabs} from '@site/src/components/docs/experimental-docs-tabs';
+import {LuvsBenchmark} from '@site/src/components/docs/luvs-benchmark';
 
 # luVS: GPU Vector Similarity and Clustering
 
@@ -214,6 +215,20 @@ kernels, or FAISS implementations are copied into this module. It is not affilia
 by NVIDIA or the RAPIDS project, and it neither implements a compatible cuVS API nor claims feature
 parity.
 
+## Live CPU versus WebGPU benchmark
+
+Run the benchmark explicitly to compare the same deterministic vectors on your browser's CPU and
+WebGPU adapter. Dataset size, dimensions, query count, K, selection density, IVF list count, and
+probe count are configurable. The exact GPU paths are checked against an independent CPU oracle;
+the approximate IVF-flat path reports recall@K against exact search.
+
+<LuvsBenchmark />
+
+GPU query measurements include command encoding, submission, and an explicit completion fence.
+Initial upload, IVF training/index construction, and correctness readback are reported separately.
+Warmups precede repeated measured runs, and displayed query times are medians. Results depend on
+the current browser, WebGPU adapter, data dimensions, filtering, thermal conditions, and workload.
+
 ## Fixed-size GPU table embedding columns
 
 High-dimensional values such as 384-, 768-, or 1,536-component embeddings are not GPU vertex

docs/whats-new.md

@@ -69,6 +69,7 @@ Target Release Date: Q3, 2026
 - **Reusable command-graph contributors** - `GPUCommandGraphContributor` gives small algorithm libraries a structural `addToGraph()` contract, while public aligned-view binding and typed transient-view helpers let those libraries extend command graphs without a runtime registry or hidden submission.
 - **Optional GPU geospatial kernels** - The side-effect-free `@luma.gl/experimental/geospatial` subpath contributes cuSpatial-compatible sinusoidal projection, haversine distance, pairwise planar distances, four-state point-in-polygon classification, nearest-linestring results, grid indexing, and point spatial queries to caller-owned command graphs, including raw binary64 coordinate inputs.
 - **GPU projection patches** - The optional `@luma.gl/experimental/luproj` subpath compiles arbitrary CPU projection providers into adaptive local polynomial patches and projects chunk-preserving coordinate vectors through WebGPU command graphs without discarding raw binary64 source precision.
+- **GPU vector similarity and clustering** - The optional [`@luma.gl/experimental/luvs`](/docs/api-reference/experimental/luvs) backend searches ordinary fixed-size-list GPU table columns with exact squared-Euclidean, cosine, and inner-product rankings; GPU-resident linked-selection masks; deterministic bounded top-K output; GPU k-means; and explicitly approximate IVF-flat search. Existing `@luma.gl/arrow` table adapters upload Arrow embedding columns, while source IDs, validity, batch boundaries, ownership, and rendering remain caller-controlled.
 - **GPU scan, compaction, and indirect drawing** - Typed graph views compose hierarchical `uint32` scan, stable ID compaction, and GPU-written `DrawCommandBuffer` instance counts. Scan and compaction accept fixed-width `GPUVector` imports as one logical sequence while preserving chunk topology. The [GPU Trace Viewer](/examples/experimental/gpu-trace-viewer) demonstrates the path over up to four million spans, while [GPU Frustum Culling](/examples/experimental/gpu-frustum-culling) applies it to indexed indirect rendering of a 3D instance field.
 - **GPU virtual-geometry selection** - [`GPUVirtualGeometrySelection`](/docs/api-reference/experimental/gpu-primitives/gpu-virtual-geometry-selection) traverses breadth-level cluster forests with conservative sphere-frustum tests and pixel-scale geometric error, then reuses stable visibility compaction to publish a deterministic cluster frontier and capacity-safe indirect instance count without CPU readback.
 - **Virtual Geometry Canyon** - The [WebGPU showcase](/examples/experimental/virtual-geometry-canyon) drives a 4×4, six-refinement terrain forest through GPU-only LOD selection and one indexed indirect draw. A shared grid, exact parent-triangle geomorphing, and skirts visualize more than 41 million potential leaf triangles without a per-frame traversal or readback on the CPU.
@@ -95,6 +96,7 @@ Target Release Date: Q3, 2026
 
 - **Composite GPU inputs** - `GPUInputSchema.attributeNames` maps one logical table column to several shader attributes, allowing a shared matrix buffer to feed portable vertex attributes or a WebGPU storage binding without repacking. Ordinary inputs retain the singular `attributeName`.
 - **Generic GPU tables** - Canonical `GPUData`, `GPUVector`, `GPURecordBatch`, and `GPUTable` runtime classes for reusable non-Arrow-specific GPU table ownership and batching.
+- **Fixed-size-list GPU columns** - First-class `fixed-size-list<float32,768>` formats describe arbitrary fixed-width storage rows without inventing unsupported vertex formats; vectors retain logical table-row counts, flattened element counts, preserved batches, and caller-owned storage.
 - **Table-backed rendering** - `GPUTableModel` draws preserved table batches, and `GPUTableGeometry` exposes packed static GPU tables as renderable geometry.
 - **Vertex storage planning** - `GPUTableBufferPlanner` now checks vertex-stage storage buffer limits before choosing storage-backed table attributes, allowing core WebGPU devices to fall back to vertex attributes when needed.
 - **Execution helpers** - `TableTransform`, `GPUTableComputation`, generated-buffer batch planning, and `GPUTableBufferPlanner` now live beside the generic table runtime instead of the Arrow adapter module.
@@ -109,6 +111,7 @@ Target Release Date: Q3, 2026
 
 - **Arrow shader layouts** - `getArrowBufferLayout()` maps Arrow scalar and `FixedSizeList` columns to shader attribute formats from a shader-first layout, including direct `arrow.Vector` sources and Arrow table path mappings.
 - **Arrow GPU adapters** - Arrow factories, append helpers, and readback helpers bridge Apache Arrow inputs into `@luma.gl/tables` objects and preserve chunked UTF-8 GPU vector input for text workflows.
+- **High-dimensional Arrow storage columns** - Existing Arrow table/vector adapters map wide `FixedSizeList` values directly into row-aligned fixed-size-list GPU columns, with optional named validity siblings and preserved parent/child nulls, record batches, and source identity.
 - **Variable-length Arrow attribute lists** - `GPUVector` can retain chunked nested list columns whose elements contain one to four numeric components, covering scalar streams plus tuple-style data such as XY, XYZ, and XYZM coordinates for future path-rendering workflows.
 - **Closed Arrow path normalization** - `closeArrowPaths()` appends explicit closing vertices only for closed Float32 absolute or origin-relative delta path rows whose endpoints differ beyond an epsilon, using WebGPU compute when available with equivalent CPU fallback semantics.
 - **`ArrowPathModel`** - New attribute-backed path renderer consumes prepared Float32 XY, XYZ, and XYZM path props, expands path rows into packed per-segment render records, and supports Float64 source paths through CPU-prepared Float32 deltas plus CPU-updated view origins.

test/examples/luvs-live-benchmark-docs.node.spec.ts

@@ -0,0 +1,262 @@
+// luma.gl
+// SPDX-License-Identifier: MIT
+// SPDX-FileCopyrightText: Copyright (c) vis.gl contributors
+
+import {readFileSync} from 'node:fs';
+import {createRequire} from 'node:module';
+
+import React from 'react';
+import {renderToString} from 'react-dom/server';
+import typescript from 'typescript';
+import {beforeEach, describe, expect, test, vi} from 'vitest';
+
+type MockLuvsBenchmarkPanelProps = {
+  title: string;
+  description: string;
+  runLabel: string;
+  onRun: () => Promise<React.ReactNode>;
+};
+
+const benchmarkSource = readFileSync(
+  new URL('../../website/src/components/docs/luvs-benchmark.tsx', import.meta.url),
+  'utf8'
+);
+const benchmarkRuntimeSource = readFileSync(
+  new URL('../../website/src/components/docs/luvs-benchmark-runtime.ts', import.meta.url),
+  'utf8'
+);
+const documentationSource = readFileSync(
+  new URL('../../docs/api-reference/experimental/luvs.md', import.meta.url),
+  'utf8'
+);
+const navigationSource = readFileSync(
+  new URL('../../website/src/components/docs/experimental-docs-tabs.tsx', import.meta.url),
+  'utf8'
+);
+const sidebarSource = readFileSync(
+  new URL('../../docs/table-of-contents.json', import.meta.url),
+  'utf8'
+);
+const experimentalPackage = JSON.parse(
+  readFileSync(new URL('../../modules/experimental/package.json', import.meta.url), 'utf8')
+) as {exports: Record<string, {import: string; types: string}>};
+const experimentalMainIndex = readFileSync(
+  new URL('../../modules/experimental/src/index.ts', import.meta.url),
+  'utf8'
+);
+
+const transpiledBenchmark = typescript.transpileModule(benchmarkSource, {
+  compilerOptions: {
+    esModuleInterop: true,
+    jsx: typescript.JsxEmit.ReactJSX,
+    module: typescript.ModuleKind.CommonJS,
+    target: typescript.ScriptTarget.ES2022
+  }
+});
+
+const createDevice = vi.fn();
+const runLuvsBenchmark = vi.fn();
+let selectedDevice: Record<string, unknown> | undefined;
+let benchmarkPanelProps: MockLuvsBenchmarkPanelProps | undefined;
+
+const nativeRequire = createRequire(import.meta.url);
+function requireLuvsBenchmarkDependency(moduleName: string): unknown {
+  if (moduleName === './luvs-benchmark-runtime') {
+    return {
+      LUVS_BENCHMARK_MEASURED_ITERATIONS: 5,
+      LUVS_BENCHMARK_WARMUP_ITERATIONS: 1,
+      runLuvsBenchmark
+    };
+  }
+  if (moduleName === '../../react-luma/store/device-store') {
+    return {
+      createDevice,
+      useStore: (selector: (state: {presentationDevice?: unknown; device?: unknown}) => unknown) =>
+        selector({presentationDevice: selectedDevice})
+    };
+  }
+  if (moduleName === './live-benchmark-panel') {
+    return {
+      LiveBenchmarkPanel: (props: MockLuvsBenchmarkPanelProps) => {
+        benchmarkPanelProps = props;
+        return React.createElement(
+          'section',
+          null,
+          React.createElement('h3', null, props.title),
+          React.createElement('p', null, props.description),
+          React.createElement('button', null, props.runLabel)
+        );
+      }
+    };
+  }
+
+  return nativeRequire(moduleName);
+}
+
+const benchmarkModule: {exports: Record<string, React.ComponentType>} = {exports: {}};
+const loadLuvsBenchmark = new Function(
+  'require',
+  'module',
+  'exports',
+  transpiledBenchmark.outputText
+);
+loadLuvsBenchmark(requireLuvsBenchmarkDependency, benchmarkModule, benchmarkModule.exports);
+const LuvsBenchmark = benchmarkModule.exports.LuvsBenchmark;
+
+beforeEach(() => {
+  createDevice.mockReset();
+  runLuvsBenchmark.mockReset();
+  selectedDevice = undefined;
+  benchmarkPanelProps = undefined;
+});
+
+describe('luVS live vector-similarity benchmark documentation', () => {
+  test('publishes one optional package entry and embeds its reference in both sidebars', () => {
+    expect(experimentalPackage.exports['./luvs']).toMatchObject({
+      import: './dist/luvs/index.js',
+      types: './dist/luvs/index.d.ts'
+    });
+    expect(experimentalMainIndex).not.toContain("from './luvs");
+    expect(documentationSource).toContain(
+      "import {LuvsBenchmark} from '@site/src/components/docs/luvs-benchmark';"
+    );
+    expect(documentationSource).toContain('<LuvsBenchmark />');
+    expect(sidebarSource.match(/"api-reference\/experimental\/luvs"/g)).toHaveLength(2);
+    expect(navigationSource).toContain("href: '/docs/api-reference/experimental/luvs'");
+  });
+
+  test('documents fixed-size GPU table columns, Arrow ingestion, filters, and approximate IVF', () => {
+    expect(documentationSource).toContain("GPUVector<'fixed-size-list<float32,768>'>");
+    expect(documentationSource).toContain("from '@luma.gl/arrow'");
+    expect(documentationSource).toContain('makeGPUTableFromArrowTable');
+    expect(documentationSource).toContain('validityColumns');
+    expect(documentationSource).toContain('importGPUEmbeddingTable');
+    expect(documentationSource).toContain('Null source identifiers are');
+    expect(documentationSource).toContain('Nullable embedding data without a selected');
+    expect(documentationSource).not.toContain('makeGPUEmbeddingMatrixFromArrow');
+    expect(documentationSource).not.toContain('ownsValues');
+    expect(documentationSource).toContain('filterMask: selection.mask');
+    expect(documentationSource).toContain('GPUKMeans');
+    expect(documentationSource).toContain('GPUIVFFlatIndex');
+    expect(documentationSource).toContain('listRowIndices');
+    expect(documentationSource).toContain('traverse the selected inverted lists directly');
+    expect(documentationSource).toContain('bounded GPU hash');
+    expect(documentationSource).toContain('Float32 distance or inner product overflows');
+    expect(documentationSource).toContain('**approximate**');
+    expect(documentationSource).toContain('zero-copy');
+  });
+
+  test('server-renders every workload control without creating a GPU device or starting work', () => {
+    const markup = renderToString(React.createElement(LuvsBenchmark));
+
+    for (const label of [
+      'Dataset rows',
+      'Dimensions',
+      'Queries',
+      'Nearest neighbors (K)',
+      'Selected rows (%)',
+      'IVF lists',
+      'IVF probes',
+      'Run live CPU and WebGPU vector benchmark'
+    ]) {
+      expect(markup).toContain(label);
+    }
+    expect(markup).toContain('384');
+    expect(markup).toContain('768');
+    expect(markup).toContain('1,536');
+    expect(createDevice).not.toHaveBeenCalled();
+    expect(runLuvsBenchmark).not.toHaveBeenCalled();
+  });
+
+  test('requests a WebGPU device only when the reader starts the benchmark', async () => {
+    createDevice.mockRejectedValue(new Error('Deferred WebGPU device request'));
+    renderToString(React.createElement(LuvsBenchmark));
+
+    expect(createDevice).not.toHaveBeenCalled();
+    await expect(benchmarkPanelProps!.onRun()).rejects.toThrow('Deferred WebGPU device request');
+    expect(createDevice).toHaveBeenCalledOnce();
+    expect(createDevice).toHaveBeenCalledWith('webgpu-core');
+  });
+
+  test('renders verified CPU, exact, filtered, and IVF results after an explicit run', async () => {
+    selectedDevice = {type: 'webgpu'};
+    runLuvsBenchmark.mockResolvedValue({
+      results: [
+        {label: 'CPU exact', medianMilliseconds: 2, resultCount: 10, candidateCount: 2048},
+        {label: 'WebGPU exact', medianMilliseconds: 1, resultCount: 10, candidateCount: 2048},
+        {
+          label: 'WebGPU exact + selection',
+          medianMilliseconds: 0.8,
+          resultCount: 10,
+          candidateCount: 512
+        },
+        {
+          label: 'WebGPU IVF-flat + selection',
+          medianMilliseconds: 0.5,
+          resultCount: 10,
+          candidateCount: 128,
+          recall: 0.75
+        }
+      ],
+      uploadMilliseconds: 3,
+      indexBuildMilliseconds: 4,
+      indexByteLength: 4096,
+      options: {
+        datasetRowCount: 2048,
+        dimensions: 128,
+        queryCount: 4,
+        resultCount: 10,
+        filterPercentage: 25,
+        listCount: 8,
+        probeCount: 2
+      },
+      timestampQueries: false,
+      deviceLabel: 'Reader GPU'
+    });
+    renderToString(React.createElement(LuvsBenchmark));
+
+    const output = await benchmarkPanelProps!.onRun();
+    const markup = renderToString(output as React.ReactElement);
+
+    expect(createDevice).not.toHaveBeenCalled();
+    expect(runLuvsBenchmark).toHaveBeenCalledOnce();
+    expect(markup).toContain('CPU exact');
+    expect(markup).toContain('WebGPU exact + selection');
+    expect(markup).toContain('WebGPU IVF-flat + selection');
+    expect(markup).toContain('75.0%');
+    expect(markup).toContain('Recall@K');
+    expect(markup).toContain('Reader GPU');
+    expect(markup).toContain('completion fence');
+  });
+
+  test('runs independent CPU, exact, filtered, and IVF paths with fenced and separated timings', () => {
+    expect(benchmarkRuntimeSource).toContain('runCPUEmbeddingSearch(fixture, options)');
+    expect(benchmarkRuntimeSource).toContain(
+      'score = Math.fround(score + Math.fround(difference * difference))'
+    );
+    expect(benchmarkRuntimeSource).toContain('FLOAT32_RANKING_TOLERANCE');
+    expect(benchmarkRuntimeSource).toContain('new GPUTable({batches})');
+    expect(benchmarkRuntimeSource).toContain('new GPURecordBatch({');
+    expect(benchmarkRuntimeSource).toContain('fixed-size-list<float32,${dimensions}>');
+    expect(benchmarkRuntimeSource).toContain('importGPUEmbeddingTable(graph, dataset');
+    expect(benchmarkRuntimeSource).not.toContain('GPUEmbeddingMatrix');
+    expect(benchmarkRuntimeSource).toContain('new GPUSimilaritySearch({');
+    expect(benchmarkRuntimeSource).toContain('new GPUIVFFlatIndex({');
+    expect(benchmarkRuntimeSource).toContain('buffers.listRowIndices');
+    expect(benchmarkRuntimeSource).toContain("'list-row-indices'");
+    expect(benchmarkRuntimeSource).toContain('filterMask: importLuvsView(');
+    expect(benchmarkRuntimeSource).toContain("fallback: 'none'");
+    expect(benchmarkRuntimeSource).toContain(
+      'validateLuvsOutput(actual, oracle, label, approximate)'
+    );
+    expect(benchmarkRuntimeSource).toContain('device.submit(commandEncoder.finish())');
+    expect(benchmarkRuntimeSource).toContain('const fence = device.createFence()');
+    expect(benchmarkRuntimeSource).toContain('await fence.signaled');
+    expect(benchmarkRuntimeSource).toContain('execution.encoding.readTimings()');
+    expect(benchmarkRuntimeSource).toContain('uploadMilliseconds');
+    expect(benchmarkRuntimeSource).toContain('indexBuildMilliseconds');
+    expect(benchmarkRuntimeSource).toContain('readbackMilliseconds');
+    expect(benchmarkRuntimeSource).toContain('rerankMilliseconds');
+    expect(benchmarkSource).toContain('Recall@K');
+  });
+});