feat(experimental): add table-backed GPU k-means and IVF-flat

[ibgreen-openai]

Goals

Add deterministic WebGPU k-means clustering and genuinely indexed IVF-flat approximate similarity search over borrowed, chunk-preserving fixed-size-list GPU table columns. Explain the abstractions, their purpose, ownership boundaries, and accuracy/performance tradeoffs directly in this PR.

Changes

• Add a standalone 500-line luVS reference and experimental overview/navigation in the same PR as the algorithms.
• Explain why vector similarity is useful in visualization, why embeddings remain ordinary GPU table columns, and why application-facing stable IDs must remain distinct from logical source rows.
• Document exact-search versus indexed-search complexity, deterministic WebGPU k-means without Float32 atomics, convergence, empty clusters, and explicit graph execution.
• Include a worked IVF inverted-list example showing centroid labels, counts, prefix offsets, stable source IDs, and sorted logical row references.
• Explain probe count, recall@K, restrictive-filter fallback, persistent index memory requirements, binding limits, explicit rebuilds, and recovery after WebGPU device loss.
• Implement bounded Lloyd iterations with deterministic seeding, preserved empty-cluster centroids, GPU labels/status, and segmented Float32 reductions.
• Build IVF-flat centroids, counts, prefix offsets, persisted stable IDs, and independently stored logical source-row references inside caller-owned command graphs.
• Binary-search sorted per-list row references and rerank only candidates selected by the probed inverted lists.
• Preserve source identity when a prebuilt index is imported into another graph; validate direct graph descriptors and input/output alias boundaries before allocation.
• Support configurable lists/probes, bounded query-output tiling, caller-owned masks, explicit restrictive-filter fallback, empty inputs, zero K, and every exact metric.
• Keep index buffers caller-owned while borrowing standard GPUTable/GPUVector embedding storage.

Verification

• nvm use (Node 22.22.1).
• yarn install: the configured package registry returns HTTP 403 for required tarballs; verification reuses the existing isolated dependency tree and CI performs the normal immutable install.
• Focused k-means and index-directed IVF Node and actual Chromium/WebGPU suites.
• Both standalone documentation pages compile as MDX.
• Documentation/attribution regressions verify conceptual explanations, IVF identity layout, ownership, and device-loss guidance.
• yarn lint fix.
• yarn build after the final implementation and formatting changes.
• Full Node pre-commit suite: 1,131 passed, one skipped.
• Previous full actual Chromium/WebGPU coverage for the complete stack.
• yarn website:build.
• (cd website && yarn build).

Risks

Reduced IVF probing is approximate by design; each persistent packed index array must fit one storage binding and oversized indexes fail before graph mutation. An unrelated existing Billion-Point Spatial Atlas visual smoke intermittently loses its SwiftShader device on both this branch and master; its failure happens before luVS coverage starts and is being investigated separately.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a4d74873a6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".