Share Gaussian preparation and radix sort with Metal compute

[snkmcb]

Summary

Metal Gaussian rendering still prepares and sorts particles on the CPU. Move the existing Vulkan projection, covariance, SH, culling/compaction and portable radix-sort kernels into shared Slang sources, then compile Metal wrappers into the embedded and installed Gaussian library. A native Metal compute harness compares the complete preparation-to-sort submission with the CPU reference.

The transformed SH regression also exposed an existing GPU bug: row cofactors were used as inverse-matrix rows, applying the inverse transpose. Correct the shared calculation so directional SH matches the CPU reference under rotation and nonuniform scale.

Impact

• Shared shader algorithms retain backend-owned bindings and record access. Metal uses explicit buffer slots and byte-addressed 64-byte prepared records to avoid native float3 padding differences.
• Add host ABI assertions, generated binding/layout checks, SPIR-V compile coverage, artifact identity and installed compute-artifact checks. Preserve non-finite classification by disabling native Metal fast math for compute kernels.
• Renderer GPU scheduling is not enabled by this change. Metal still uses CPU preparation/sorting; prefer falls back and require rejects. Persistent attributes, frame scheduling and indirect raster integration remain open.
• No public API changes or new Core-only compiler dependency. Update the changelog, execution design, support matrix and current roadmap.

Verification

Local runtime environment: Apple M3, macOS 15.7.9, Xcode 16.4, Slang 2026.8.

• ctest --test-dir build-metal --output-on-failure: 24/24 passed, no skips (Debug).
• ctest --test-dir build/metal-compute-release --output-on-failure: 24/24 passed, no skips (fresh Release build).
• ctest --preset core: 17/17 passed.
• Existing Metal image/AOV/lifetime tests and install-tree consumers passed. New compute tests cover SH degrees 0–3, projection/sorting policies, transformed resources, non-finite inputs, near/far bounds, workgroup boundaries, multiple resources, hierarchical scans, device-written counts and empty/all-rejected input.
• Floating-point field tolerance is documented as 2e-4 * max(1, abs(reference)); identities, classifications, counters and sentinel placement match exactly. Intentional distance ties use binary-exact coordinates.
• Vulkan production wrappers compile to SPIR-V locally. Vulkan/HgiVulkan runtime validation is running in GPU capability CI for commit 880517d (queued at PR creation). Native viewport/HgiMetal comparisons and performance captures were not rerun for this change.

Review checklist

• The change follows the contribution guidelines.
• Observable behavior has tests or a documented reason tests are not needed.
• User-facing changes are recorded under Unreleased in the changelog.
• Support claims and command examples match validated behavior.