Skip to content

Integrate Metal Gaussian GPU execution into renderer - #114

Merged
snkmcb merged 1 commit into
mainfrom
codex/metal-gaussian-renderer
Sep 27, 2026
Merged

snkmcb merged 1 commit into
mainfrom
codex/metal-gaussian-renderer

Conversation

@snkmcb

@snkmcb snkmcb commented Sep 27, 2026

Copy link
Copy Markdown
Member

Summary

Metal previously exposed GPU Gaussian preparation, sorting and gather only through a private test harness. Renderer frames still prepared/sorted particles on the CPU, prefer fell back, and require rejected execution. This change connects immutable attribute residency and the completion-safe executor to renderer frame contexts and indirect raster. GPU camera frames perform no CPU particle preparation, prepared-stream upload or warmed GPU allocation.

  • Encode attribute updates, preparation, radix sort, gather and raster in one submission. Resolve small per-resource counters only after completion, without CPU scheduling dependencies between GPU stages.
  • Preserve leases across in-flight frames and AOV exports; invalidate dependent residency after upload failure. Drain abandoned commands on C++ exceptions, including failures after successful compute encoding.
  • Expose bounded residency/scratch pools, attribute/raster telemetry, and Metal dispatch/device-copy statistics. Add Hydra environment and viewport CLI budget overrides for larger scenes.
  • Keep CPU rendering as the default/reference. SortedStream supports prefer and require; Metal Tiled falls back to GPU SortedStream in prefer and reports Unsupported in require. Capacity failures fall back to CPU in prefer and report the original error in require.

Impact

  • Areas: Metal, Hydra, native viewport, tests and documentation. Core and shared shader algorithms are unchanged.
  • Adds Metal-specific options/statistics; no native API types enter Core. Default Gaussian residency and scratch budgets are independently 1 GiB, separate from the mesh/texture heap.
  • Updates the changelog, support matrix, execution design, build guide and remaining roadmap work.
  • Tile raster, per-stage GPU timestamps, broad host image/performance parity and actual device-command failure injection remain unverified or unimplemented.

Verification

Apple M3, macOS 15.7.9, Apple clang 17, Slang 2026.8. Debug uses the existing macOS 14.5 SDK configuration; Release uses the Xcode 16.4 SDK. Runtime tests ran outside the sandbox with Metal API/GPU validation enabled.

cmake --build build-metal --parallel 4
MTL_DEBUG_LAYER=1 MTL_SHADER_VALIDATION=1 ctest --test-dir build-metal --output-on-failure
# 25/25 passed, no skips

cmake --build build/metal-compute-release --parallel 4
MTL_DEBUG_LAYER=1 MTL_SHADER_VALIDATION=1 ctest --test-dir build/metal-compute-release --output-on-failure
# 25/25 passed, no skips

# Hydra/viewport Release: hdMerlin, merlin-viewport and relevant test targets built.
MTL_DEBUG_LAYER=1 MTL_SHADER_VALIDATION=1 ctest \
  --test-dir build/cy2026-macos-arm64-py313-usd--viewport-usd \
  -R 'merlin-hydra2-(render-settings|discovery|render-buffer|hgi-metal-bridge)$|merlin-viewport-settings$' \
  --output-on-failure
# 5/5 passed, no skips

build/metal-compute-release/tests/merlin-metal-gaussian-tests --benchmark
# 148 CSV samples across CPU/GPU, 64K/1M particles, cold/static/camera/local edit

git diff --check
# passed

Renderer tests compare RGBA8 within 2 channel units and exact depth/resource/particle IDs through camera motion, resize, mesh composition, edits, visibility, removal/reintroduction and all-rejected frames. Coverage includes range-only uploads, source replacement, unresolved submissions, AOV leases, budget fallback/recovery and post-compute render failure. The install consumer executes the embedded GPU path. CLI budget parsing rejects zero, negative and overflowing values.

At 1,048,576 particles and 1024x768 without image readback, 12-sample camera wall-time medians were 133.5 ms CPU / 29.0 ms GPU. Static medians were 13.4 ms CPU / 28.8 ms GPU because GPU frames still recompute while the CPU path reuses its prepared stream. These are local synthetic measurements, not a universal performance claim; backend construction and source edits are outside the timer. Raw measurements remain local under the report policy.

Interactive checks displayed a local 5,834,784-particle SH-degree-3 Garden stage in both usdview/HgiMetal GPU-copy presentation and the dedicated Metal viewport, using a 3 GiB residency budget. Telemetry confirmed GPU raster, zero CPU preparation and zero steady attribute uploads. usdview also exercised Tiled Prefer fallback. The scene and screenshots are not included, and these checks do not establish general host parity.

Existing SDK-header/deprecation and duplicate-library warnings remain; no new Gaussian-code warning was observed. Vulkan runtime and non-Apple hardware were not exercised locally.

Review checklist

  • The change follows the contribution guidelines.
  • Observable behavior has tests or a documented reason tests are not needed.
  • User-facing changes are recorded under Unreleased in the changelog.
  • Support claims and command examples match validated behavior.

@snkmcb
snkmcb merged commit f7794e5 into main Sep 27, 2026
16 checks passed
@snkmcb
snkmcb deleted the codex/metal-gaussian-renderer branch September 27, 2026 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant