WebGPU: allow sampling multisampled render targets (custom MSAA resolves for HDR color + depth)

[SashaRX]

Area: graphics · WebGPU

Problem

On WebGPU there is currently no supported way to bind a multisampled render
target as a texture in a user shader. Two things block a custom MSAA resolve:

1. The multisampled COLOR buffer is created with RENDER_ATTACHMENT usage only —
   there is no option to add TEXTURE_BINDING, so its samples can never be read
   in a shader. See WebgpuRenderTarget.initColor
   (src/platform/graphics/webgpu/webgpu-render-target.js):

   const multisampledTextureDesc = {
     size: [width, height, 1], dimension: '2d', sampleCount: samples, format,
     usage: transientColor
       ? GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.TRANSIENT_ATTACHMENT
       : GPUTextureUsage.RENDER_ATTACHMENT   // <- no TEXTURE_BINDING option
   };

2. BindGroupFormat/BindTextureFormat cannot express a multisampled texture
   binding. BindTextureFormat's constructor has no multisampled parameter, and
   WebgpuBindGroupFormat hardcodes it off
   (src/platform/graphics/webgpu/webgpu-bind-group-format.js):

   const multisampled = false;   // <- never true
   ...
   texture: { sampleType: gpuSampleType, viewDimension, multisampled }

   So even though the WGSL reflector already recognises texture_multisampled_2d
   and texture_depth_multisampled_2d
   (webgpu-shader-processor-wgsl.js, textureBaseInfo), a shader that binds one
   gets a bind-group layout that says multisampled: false → validation error.

The engine already does exactly this resolve internally — WebgpuResolver
(src/platform/graphics/webgpu/webgpu-resolver.js) binds
texture_depth_multisampled_2d, does textureLoad(img, coord, 0u), and runs a raw
layout: "auto" pipeline that bypasses BindGroupFormat. The capability exists;
it just isn't reachable from user code. (Internal MS depth buffers even opt into
TEXTURE_BINDING already — else if (samples > 1) in the depth init path — but MS
color never does, and neither is bindable through a BindGroupFormat.)

This is the failure class the WebGPU spec deliberately pushed to userland: there is
no resolveTarget for depth (gpuweb/gpuweb#108), and engines that need per-sample
resolves do a manual pass — the same approach Godot took for its depth resolve
(godotengine/godot#80991).

Use case

A planetary-atmosphere HDR pipeline (Hillaire 2020 scattering on shell geometry,
WebGPU + WebGL2) needs custom MSAA resolves that the hardware box resolve
cannot provide:

• Color: a reversible / policy-driven color resolve (the UE4/Karis
  "reversible tonemap" resolve family). On clean frames we tonemap-in-resolve
  per sample straight to LDR; when HDR post passes still follow (volumetric fog,
  aerial perspective) we need a coverage-linear per-sample mean kept in HDR so the
  later composite scene·T + L stays energy-consistent with the resolved depth.
  The hardware resolve gives neither.
• Depth: depth-consuming post passes (fog, aerial perspective) need the
  multisampled scene depth resolved into custom channels — coverage-weighted mean,
  MIN (nearest-geometry classifier), and coverage — not sample 0 and not an
  averaged depth. That requires reading all samples of the MS depth buffer in a
  shader.

Current workaround (honest)

1. Monkey-patch device.wgpu.createTexture to OR TEXTURE_BINDING into the usage
   of multisampled HDR-color and depth textures before the scene RT initialises.
2. Run the resolves as raw-WebGPU fullscreen passes with a hand-built
   createBindGroupLayout({ texture: { multisampled: true, sampleType: 'unfilterable-float' } }), bypassing BindGroupFormat entirely (exactly as the
   internal WebgpuResolver does), encoded into the engine's shared frame command
   encoder via getCommandEncoder() so ordering vs. the scene render is correct.

This works but reaches into private engine internals (rt.impl.colorAttachments[0] .multisampledBuffer, texture.impl.gpuTexture, getCommandEncoder) and breaks any
time those shapes change.

Proposed API

• A RenderTarget/Texture opt-in to make the multisampled color buffer
  bindable, e.g. new RenderTarget({ ..., samples: 4, bindMultisampled: true })
  (or a texture flag), which ORs TEXTURE_BINDING into the MS color descriptor —
  mirroring the TEXTURE_BINDING the depth path already sets.
• Multisampled support in BindGroupFormat: a multisampled flag on
  BindTextureFormat (and matching texture_depth_multisampled_2d), so
  WebgpuBindGroupFormat can emit multisampled: true for
  texture_multisampled_2d / texture_depth_multisampled_2d bindings the WGSL
  reflector already understands. A TEXTUREDIMENSION_2D_MS enum could carry it, or
  a boolean on the format — the reflection side is already in place.
• Optionally, expose the existing WebgpuResolver depth-resolve entry point (or a
  general per-sample resolve hook) so users don't re-implement the raw pipeline.

Prior internal art that already does most of this: #5485 (custom shader-based MS
depth resolve), #6917/#6932 (partial MS-RT support). Adjacent precedent for exposing
WebGPU texture-bind controls to user shaders: #8734.

Happy to contribute a PR if the shape is agreed.

[mvaligursky]

Thanks again for the detailed writeup. We've been reviewing the design space here, and rather than the minimal opt-in proposed above, we want to expose the full WebGPU capability surface. Posting the direction we intend to take — subject to change during development.

Design direction: first-class multisampled textures

Instead of an RT-level bindMultisampled flag with an accessor, Texture gains a samples option (WebGPU-gated), and RenderTarget accepts multisampled textures directly as attachments, with optional per-attachment resolve targets:

// usage = RENDER_ATTACHMENT | TEXTURE_BINDING; validated: no mips, no arrays, no upload/lock
const msColor = new Texture(device, { format: PIXELFORMAT_RGBA16F, width, height, samples: 4 });
const msId    = new Texture(device, { format: PIXELFORMAT_RGBA16U, width, height, samples: 4 });
const msDepth = new Texture(device, { format: PIXELFORMAT_DEPTH32F, width, height, samples: 4 });

const rt = new RenderTarget({
    colorBuffers:   [msColor, msId],     // samples inferred; must match across attachments
    resolveBuffers: [resolveTex, null],  // per attachment; null = no hardware resolve
    depthBuffer:    msDepth              // bindable as texture_depth_multisampled_2d
});

The existing path (single-sampled colorBuffer + samples: 4 → internally allocated MSAA buffer + automatic resolve) remains unchanged as sugar over the explicit form — and remains the only form on WebGL2.

Why this shape instead of the flag:

• Multisample and resolve capability are independent per format. Integer formats (rgba8uint, rgba16uint, ...) and r32float are multisample-capable but not resolve-capable — an MSAA attachment in these formats cannot legally have a resolveTarget, so "MS attachment with no resolve texture" must be expressible. A design where the MS buffer is an internal companion of a resolve-target colorBuffer cannot reach that format class at all.
• Everything interesting is per-texture / per-attachment in WebGPU — bindable, transient and copyable are usage flags on the texture; resolveTarget is optional per attachment. Global RT flags are the wrong granularity, and each new one multiplies flag interactions.
• Stable identity — the user owns the MS textures, so nothing gets recreated behind their back on resize / context restore; no wrapper texture, no "fetch it again" rules.
• Room for the rest of the capability surface: MS↔MS copies (WebGPU allows them: equal sample counts, full subresource) ride the normal copy API; stencil-aspect reads (texture_multisampled_2d<u32>) ride future aspect-view support; a future depth-resolve extension becomes a resolveDepthBuffer slot; ExpandResolveTexture-style loadOps slot into the per-attachment RenderPass color ops we already have.

Notes:

• MS textures can only be filled by rendering (WebGPU requires RENDER_ATTACHMENT usage and forbids STORAGE_BINDING, mips and array layers), so the Texture surface will validate accordingly.
• Related per-sample pipeline state (WGSL sample_mask in/out builtins, GPUMultisampleState.mask / alphaToCoverageEnabled) is on the same track but separate work.

Landing plan

1. The WGSL reflection + BindGroupFormat layer from Allow sampling multisampled render targets on WebGPU #9167 (texture_multisampled_2d / texture_depth_multisampled_2d reflection, multisampled bind-group entries, sample-type coercion) lands first, separately — any design shape needs it. See the comment on the PR.
2. The Texture / RenderTarget surface above follows.