WebGPU render tests: canvas capture in headless CI needs --enable-gpu + Vulkan compositing + a display

[chrisgervang]

A place to work out WebGPU render testing. Found while trying to enable a WebGPU render test in deck.gl (deck.gl#10520); it affects luma's SnapshotTestRunner identically, since both capture by screenshotting the canvas region.

Problem

In a default headless Chromium configuration, a WebGPU canvas renders but captures as blank, so golden-image diffs see a white frame regardless of what was drawn. WebGL is unaffected.

webgl   toDataURL=16384   screenshot=16384
webgpu  toDataURL=    0   screenshot=    0     (128x128 canvas cleared red)

This is a compositor interop problem, not a renderer problem. Dawn runs on Vulkan-SwiftShader while the compositor defaults to ANGLE/GL-SwiftShader, and the WebGPU swapchain image never crosses that boundary.

Working configuration

The WebGPU flags are additive to the existing ones, not a replacement:

xvfb-run -a   +   headless: true   +   stock Playwright chromium

--use-angle=swiftshader --enable-unsafe-swiftshader             (existing)
--enable-unsafe-webgpu --ignore-gpu-blocklist
--enable-gpu --enable-features=Vulkan --use-vulkan=swiftshader

⚠️ Keep --use-angle=swiftshader. Dropping it shifts WebGL rasterization enough to fail column-lnglat-extruded-wireframe at 97.88% — a real regression that only shows up when running a full suite, not a single case.

No pinning is required: stock Playwright Chromium, headless: true, no executablePath. The only new dependency is xvfb-run, already present on ubuntu-22.04 runners.

Two independent conditions are required, on top of Vulkan compositing — measured by isolating them:

DISPLAY	--enable-gpu	red px captured
none	yes	0
none	no	0
Xvfb	yes	16384
Xvfb	no	0

--enable-gpu stops headless forcing software rendering; the resulting driver autodetection needs an X display on Linux. Without a display WebGL is unaffected and WebGPU simply captures blank, so the flags are safe on macOS and for local runs without xvfb.

In the broken configurations there is a second symptom: calling getCurrentTexture() kills the device outright (A valid external Instance reference no longer exists, Dawn's DeviceLostReason::InstanceDropped), and every later operation fails, including readback from unrelated offscreen textures. It is not a lifetime or GC artifact — retaining the adapter and device references, and drawing to the acquired texture, change nothing. In working configurations the device survives.

Upstream status

This is not an open upstream limitation. GPU support in headless was fixed in crbug 40540071 (2024), whose closing comment describes exactly the mechanism above — --enable-gpu disabling forced software rendering, and Linux autodetection requiring an X display. crbug 40274484 was fixed in 2025. The remaining referenced trackers are macOS-specific.

Nor is SwiftShader going away for this use case. docs/gpu/swiftshader.md scopes the deprecation to the automatic WebGL fallback and states the headless/GPU-less testing use case "will still be supported by opting in"; --use-vulkan=swiftshader is a documented switch.

Status in deck.gl

The recipe is wired into deck's render project and CI in deck.gl@241d28b. The webgpu row of the LineLayer case is enabled knowing it currently fails, so the remaining gap is visible and tracked rather than invisible. CI already uploads *-fail.png and *-diff.png on failure, which makes the diff image the running measurement.

Repro of the failure

Raw WebGPU, no luma.gl, in the default headless configuration. Clears an offscreen texture red and reads it back, varying only how far we touch an unrelated canvas context first. node repro.mjs, needs playwright + chromium.

import {chromium} from 'playwright';
import {createServer} from 'node:http';

// Served over localhost because WebGPU needs a secure context - about:blank and
// page.setContent() both leave navigator.gpu undefined, which looks exactly like
// "no WebGPU support" and is worth knowing before you debug it for an hour.
const server = createServer((_, r) => {
  r.writeHead(200, {'Content-Type': 'text/html'});
  r.end('');
}).listen(0);

const browser = await chromium.launch({
  args: [
    '--enable-unsafe-webgpu',
    '--ignore-gpu-blocklist',
    '--use-angle=swiftshader',
    '--enable-unsafe-swiftshader'
  ]
});
const page = await browser.newPage();
await page.goto(`http://localhost:${server.address().port}/`);

for (const stage of ['none', 'configure', 'getCurrentTexture']) {
  const result = await page.evaluate(async stage => {
    const canvas = document.createElement('canvas');
    canvas.width = 128;
    canvas.height = 128;
    document.body.appendChild(canvas);

    const device = await (await navigator.gpu.requestAdapter()).requestDevice();
    if (stage !== 'none') {
      const ctx = canvas.getContext('webgpu');
      ctx.configure({device, format: navigator.gpu.getPreferredCanvasFormat()});
      if (stage === 'getCurrentTexture') ctx.getCurrentTexture();
    }

    // Clear an *offscreen* texture red and read it back - the canvas is never the render target,
    // so a failure here means touching the canvas context killed the device.
    const texture = device.createTexture({
      size: [128, 128],
      format: 'rgba8unorm',
      usage: GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.COPY_SRC
    });
    const buffer = device.createBuffer({
      size: 512 * 128,
      usage: GPUBufferUsage.COPY_DST | GPUBufferUsage.MAP_READ
    });
    const enc = device.createCommandEncoder();
    enc.beginRenderPass({
      colorAttachments: [
        {
          view: texture.createView(),
          clearValue: {r: 1, g: 0, b: 0, a: 1},
          loadOp: 'clear',
          storeOp: 'store'
        }
      ]
    }).end();
    enc.copyTextureToBuffer({texture}, {buffer, bytesPerRow: 512}, {width: 128, height: 128});
    device.queue.submit([enc.finish()]);

    try {
      await buffer.mapAsync(GPUMapMode.READ);
      return `pixel [${[...new Uint8Array(buffer.getMappedRange())].slice(0, 4)}]`;
    } catch (e) {
      return String(e);
    }
  }, stage);
  console.log(`canvas ${stage.padEnd(17)} -> ${result}`);
}

await browser.close();
server.close();

canvas none              -> pixel [255,0,0,255]
canvas configure         -> pixel [255,0,0,255]
canvas getCurrentTexture -> AbortError: A valid external Instance reference no longer exists.

Result in deck's harness

Capture is fixed. WebGPU renders the full frame at correct scale and colour — a 185 KB PNG where the blank frame was 2.4 KB. The full render suite is 169 passing with the same 4 pre-existing network failures, so nothing else moved.

The two backends do not yet agree pixel-for-pixel: 83.25% against a 99% threshold. What that is and is not:

• Ink coverage: golden (WebGL) 89,992 px vs WebGPU 73,778 px — WebGPU is crisper, with more fully-solid and fewer partial pixels. MSAA-resolved WebGL and analytically-antialiased WebGPU rasterize 2px lines differently.
• It is not a misalignment: shifting the WebGPU image by ±1px in each direction peaks at 84.98%, versus 83.25% unshifted. A systematic offset would have jumped to ~97%. There is a slight vertical bias worth a separate look.

The WebGPU ports are still in progress and convergence is the intent, so this number is a measure of remaining porting distance rather than a property of the approach.

Current state

• luma: SnapshotTestRunner captures via getBoundingBoxInPage(canvas) + screenshot (modules/test-utils/src/snapshot-test-runner.ts:46-58). Note test/render/render.spec.ts is not wired to any npm script or CI workflow today, which is probably why this never surfaced.
• deck: the vitest render suite does the same. Before the recipe, WebGPU cases returned pure white, which reads as an 84% match and looks like an antialiasing regression rather than an infrastructure failure.
• Offscreen rendering plus copyTextureToBuffer has worked throughout, in every configuration. I used it to find a real deck bug this week — deck emitted WebGL-convention clip depth (-w <= z <= w) where WebGPU wants 0 <= z <= w, so everything was clipped (deck.gl@d7996ed).

Options

A. Vulkan compositor + Xvfb, keep canvas screenshots

The only option that tests the real presentation path — swapchain format (bgra8unorm vs rgba8unorm), alphaMode: 'premultiplied', DPR, resize, compositing. Not academic: deck's premultiplied handling exists because of canvas alphaMode. Needs no test rewrites, and is already working in deck.

Against: xvfb-run has to exist in CI images and on dev machines, which splits local from CI and excludes macOS devs. Version drift is a real risk — the recipe is a supported configuration rather than a documented contract, and its failure mode is a blank frame that reads as a rendering regression rather than as broken infrastructure. Slow: SwiftShader Vulkan plus a PNG round-trip.

If we take this path, add a canary: a trivial known-red-pixel capture asserted before the suite runs, so a future Chromium regression fails as "capture is broken" rather than as dozens of mismatched goldens.

B. Offscreen render target + readback

Hermetic and deterministic — no compositor, no window manager, no region math, no browser chrome in frame, no Xvfb. Identical headless on every OS, so local matches CI. Gives the actual rendered texture rather than a PNG the compositor may have colour-managed or rescaled. Unlocks numeric assertions image diffing cannot express. Same mechanism on both backends.

Against: tests none of the presentation path — everything in A's favour goes uncovered. Some tests genuinely need a canvas (picking, interaction, default-framebuffer reads).

C. Real GPU runners

The only option that validates real driver behavior, real MSAA, real precision; software rasterizers hide driver bugs, and presentation just works.

Against: GitHub-hosted runners have no GPUs, so self-hosted or third-party. Goldens become GPU-dependent — different vendors rasterize differently, a notorious maintenance sink.

Suggested shape

B for the bulk of the suite, plus a small A-based smoke suite covering presentation. B gives speed, determinism and assertions image diffing cannot make; a handful of A cases keep the swapchain/alphaMode/DPR path guarded. Either alone leaves a real gap. C as an optional nightly only if per-GPU baselines are acceptable.

For reference, MapLibre Native sidesteps all of this: its WebGPU backend is Dawn built from source, and mbgl-render-test-runner renders into a headless frontend reading back to mbgl::PremultipliedImage. No browser, no compositor.

Open questions

• Is test/render/ meant to be revived for v10, or superseded? That decides where this work lands.
• How should we track backend convergence while the port lands — keep the case enabled and failing as a visible measurement (what deck does today), or gate it behind a temporary per-backend threshold that ratchets down?
• Should this hang off #2743 or #2550, or stand alone?

Happy to prototype either the offscreen capture path or the Xvfb wiring, whichever direction you prefer.