q5-webgpu: Looping graphics instances is disabled

[quinton-ashley]

In q5 WebGPU you can use createGraphics to create static imagery, but looping is disabled by default. Why?

createGraphics(w, h, C2D)

Currently, for WebGPU to display any changes made to graphics objects backed by Canvas2D, copyExternalImageToTexture must be run. This function has a huge performance cost. Running it multiple times every frame is extremely slow, resulting in huge frame drops as the amount of graphics objects increases above a meager 20 (as tested in Chrome on my MacBook Air M1 2020).

I worked around this with C2D backed images in WebGPU by only copying the C2D canvas to image.texture (its WebGPU texture) when the image is modified. Image modification functions like filter set img.modified = true, then when the image is displayed by the image function its canvas content is copied to the texture which is displayed on the webgpu canvas, then img.modified is set to false so this only occurs when necessary.

I think that's fine for images, but users expect to be able to edit graphics objects as often every frame in typical use.

There are plans among web browser developers for WebGPU to be able to display Canvas2D canvases without incurring the performance cost of making a copy to a separate texture. I'm super excited about this!

https://github.com/fserb/canvas2D/blob/master/spec/webgpu.md

As soon as it's rolled out, I'll implement it.

createGraphics(w, h, WEBGPU)

createGraphics can also be backed by WebGPU but performance is still pretty bad. Every graphics object has its own set of buffers for colors, transforms, vertices, etc. With several graphics objects in use, frame drops are intermittent due to a lot of GC. If the user creates too many graphics objects (>50 on my MacBook Air M1 2020), it's only a matter of time before a crash occurs.

I think improving their performance would require a big structural change, having a q5 WebGPU instance and all its graphics objects use the same buffers, but that's not something I want to do.

Another issue is somehow there's something wrong with semi-transparent coloring or blending when the texture is displayed.

See the semi-transparent magenta colored rect in this example. The graphics object should look exactly the same, but it looks darker.

let q = await Q5.WebGPU();
// let q = new Q5();

Q5.experimental = true;

createCanvas(400);
displayMode('maxed');
colorMode(RGB, 1);

let g = createGraphics(100, 100, WEBGPU);
g.colorMode(RGB, 1);
g.background(1, 0, 1, 0.5);

q.draw = () => {
	background(0.8);

	fill(0.2);
	circle(mouseX, mouseY, 100);

	noStroke();
	fill(1, 0, 1, 0.5);
	rect(0, 0, 100, 100);

	g.text(g.frameCount, -50 + g.frameCount * 10, 0);

	image(g, 100, 0);

	// save(g, 'test.png');

	noLoop();
};

q.mousePressed = () => {
	redraw();
};

I'd like to know why this happens. Maybe it has something to do with the WebGPU canvas alphaMode being premultiplied. My guess is whatever conversion is needed for it to render properly would have a significant performance cost though.