fix(video): move AV1 planar conversion onto Metal

[bobsupra]

Summary

Moves AetherEngine's common AV1 planar 4:2:0 conversion from CPU sws_scale to a small Metal compute path while preserving the existing CVPixelBuffer and SampleBufferRenderer pipeline.

This was found through NuvioTVOS #40. On an Apple TV 4K running tvOS 26.6, a 3840x2160 23.976 fps AV1 MKV with TrueHD 5.1 routed through dav1d software decode, measured about 464% process CPU, and held a 0.0 s display cushion. The same source played smoothly in Stremio's KSPlayer pipeline. Aether's software decoder was performing a second full-frame planar YUV -> NV12/P010 CPU conversion after dav1d.

Implementation

• Adds a session-scoped MetalYUVConverter for AV1 YUV420P, YUVJ420P, and YUV420P10LE.
• Uploads the three source planes into reused Metal textures.
• Writes NV12/P010 into the existing IOSurface-backed CVPixelBuffer pool.
• Preserves color/SAR attachments, sample-buffer rendering, subtitles, seeking, PiP, and frame pacing.
• Handles FFmpeg's low-aligned 10-bit samples and writes high-aligned P010 values.
• Falls back to sws_scale for unsupported layouts, invalid/negative strides, missing Metal, texture/shader failures, or GPU command failures.
• Disables a failed Metal path for the remainder of the session to avoid retrying every frame.

Tests

• swift test --filter MetalYUVConversionPolicyTests
  • Real Metal NV12 conversion with byte readback.
  • Real Metal P010 conversion with byte readback and alignment checks.
  • Format-routing and negative-stride fallback coverage.
• Full swift test: 2,415 tests in 329 suites passed.
• Nuvio host builds passed for generic tvOS Simulator and generic physical tvOS destinations.

Device verification status

The original high-CPU failure was captured on Apple TV 4K / tvOS 26.6. The converter and exact output were exercised on a macOS Metal device, and the tvOS targets compile and link, but the affected source still needs a post-change physical Apple TV performance retest. Opening as draft for that device validation and maintainer feedback.

[superuser404notfound]

Thank you for digging into this, and for a writeup that made the branch easy to read. The fallback design, the session-scoped disable and the format policy are careful work, and the two pixel-buffer-pool findings buried in it are real (more on those below).

The premise does not survive measurement, though, so it is worth settling that before the device retest.

Benchmarked on an M1 (macOS 26.5.2), 3840x2160, 60 warm iterations, comparing what convertFrameToPixelBuffer does on main against MetalYUVConverter.convert from this branch, same source frame, same destination pool format:

path	YUV420P to NV12	YUV420P10LE to P010
sws_scale (current)	0.374 ms CPU, 0.374 ms wall	1.579 ms wall
Metal (this branch)	3.598 ms CPU, 7.398 ms wall	7.868 ms wall

At 23.976 fps the conversion this PR removes costs 0.9 % of one core. It cannot be a measurable share of a 464 % process reading, so what is burning the CPU in NuvioTVOS #40 is dav1d decoding 4K AV1, not the conversion that follows it.

And the replacement is the more expensive path: roughly 10x the CPU and 20x the wall time of sws_scale. 2.170 ms of the 3.598 ms CPU is the three MTLTexture.replace calls on their own. A .shared texture from texture2DDescriptor has a tiled layout, so replace is a full-plane CPU swizzle, and all three planes are copied before the GPU sees anything. waitUntilCompleted then parks the decode thread for 7.4 ms per frame, which is 17.7 % of that thread's wall budget at 23.976 fps and serializes the decode loop behind the GPU.

The shape that would pay is the one that deletes the upload instead of adding it: a custom get_buffer2 on the AVCodecContext handing dav1d page-aligned, MTLBuffer-backed storage, so decoded planes are already GPU-visible and the compute pass reads them with no copy, plus a completion handler rather than a blocking wait. That is a much larger change, and it would still only be chasing 0.9 %.

For the underlying report, the measurement worth taking on the device is dav1d itself. A Time Profiler capture on the Apple TV will attribute the 464 % honestly, and thread_count plus dav1d's frame delay are the levers that sit there. Worth checking too whether the comparison player produces an NV12 CVPixelBuffer at all, or renders the planar frame straight through its own Metal display shader: that would be a renderer-level difference rather than a conversion-level one, and it is not a trade available here, because AVSampleBufferDisplayLayer is what PiP, the subtitle overlay, the HDR attachments and frame pacing all hang off.

Two pieces of this branch are worth having regardless of the Metal path, and I would happily take them as their own PR:

• latching use10Bit from the frame's pixel format, so a 10-bit frame whose codecpar never carried bits_per_raw_sample is not truncated into an 8-bit pool
• keying the pixel buffer pool on bit depth (poolIs10Bit). Latent on main, since use10Bit is fixed at open today, and a genuine hole the moment anything flips it, which the first item does

Two notes for the record, since they are divergences from sws_scale rather than optimizations: the kernels clamp video-range samples to 16..235 and 16..240 where sws_scale copies them verbatim, so legal sub-16 and super-235 excursions get crushed; and frame.color_range == AVCOL_RANGE_JPEG on a plain YUV420P frame triggers a full-to-limited compression on the Metal path that swscale does not perform for that format, so the fast path and its own fallback would disagree on pixels inside one session.

Leaving this open as a draft rather than closing it, in case the zero-copy direction is interesting. The benchmark is a small test target addition and easy to share if it helps.

[bobsupra]

Thanks so much for the detailed benchmark, profiling breakdown, and insights. That makes total sense—the planar conversion is only ~0.9% of a core, so the 400%+ CPU is indeed overwhelmingly dav1d software decoding 4K AV1 frames on devices lacking hardware AV1 decode in VideoToolbox.

Closing this PR and deleting the branch. We'll split the 10-bit pool latching and bit-depth pool keying fixes into a separate, clean PR.

If you have any recommendations or thoughts on dav1d configuration (e.g. thread_count, frame delay, buffering/preroll, or frame-dropping strategies) for constrained/tvOS environments handling high-bitrate 4K AV1 software decoding, we'd love any guidance you might have!