Halve export lookahead VRAM via 8-bit downconvert - #431
Open
RufanMelfor wants to merge 2 commits into
Open
Conversation
…bit downconvert The lookahead pool's VRAM cost scales with source bit depth; 10-bit DJI footage with AI tracking enabled could exceed the budget on smaller GPUs (see FRICTION.md for the full root-cause writeup). Adds an opt-in LookaheadBitDepth::Reduced8Bit mode that downconverts each buffered frame to 8-bit NV12 via a small GPU render pass (lookahead_downconvert.wgsl) instead of the default bit-exact copy. Implemented for both the cross-platform VramPool (Linux/macOS) and Windows' D3D11 staging/import path (D3d11StagingPool::plane_source + an explicit device.poll to close a cross-API sync gap between the D3D11 write and the wgpu DX12 read of the same shared-handle slot). Verified end-to-end with a real export against 3840x2880 10-bit HEVC DJI footage + live AI tracking on an RTX 3060 Ti: VRAM budget dropped from "needs 4.71 GB" to "needs 2.36 GB" (exactly half) with a clean 150/150 frame encode, no hangs or corruption. Linux/macOS path is code-reviewed and test-covered but not live-verified this session (no such hardware available) - see FRICTION.md. Wired through StitchJob::lookahead_reduced_bit_depth(bool), the CLI --lookahead-reduced-bit-depth flag, and a reco-gui export panel checkbox. Default is Native (bit-exact), so existing behavior is unchanged unless a caller opts in. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…gin/main The Tip component is defined by a separate fork-only commit not included in this cherry-pick. Keep the plain CheckBox to match origin/main's UI conventions; the explanatory text moves to a comment.
|
All contributors have signed the CLA. Thank you! |
Contributor
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an opt-in export setting that buffers the lookahead window at 8-bit
instead of the source's native bit depth, roughly halving its VRAM cost on
10-bit footage (e.g. DJI Action 4 HEVC). This lets a longer lookahead fit on
lower-VRAM cards; sources that are already 8-bit are unaffected.
render::lookahead_downconvertGPU pass (WGSL shader) that downconvertsbuffered lookahead frames to 8-bit before they enter the VRAM pool.
--export-lookahead-8bitCLI flag and a matching GUI checkbox under thelookahead/smoothness slider in the export panel.
little gradient smoothness (sky, grass, floodlit surfaces) for memory —
intended for VRAM-constrained setups, off by default.
Test plan
cargo fmt --all -- --checkcargo build --workspace --all-targets(excludingreco-obs/rig-calib,which require GStreamer/libcamera dev packages not present in this
environment)
cargo test --workspace— all passing tests green; the twointerop::cudatests andmatroska_reader_sees_partial_writesfailidentically on plain
main/origin/main(pre-existingenvironment/timing issues, unrelated to this change)
render::lookahead_downconvert::tests::y_plane_downconvert_matches_expected_8bit_values