Repository navigation
feat(detect): NVDEC frame source via ffmpeg pipe with cv2 fallback - #18
Merged
Merged
Conversation
CI has neither the face ONNX nor the centerface extra; the logging test ran detect_and_track with a real detector and failed on ModuleNotFoundError (deface). The nvdec parity test had the same gap, hidden by its skip.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Stage 1 (
detect_and_track) is the long pole of every GoMoveShift drive: ~3.2 h for a 48-min 4K60 HEVC clip at 15-20 fps while the GPU sits near 30%. The loop reads every frame throughcv2.VideoCapture(CPU decode; the installed cv2 has no CUDA). Measured on this box, 240 frames of a real 4K60 HEVC master while a drive was running:-hwaccel cuda -c:v hevc_cuvid) piped bgr24, this PRSame shape, same fps and frame-count metadata, same sampled frame indices.
What
redactcam/frames.py(new):iter_frames(path, *, decode="auto", start_frame=0, capture=None) -> (FrameMeta, iterator). Metadata still fromcv2.VideoCapture. Backendscv2(unchanged behaviour) andnvdec(ffmpeg pipe; codec from ffprobe →hevc_cuvid/h264_cuvid/av1_cuvid).read_fullloopsreadintountil a whole frame; zero bytes = end of stream; a partial frame or an early end (nonzero exit, or >1% short of the container's count) raisesFrameSourceError, never a silent truncation. Close: stdout closed, 10 s wait, terminate, kill.auto(and an explicitnvdec) fall back to cv2 with one INFO line when: ffmpeg missing, no-hwaccel cuda, no*_cuviddecoder for the codec, the 1-frame probe fails, the pipe yields no first frame, the clip carries a rotate tag / display matrix (not proven to apply under cuda), ffprobe and cv2 disagree on frame size, orstart_frame > 0(checkpoint resume; NVDEC path does not seek).REDACTCAM_DECODE=cv2|nvdec|autooverrides.detect.detect_and_trackanddetect_regionsconsume frames through it (decode=argument, default auto). Return types unchanged. Chosen backend logged (detect: decode=…) only while progress logging is on.Tests
tests/test_frames.py(new): cv2/nvdec parity on synthetic h264 and hevc clips (frame_count, shape, dtype, mean abs diff < 2), short-read pipe reader, partial final frame raises,autofallback when ffmpeg is missing or the probe fails, env override, identical sampled indices indetect_and_trackfor both backends (stub detector). NVDEC tests skip with a reason where cuda/cuvid is absent; on this box they ran.ImportErrorforredactcam.frames(.data/logs/nvdec-red.log).nvdec-green.log); dispatcher re-run unpiped: 292 passed (nvdec-verify.log).ruff check .clean.ruff format --checkflags 15 files, 9 untouched by this PR: the venv's ruff differs from the repo's pin; nothing reformatted.test_zero_emits_no_progress_lines) caught the new log line firing with progress off; the code was fixed, not the test.Not done: seeking for
start_frame > 0on the NVDEC path (ffmpeg-ssaccurate seek is a follow-up); rotated clips stay on cv2;coverage.pyandtimeline.pystill opencv2.VideoCapturedirectly. Diff 611 sum.