Skip to content

[FEATURE] GPU-accelerated video decoding support (NVIDIA/AMD/Intel) #266

Description

@ake-app

Which interface does this affect?

  • GUI (desktop app)
  • CLI (trcc commands)
  • REST API (trcc serve)
  • All / not sure

Device (if relevant)

All LCD/LED devices (feature is general to all supported hardware)

Is your feature request related to a problem?

Yes. Video playback on the LCD currently uses CPU-based decoding via FFmpeg. On systems with AMD, NVIDIA, or Intel GPUs, video decoding could be significantly more efficient by leveraging GPU video decoders (NVDEC, VAAPI) instead of using the CPU, which would:

  • Reduce CPU load during video playback on the LCD
  • Improve performance on lower-end systems
  • Better utilize existing GPU resources already being monitored by TRCC
  • Enable smooth playback of higher-bitrate video on resource-constrained devices

Describe the solution you'd like

Implement GPU-accelerated video decoding with graceful CPU fallback:

Current Implementation (CPU-only)

File: src/trcc/services/media.pyVideoDecoder class

The app currently pipes FFmpeg to raw RGB24 frames using CPU decoding:

class VideoDecoder:
    """Decode a video file to a list of in-memory RGB24 frames via ffmpeg."""
    
    def decode(self) -> list[RawFrame]:
        """Run ffmpeg, return the decoded frames."""
        cmd: list[str] = ["ffmpeg", "-hide_banner", "-loglevel", "error"]
        if self.rotation_degrees:
            cmd += ["-display_rotation", str(self.rotation_degrees)]
        if self.duration_s:
            cmd += ["-t", f"{self.duration_s:.3f}"]
        cmd += [
            "-i", str(self.path),
            "-r", str(self.fps),
            "-s", f"{w}x{h}",
            "-f", "rawvideo",
            "-pix_fmt", "rgb24",
            "pipe:1",
        ]
        proc = subprocess.run(cmd, check=True, capture_output=True)

The app already has GPU detection infrastructure:

File: src/trcc/adapters/sensors/hwmon.py — GPU discovery (lines 484-514)

def discover_amd_gpus(devices: list[HwmonDevice]) -> list[GpuSource]:
    """Find amdgpu hwmon entries, link them to /sys/class/drm cards."""
    gpus: list[GpuSource] = []
    for i, dev in enumerate(d for d in devices if d.driver == _AMD_DRIVER):
        gpus.append(AmdGpu(i, dev, _find_drm_card_for_hwmon(dev.path)))
    return gpus

def discover_intel_gpus(devices: list[HwmonDevice]) -> list[GpuSource]:
    """Find i915/xe hwmon entries."""
    # ... existing implementation

Proposed GPU Acceleration

Add FFmpeg hardware acceleration flags based on detected GPU:

NVIDIA (NVDEC):

ffmpeg -hwaccel cuda -i input.mp4 -f rawvideo -pix_fmt rgb24 pipe:1

AMD (VAAPI):

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -i input.mp4 -f rawvideo -pix_fmt rgb24 pipe:1

Intel (VAAPI):

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -i input.mp4 -f rawvideo -pix_fmt rgb24 pipe:1

Implementation Steps

  1. Extend VideoDecoder.__init__() to accept optional hwaccel: str | None parameter
  2. Detect available GPU decoders at startup (reuse existing discover_amd_gpus(), discover_intel_gpus(), and NVML infrastructure from src/trcc/adapters/sensors/nvml.py)
  3. Build FFmpeg command with appropriate -hwaccel flag (or none for CPU fallback)
  4. Wrap subprocess call with try/except to gracefully degrade to CPU decode on GPU failure
  5. Log acceleration status at INFO level:
    • "VideoDecoder: GPU-accelerated decode (NVIDIA NVDEC)"
    • "VideoDecoder: GPU-accelerated decode (AMD VAAPI)"
    • "VideoDecoder: CPU decode fallback (GPU unavailable or failed)"

Backward Compatibility

  • Seamless fallback to CPU decoding if GPU acceleration unavailable
  • No changes to public API or playback behavior
  • Existing code paths unaffected

Describe alternatives you've considered

  1. CPU-only (status quo) — works but suboptimal on systems with powerful GPUs
  2. GPU acceleration with error crash — risky; fallback is essential
  3. User opt-in toggle — adds UI complexity; auto-detection is simpler

Additional context

This feature complements existing GPU monitoring (temperature, usage, VRAM, clock speed). By reusing the GPU detection infrastructure already in place for metrics, video playback can leverage the same GPUs being displayed on the LCD.

Related code references:

  • src/trcc/adapters/sensors/hwmon.py — GPU discovery (hwmon-based)
  • src/trcc/adapters/sensors/nvml.py — NVIDIA GPU detection (NVML-based)
  • src/trcc/services/media.py — Video decoding service
  • src/trcc/services/video_export.py — Video export (also uses FFmpeg, would benefit)

Testing considerations:

  • NVIDIA GPU: test -hwaccel cuda with various video codecs (H.264, H.265, VP9)
  • AMD GPU: test -hwaccel vaapi with various codecs
  • Intel GPU: test -hwaccel vaapi (iGPU + Arc discrete)
  • Verify CPU fallback on systems without GPU
  • Verify CPU fallback when GPU acceleration fails
  • Performance benchmarks: CPU vs GPU decode (frame decode time, CPU %, memory)
  • Multi-GPU systems: ensure correct GPU selection

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions