Skip to content

feat: Automatic scene complexity-based encoding quality adaptation #57

Description

@wmehanna

Problem

Currently, BitBonsai applies uniform encoding quality (CRF) across entire videos. This is inefficient because different scenes have vastly different encoding complexity:

  • Simple scenes (static content, low detail): Need fewer bits — current approach wastes bits here
  • Complex scenes (high motion, grain, high detail): Need more bits to maintain quality — current approach may under-encode here

Proposed Solution

Implement automatic scene complexity analysis before/during encoding that:

  1. Detect scene changes using FFmpeg's scene detection filter to build a scene complexity map
  2. Classify scene difficulty based on spatial/temporal complexity metrics
  3. Adapt CRF per-segment: Apply lower CRF (higher quality) to complex scenes, higher CRF (lower quality, more compression) to simple scenes — all while maintaining overall bitrate efficiency
  4. Use FFmpeg's per-title encoding pattern: Generate weighted encoding complexity analysis that informs the encoding pass

This is transparent to users — no configuration needed.

Why This Matters

  • Better quality at same bitrate: Bits allocated where eyes are most sensitive
  • Better compression at same quality: Simple scenes compressed aggressively without perceptual loss
  • Industry-standard technique: Netflix and others use per-scene complexity analysis extensively
  • Complements existing VMAF validation: After encoding, VMAF already validates quality; this makes the encoding itself smarter

Technical Approach

In FfmpegService:

  1. Add analyzeSceneComplexity(inputPath) method using scene detection filter
  2. Build complexity profile: segments with complexity scores (0-1)
  3. Map complexity to CRF adjustments: baseCRF + (1 - complexity) * crfDelta
  4. Use FFmpeg segment filtering or two-pass approach to apply per-segment CRF

Example complexity classification:

  • Complexity 0.0-0.3: Static title screens, fade transitions — CRF +4 (aggressive compression)
  • Complexity 0.3-0.7: Normal content — CRF base (no change)
  • Complexity 0.7-1.0: Action sequences, grainy film, CGI — CRF -4 (protect quality)

Labels

enhancement, video-encoding

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions