Skip to content

Add audio-sync waveform panel - #432

Open
RufanMelfor wants to merge 1 commit into
reco-project:mainfrom
RufanMelfor:feat/audio-sync-waveform
Open

Add audio-sync waveform panel#432
RufanMelfor wants to merge 1 commit into
reco-project:mainfrom
RufanMelfor:feat/audio-sync-waveform

Conversation

@RufanMelfor

@RufanMelfor RufanMelfor commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Adds a collapsible "Audio Sync Waveform" panel to the calibration
controls, showing overlaid left/right peak-amplitude waveforms for a
window of audio around the playhead. A sync_offset frame error shows
up as a visible shift between the two traces, instead of requiring a
guess-and-check loop against the rendered preview.

  • reco-io: calibration_io::extract_audio_pcm_window() extracts a
    windowed audio segment (ffmpeg -ss/-t) instead of always reading
    from the start of the file; extract_audio_pcm() now delegates to it.
  • reco-gui: new waveform module downsamples PCM into a smoothed
    peak envelope, normalized per L/R pair so a real mic-level mismatch
    between the two cameras stays visible rather than being independently
    stretched away. A background thread re-extracts it as the playhead
    moves (throttled, polled from the always-running timer tick so it
    keeps updating while paused, not only while playing/seeking).
  • Window width (frames around the playhead) and panel height are
    live-adjustable.
  • Changing sync_offset forces a recompute so the panel reflects the
    new offset instead of showing a stale envelope.

This is a focused port of the audio-sync waveform panel only - it does
not include the separate "auto-detect sync offset" feature (IMU/audio
cross-correlation), which is unrelated UI/logic and can be its own PR
if wanted.

Test plan

  • cargo fmt --all -- --check
  • cargo check -p reco-gui --all-targets (full link skipped locally
    to avoid clobbering a running dev build; cargo build was
    confirmed to succeed up to the link step)
  • cargo test -p reco-gui -p reco-io — all passing, including 12 new
    waveform::tests unit tests
  • cargo clippy -p reco-gui -p reco-io --all-targets -- -D warnings
    — no new findings; failures shown are the same pre-existing
    reco-core dead-code/unsafe-ptr issues already tracked by Clear pre-existing clippy -D warnings failures on Windows #423
Audio Sync Wavefrom

Adds a collapsible "Audio Sync Waveform" panel to the calibration
controls showing overlaid L/R peak-amplitude envelopes for a window
of audio around the playhead, so a sync_offset frame error shows up
as a visible shift between the two traces instead of requiring a
guess-and-check loop against the rendered preview.

- reco-io: calibration_io::extract_audio_pcm_window() extracts a
  windowed audio segment (ffmpeg -ss/-t) instead of always reading
  from the start of the file; extract_audio_pcm() now delegates to it.
- reco-gui: new waveform module downsamples PCM into a smoothed
  peak envelope, normalized per L/R pair so a real mic-level mismatch
  stays visible instead of being independently stretched away. A
  background thread re-extracts it as the playhead moves (throttled,
  polled from the always-running timer tick so it keeps updating
  while paused, not just while playing/seeking).
- Window width (frames around the playhead) and panel height are
  live-adjustable via two LineEdits.
- Changing sync_offset forces a recompute so the panel reflects the
  new offset instead of showing a stale envelope.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

All contributors have signed the CLA. Thank you!
Posted by the CLA Assistant Lite bot.

@RufanMelfor

Copy link
Copy Markdown
Contributor Author

I have read the CLA Document and I hereby sign the CLA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

2 participants