Skip to content

getWaveformPortion corrupts audio buffer & leaks audio samples on negative start time offsets #7604

Description

@codeCraft-Ritik

Describe the bug

Describe the bug

When using getWaveformPortion or visualizeAudioWaveform with an audio track that starts at an offset (for example, audio starting at t = 2.0s with a visualization window requested at t = 0), getWaveformPortion corrupts the returned waveform buffer.

In packages/media-utils/src/get-waveform-portion.ts:

const clampedStart = Math.max(startSample, 0);
const clampedEnd = Math.min(waveform.length, endSample);

const padStart = samplesBeforeStart > 0 ? new Float32Array(samplesBeforeStart).fill(0) : null;
const padEnd = samplesAfterEnd > 0 ? new Float32Array(samplesAfterEnd).fill(0) : null;
const arrs = [padStart, waveform.slice(clampedStart, clampedEnd), padEnd].filter(NoReactInternals.truthy);

When endSample < 0 (e.g. startSample = -2000, endSample = -1500), clampedEnd evaluates to -1500. In JavaScript, Float32Array.prototype.slice(0, -1500) treats negative numbers as offsets from the end of the array. For a 10,000-sample track, slice(0, -1500) returns 8,500 samples of actual audio data instead of silence. In addition, samplesBeforeStart and samplesAfterEnd use unbounded differences, resulting in an oversized buffer containing 10,500 samples when only 500 samples of silence were requested.

Steps To Reproduce

Steps To Reproduce

  1. Call getWaveformPortion with a window in negative time before audio start:
import { getWaveformPortion } from '@remotion/media-utils';

const portion = getWaveformPortion({
  audioData: {
    sampleRate: 1000,
    numberOfChannels: 1,
    durationInSeconds: 10,
    channelWaveforms: [new Float32Array(10000).fill(0.9)],
  },
  startTimeInSeconds: 0,
  durationInSeconds: 0.5,
  dataOffsetInSeconds: 2.0, // Audio starts at 2s, window is -2s to -1.5s
  numberOfSamples: 50,
});
  1. Inspect the returned bars: they contain amplitude 0.9 (leaked from the audio track) instead of 0.

Expected behavior

Expected behavior

When requesting audio data before an audio track begins, getWaveformPortion should return an array consisting entirely of 0 amplitude padding matching the exact duration requested. clampedStart and clampedEnd must be bounded to [0, waveform.length].

Suggested Fix

--- a/packages/media-utils/src/get-waveform-portion.ts
+++ b/packages/media-utils/src/get-waveform-portion.ts
@@ -63,14 +63,14 @@ export const getWaveformPortion = ({
 			audioData.sampleRate,
 	);
 
-	const samplesBeforeStart = 0 - startSample;
-	const samplesAfterEnd = endSample - waveform.length;
+	const padStartLength = Math.max(0, Math.min(endSample, 0) - startSample);
+	const clampedStart = Math.max(0, Math.min(waveform.length, startSample));
+	const clampedEnd = Math.max(0, Math.min(waveform.length, endSample));
+	const padEndLength = Math.max(0, endSample - Math.max(startSample, waveform.length));
 
-	const clampedStart = Math.max(startSample, 0);
-	const clampedEnd = Math.min(waveform.length, endSample);
-
 	const padStart =
-		samplesBeforeStart > 0
-			? new Float32Array(samplesBeforeStart).fill(0)
+		padStartLength > 0
+			? new Float32Array(padStartLength).fill(0)
 			: null;
 	const padEnd =
-		samplesAfterEnd > 0 ? new Float32Array(samplesAfterEnd).fill(0) : null;
+		padEndLength > 0 ? new Float32Array(padEndLength).fill(0) : null;
 	const arrs = [

Packages

@remotion/media-utils

Additional Context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions