Build Resilient Multi-Layered AudioDecoder Preprocessor Fallback Pipeline

[purvanshjoshi]

Technical Overview

src/preprocess.py relies on standard torchaudio or dynamic library codecs to decode raw audio signals. In minimal runtime environments or virtual systems where OS-level FFmpeg codecs are missing, standard audio loading throws unhandled decoding errors.

Affected Modules

• File: src/utils.py & src/preprocess.py

Acceptance Criteria

1. Platform Independence: Preprocessing succeeds even in minimalist systems lacking native OS-level media codecs.
2. Format Support: Processes various target formats (WAV, MP3, M4A) and standardizes them to 16kHz mono.
3. Silent Fallback: Automatically tries alternative Python-native decoders if the default C++ system libraries fail.

Proposed Implementation Approach

Implement a multi-layered fallback loop within our dataset audio parsing module:

def load_audio_resilient(audio_path, target_sr=16000):
    try:
        # Layer 1: Attempt standard torchaudio loading
        import torchaudio
        waveform, sr = torchaudio.load(audio_path)
        # Resample if needed
        # ...
        return waveform
    except Exception as e1:
        try:
            # Layer 2: Fall back to native soundfile loading
            import soundfile as sf
            data, sr = sf.read(audio_path)
            # ...
            return torch.tensor(data).float()
        except Exception as e2:
            # Layer 3: Final fallback to pydub/librosa if installed
            import librosa
            data, sr = librosa.load(audio_path, sr=target_sr)
            return torch.tensor(data)

Severity & Priority

• Severity: Medium (Enhances cross-environment compatibility)
• Priority: P2

[Saksham-dev-del]

I would like to work on this issue. I will implement a load_audio_resilient function in src/utils.py/src/preprocess.py that:

Attempts standard torchaudio loading first.
Falls back to soundfile if that fails.
Falls back further to librosa as a final layer.
Standardizes all outputs to 16kHz mono, supporting WAV, MP3, and M4A formats.

I will test this across environments with and without native OS-level FFmpeg codecs to confirm platform independence.

Could you please assign this issue to me? I will submit a PR for review once complete

[purvanshjoshi]

Hi @Saksham-dev-del,

Thank you for the proposal. After review, we've decided not to assign this issue in its present form. Our src/preprocess.py already implements a layered decoding strategy for Hugging Face Audio columns, and we're cautious about introducing a torchaudio → soundfile → librosa chain that falls back silently: silent fallbacks can mask real decode failures and make behavior hard to reproduce across environments.

For contributors, a more aligned approach: keep a single primary decoder (torchaudio) and add an explicit, clearly logged fallback layer that raises meaningful warnings on each level change (never silent), standardize to 16 kHz mono at one well-defined boundary, and cover the pipeline with unit tests using WAV/MP3/M4A fixtures both with and without OS-level codecs present.

We're currently assigning you 2 issues (#6 and #9). This issue remains open for other contributors to take on.