Feature Request: Add SenseVoice/Paraformer as ASR engine option

[LauraGPT]

Note

License and capability clarification (2026-07-14): FunASR is a toolkit, not a single checkpoint. The FunASR and SenseVoice repository source code is MIT; model weights follow each model card. SenseVoiceSmall supports Chinese, Cantonese, English, Japanese, and Korean, and its weights use the linked FunASR Model Open Source License Agreement. Fun-ASR-Nano-2512 is Apache-2.0. Language coverage, punctuation, and performance depend on the selected model and runtime configuration.

Feature Request

obs-localvocal provides local speech recognition for OBS. SenseVoice and Paraformer from FunASR would be strong alternative engines.

Why

• Non-autoregressive — SenseVoice (234M params) is designed for real-time scenarios, much faster than autoregressive models
• 50+ languages with automatic language detection
• ONNX models available — SenseVoice and Paraformer both have ONNX exports ready for use
• Already integrated in Sherpa-ONNX (5K+ stars) which provides C/C++ API
• Built-in VAD (FSMN-VAD) — lightweight voice activity detection
• Built-in punctuation — auto-adds punctuation to captions

Integration Path

Since obs-localvocal likely uses C/C++ (OBS plugin), Sherpa-ONNX provides:

• C API for SenseVoice and Paraformer inference
• Real-time streaming support
• Cross-platform (Windows, macOS, Linux)
• Pre-built libraries

Performance

SenseVoice achieves real-time or better inference on CPU, making it suitable for live OBS captioning without GPU requirements.

Happy to help with integration!

[Tabby]

Honestly, sounds rad ^_^
I'd be very up for it, and am very curious to see how it compares - only caveat being that it would probably need a chunk of refactoring to allow the model used to be switched from Whisper to something else (and tbh I think a lot of this plugin wants refactoring anyway to make it easier to maintain/extend)

I'm not likely to have capacity to start thinking about non-trivial changes until mid to late August I'm afraid, but after that I can make some time :)

Of course, if you're up for starting to have a go at doing that before then please do! I can either join in when I've got more capacity, or do reviews depending on how far you get

[LauraGPT]

Thanks @Tabby, that makes sense. I agree the first useful slice should probably be a small refactor rather than dropping a second model into the current Whisper path.

I took a quick source pass so the proposal is less hand-wavy. The current coupling points I see are:

• src/transcription-filter-data.h stores Whisper-specific runtime state directly (whisper_context, whisper_full_params, whisper_model_path, whisper_model_file_currently_loaded).
• src/transcription-filter.cpp treats gf->whisper_context == nullptr as the generic “ASR is not ready” check before accepting audio.
• src/whisper-utils/whisper-processing.cpp owns both model loading (init_whisper_context) and inference (run_whisper_inference).
• src/tests/localvocal-offline-test.cpp builds the current path directly around Whisper helpers.

A low-risk first PR could therefore be a facade-only refactor, with no SenseVoice option exposed yet:

• keep Whisper as the default and preserve current settings/behavior;
• add a tiny transcription-engine facade with a single supported engine value, whisper;
• route “is ASR loaded?” and “run transcription on this PCM segment” through that facade, but have the facade call the existing Whisper helpers internally;
• leave model download, UI settings, packaging, and the actual SenseVoice/Paraformer runtime for a follow-up PR;
• add one focused test/smoke path proving the facade returns the same DetectionResultWithText shape the rest of the filter already consumes.

Then the SenseVoice/Paraformer PR can plug into the facade using sherpa-onnx first, which looks like the best fit for OBS/C++ packaging. That should make review easier: first PR is maintainability only; second PR is the new ASR engine.

I will avoid starting a large branch until the interface direction is clear, but this is the shape I would use if I take a first pass.

[Tabby]

That sounds like a pretty sensible approach to me and is more or less what I had in mind too :)

I think it would make sense to add a drop-down selector for which model to use to the settings UI in the first PR, but just have it default to whisper and have it disabled. I think this would be the easiest way to have the facade default to using Whisper, by using the value from that setting as it will eventually have to do anyway. But that's the only thing I can think of that I'd change about your plan