[FEATURE] Integrated inference without external API

[fazo96]

Feature Description

I found this project whisper-rs which in theory should allow waystt to not depend on any external API provider and just run the whisper model locally. Modern machines should have no trouble doing so

Problem Statement

I am trying to build a setup where waystt works offline and does not disclose my voice to a cloud API. For now I am relying on #7 and https://speaches.ai/ but I recently discovered whisper-rs

Proposed Solution

whisper-rs crate is used to provide bindings to whisper.cpp so waystt can transcribe the text without external programs

Alternative Solutions

I started with #7 but requires setting up a local server separately for waystt to use

Use Case

Privacy-sensitive individuals, saving on API costs, and offline use.

Implementation Ideas

I am not skilled enough in rust to implement this but I will give it a shot with AI assistance. My idea is to have a separate setting to choose a model file. If a model file is provided, waystt will use that model instead of the OpenAI API. The local model use via whisper-rs can be implemented as a new provider

Additional Context

None

[sevos]

This is a great idea fully aligned with this project goals!

Please give me a heads up if you'd decide not to implement this.

Enjoy learning Rust! :)

[sevos]

Additionaly, please make use of TRANSCRIPTION_PROVIDER env variable. I use it already for switching between OpenAI and Google API.

Let's say local or local_whishper would go with whishper_rs

[fazo96]

I hope to find some time to get this built :) Thanks for the support. I'll definitely use the existing structures in the project for configuring etc.

[abenz1267]

I'd definitely appreciate a local model!

[wanderer]

also waiting on this!