Summary
Would it be possible to add an optional setting that sends an early request to the AI backend the moment a user clicks a button that will need it — the same way OpenWebUI already does — so that a sleeping AI machine has time to wake up before the actual request goes out?
Background
A very common self-hosting pattern looks like this: a small, always-on, low-power box (mini-PC, NAS, Raspberry Pi, …) runs the front-end and lightweight services, while the heavy AI work runs on a separate machine with a proper GPU. Since a GPU box idles at high power draw and runs hot and loud, a lot of people don't keep it on 24/7 — instead the small server wakes it on demand via Wake-on-LAN and lets it sleep the rest of the time.
This really isn't an edge case in the self-hosting/homelab crowd. For anyone running local AI on a GPU they'd rather not power around the clock, "small always-on host + GPU machine woken on demand" is one of the standard setups, simply because it saves a meaningful amount of power.
How OpenWebUI handles it
OpenWebUI already plays nicely with this: it makes a model-related request as soon as the page is opened (fetching the available models), which hits the backend and triggers the wake. By the time you actually send a prompt, the GPU machine is already up or well on its way.
On Dictate it doesn't have to make sth. model specific, just a general call would already be sufficient ir even better if models differ between prompts.
Current behaviour in Dictate
Dictate only contacts the AI backend when the prompt is actually submitted. In a WoL setup that means the machine only starts waking at submit time, so the full boot/wake latency gets tacked onto that first request (or it times out) — even though Dictate already knows the backend will be needed the moment the relevant button is clicked.
Feature request
It would be great to have an optional setting that fires an early, lightweight "warm-up" request to the backend as soon as a button is clicked that will require AI — e.g. like OpenWebUI does — so the machine gets a head start on waking up.
Two things I'd suggest keeping in mind:
- It should be optional / a setting (off by default), since it only makes sense for people running a wake-on-demand setup.
- Ideally it should only fire when a revision is requested, not automatically on every transcription — otherwise the GPU machine would get woken far more often than it needs to be.
Thanks a lot for Dictate — I'd be really happy to see this added, and I think it would make the app noticeably nicer to use for the selfhosted folks.
Summary
Would it be possible to add an optional setting that sends an early request to the AI backend the moment a user clicks a button that will need it — the same way OpenWebUI already does — so that a sleeping AI machine has time to wake up before the actual request goes out?
Background
A very common self-hosting pattern looks like this: a small, always-on, low-power box (mini-PC, NAS, Raspberry Pi, …) runs the front-end and lightweight services, while the heavy AI work runs on a separate machine with a proper GPU. Since a GPU box idles at high power draw and runs hot and loud, a lot of people don't keep it on 24/7 — instead the small server wakes it on demand via Wake-on-LAN and lets it sleep the rest of the time.
This really isn't an edge case in the self-hosting/homelab crowd. For anyone running local AI on a GPU they'd rather not power around the clock, "small always-on host + GPU machine woken on demand" is one of the standard setups, simply because it saves a meaningful amount of power.
How OpenWebUI handles it
OpenWebUI already plays nicely with this: it makes a model-related request as soon as the page is opened (fetching the available models), which hits the backend and triggers the wake. By the time you actually send a prompt, the GPU machine is already up or well on its way.
On Dictate it doesn't have to make sth. model specific, just a general call would already be sufficient ir even better if models differ between prompts.
Current behaviour in Dictate
Dictate only contacts the AI backend when the prompt is actually submitted. In a WoL setup that means the machine only starts waking at submit time, so the full boot/wake latency gets tacked onto that first request (or it times out) — even though Dictate already knows the backend will be needed the moment the relevant button is clicked.
Feature request
It would be great to have an optional setting that fires an early, lightweight "warm-up" request to the backend as soon as a button is clicked that will require AI — e.g. like OpenWebUI does — so the machine gets a head start on waking up.
Two things I'd suggest keeping in mind:
Thanks a lot for Dictate — I'd be really happy to see this added, and I think it would make the app noticeably nicer to use for the selfhosted folks.