Ahead-of-time API requests to improve overvall performance? #448
|
Currently, by configuring Read Aloud to use OpenAI compatible API endpoints, I'm able to generate sound by using my own https://github.com/remsky/Kokoro-FastAPI instance. But because of limit of hardware power, it must be wait a while before each paragraph is read in turn. Is there a way that while reading this paragraph, the other paragraphs after that will be requested first and everything can be seamless? Is it possible to set some interval for that.? |
Replies: 1 comment
|
All the engines are already doing prefetch, i. e. synthesizing the next section while the current one is being read. You can look in tts-engines.js However, for this to work, the inference must be faster than realtime (realtime factor < 1) . Otherwise the next section wont finish synthesizing before the current one finishes playing. Kokoro is quite demanding, unfortunately. See discussion in ken107/piper-browser-extension#10 |
All the engines are already doing prefetch, i. e. synthesizing the next section while the current one is being read. You can look in tts-engines.js
However, for this to work, the inference must be faster than realtime (realtime factor < 1) . Otherwise the next section wont finish synthesizing before the current one finishes playing.
Kokoro is quite demanding, unfortunately. See discussion in ken107/piper-browser-extension#10