Seven of the eleven providers are already just CompatSpec consts in apps/desktop/src-tauri/src/database/services/ai/compat.rs (OpenAI, Groq, DeepSeek, Kimi, GLM, Qwen, OpenRouter) — base URL, model list URL and sampling defaults, no client code. The natural next step is to let a user supply that spec themselves instead of waiting on a release to add their endpoint.
Covers: self-hosted vLLM / llama.cpp / LM Studio / TGI, Together, Fireworks, Cerebras, Azure OpenAI deployments, corporate gateways.
Proposed work:
- New
AIProvider::Custom arm whose spec is read from settings rather than a const: base URL, optional models URL, default model, temperature.
- Store the key through the existing key pool (
key_pool.rs) under a CUSTOM prefix so rotation and ai_keys_test_provider work unchanged.
build_client returns OpenAiCompatClient with the runtime spec — no new client type.- Frontend: a "Custom (OpenAI-compatible)" entry in
PROVIDER_OPTIONS (ai-provider-section.tsx) that reveals base-URL and model fields; reuse the existing key-test button for validation.
- Model list: fall back to free-text model entry when the endpoint has no
/models.
- Usage pricing (
usage.rs) has no rates for a custom endpoint — record tokens, show cost as unknown.
Depends on nothing; this is mostly config plumbing.
Seven of the eleven providers are already just
CompatSpecconsts inapps/desktop/src-tauri/src/database/services/ai/compat.rs(OpenAI, Groq, DeepSeek, Kimi, GLM, Qwen, OpenRouter) — base URL, model list URL and sampling defaults, no client code. The natural next step is to let a user supply that spec themselves instead of waiting on a release to add their endpoint.Covers: self-hosted vLLM / llama.cpp / LM Studio / TGI, Together, Fireworks, Cerebras, Azure OpenAI deployments, corporate gateways.
Proposed work:
AIProvider::Customarm whose spec is read from settings rather than a const: base URL, optional models URL, default model, temperature.key_pool.rs) under aCUSTOMprefix so rotation andai_keys_test_providerwork unchanged.build_clientreturnsOpenAiCompatClientwith the runtime spec — no new client type.PROVIDER_OPTIONS(ai-provider-section.tsx) that reveals base-URL and model fields; reuse the existing key-test button for validation./models.usage.rs) has no rates for a custom endpoint — record tokens, show cost as unknown.Depends on nothing; this is mostly config plumbing.