Custom model providers are unreachable: wire the store, and stop the picker discarding their models - #333
Conversation
Every custom-provider request returned 404. The store is built in wiring and declared on both the app options and ApiDeps, but the routes read ctx.deps from ServerOptions, and index.ts never passed it there, so ctx.deps.customProviders was always undefined and the guard at the top of each handler rejected the request. The feature is unreachable in any deployment. refreshCustomProviders goes with it: the handlers call it after a successful upsert to rehydrate the in-process registry, so without it a saved provider would not resolve until the next restart.
A model served by a custom provider never appears in the web UI picker. getBaseModel clones a template only when the catalogue entry's provider is the literal string "openrouter"; a custom provider's slug cannot be that, because the built-in provider ids are reserved. So it throws, and buildOption's catch discards the option silently: the model is absent from the picker with no error, while the server has already returned it in modelCatalog and modelsByHarness. The catalogue is the server's statement of which models this deployment may use. Honour it for any entry rather than for one hardcoded provider name.
| if (template) return cloneModel(template, id, clone.name); | ||
| } | ||
| if (fallback?.provider === "openrouter") { | ||
| if (fallback) { |
There was a problem hiding this comment.
This makes the model selectable, but it also clones all of openrouter/auto’s transport identity. I reproduced the full runtime-options → transcript path with an Anthropic-compatible custom model: the selected model and reconstructed assistant message had provider: "openrouter", api: "openai-completions", and the OpenRouter base URL. /api/turn sends only model.id, so core routing remains correct, but the client transcript metadata and provider-derived defaults are wrong. Please carry the custom provider/protocol metadata into the client model (and cover both custom protocols in the web tests) rather than retaining the OpenRouter template identity.
There was a problem hiding this comment.
Good catch, and the diagnosis was exactly right — thanks for reproducing the whole path rather than just the symptom.
Fixed in 87757b5. The protocol and endpoint already existed server-side on CustomRuntimeModel; they were simply dropped at the catalogue boundary, which is why the client had nothing better to go on than the template. customModelCatalog() now carries api and baseUrl, ModelCatalogEntry and the runtime-config payload pass them through, and getBaseModel picks the template matching the declared protocol and then overrides provider, api and baseUrl with the model's own.
So an Anthropic-protocol custom model now clones an Anthropic template and reports its own provider slug and endpoint, rather than presenting as OpenRouter over openai-completions.
An entry with no declared protocol keeps the previous openai-completions assumption, so built-in dynamic OpenRouter models are unchanged.
Tests: both custom protocols plus the unspecified-protocol default in plugins/web-ui/test/pi-models.test.ts, including an explicit assertion that an anthropic-messages model is never attributed to an openai-completions transport. The catalogue round-trip test in test/custom-providers.test.ts now asserts the protocol fields it carries.
Cloning openrouter/auto made a custom model selectable but gave it the template's transport identity: an Anthropic-protocol custom model arrived in the client as provider "openrouter" with api "openai-completions" and OpenRouter's base URL. Core routing was unaffected, since /api/turn sends only model.id, but the transcript metadata and any provider-derived default read from the model were wrong. The protocol and endpoint already exist server-side on CustomRuntimeModel and were being dropped at the catalogue boundary. They now travel with the entry, so the client clones the template matching the declared protocol and overrides provider, api and baseUrl with the model's own. An entry with no declared protocol keeps the previous OpenAI-completions assumption. Tests cover both custom protocols and the unspecified-protocol default, and assert an Anthropic-protocol model is never attributed to an OpenAI-completions transport. The catalogue round-trip test now asserts the protocol fields it carries.
Custom model providers cannot be used at all. Two independent defects sit between a registered provider and a usable model, and neither reports an error.
1. The store never reaches the routes
putCustomProvider,getCustomProvidersanddeleteCustomProviderall begin withctx.depsisServerOptions.createCustomProviderStoreis built inwiring.ts, returned on the built object, and declared on bothApiDepsand thecreateAppoptions — butindex.tspasses it only tocreateApp, never tocreateServer. Soctx.deps.customProvidersis alwaysundefinedand every request 404s.refreshCustomProvidershas the same gap and matters just as much: the handlers call it after a successful upsert to rehydrate the in-process registry, so without it a saved provider would not resolve until the next restart.Reproduce:
PUT /v1/admin/custom-providers/<id>with a valid spec. Returns 404 on any deployment.2. The picker discards models it was told about
With the store wired, a provider saves, the row persists, and
/api/runtime-configcorrectly returns the model in bothmodelsByHarnessandmodelCatalog— but the model is still absent from the web UI picker.getBaseModelclones a template only when the catalogue entry's provider is the literal string"openrouter":A custom provider's slug cannot be
"openrouter"—validateCustomProviderSpecrejects the built-in provider ids as reserved. SogetBaseModelthrows,buildOption'scatch { return null }swallows it, and the option is dropped with no error in the console or the server log.The catalogue is the server's statement of which models this deployment may use. This honours it for any entry rather than for one hardcoded provider name. The change is strictly more permissive and affects only models the server already advertised.
Reproduce: register a custom provider, add its model id to the scope's picker list, and observe it missing from the composer while
runtime-configlists it.Verification
Both fixes applied to a running instance: the
PUTreturns 200, the provider persists, the model resolves throughresolveModel, passesmodelSupportedByHarness, appears inselectableCatalogForHarness, and the model is then selectable in the composer and serves turns.No screenshot is attached. The surface in question is a model name appearing in the composer's picker, and a capture of a working instance would show organization data.
npx tsc --noEmitis clean on both changes.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.