Skip to content

Custom model providers are unreachable: wire the store, and stop the picker discarding their models - #333

Open
lmo-selected wants to merge 3 commits into
yc-software:mainfrom
lmo-selected:fix-custom-providers
Open

Custom model providers are unreachable: wire the store, and stop the picker discarding their models#333
lmo-selected wants to merge 3 commits into
yc-software:mainfrom
lmo-selected:fix-custom-providers

Conversation

@lmo-selected

@lmo-selected lmo-selected commented Aug 11, 2026

Copy link
Copy Markdown

Custom model providers cannot be used at all. Two independent defects sit between a registered provider and a usable model, and neither reports an error.

1. The store never reaches the routes

putCustomProvider, getCustomProviders and deleteCustomProvider all begin with

if (!ctx.deps.customProviders) return sendJson(ctx.res, 404, { error: "not_found" });

ctx.deps is ServerOptions. createCustomProviderStore is built in wiring.ts, returned on the built object, and declared on both ApiDeps and the createApp options — but index.ts passes it only to createApp, never to createServer. So ctx.deps.customProviders is always undefined and every request 404s.

refreshCustomProviders has the same gap and matters just as much: the handlers call it after a successful upsert to rehydrate the in-process registry, so without it a saved provider would not resolve until the next restart.

Reproduce: PUT /v1/admin/custom-providers/<id> with a valid spec. Returns 404 on any deployment.

2. The picker discards models it was told about

With the store wired, a provider saves, the row persists, and /api/runtime-config correctly returns the model in both modelsByHarness and modelCatalog — but the model is still absent from the web UI picker.

getBaseModel clones a template only when the catalogue entry's provider is the literal string "openrouter":

if (fallback?.provider === "openrouter") { ... }
throw new Error(`Unsupported model: ${id}`);

A custom provider's slug cannot be "openrouter"validateCustomProviderSpec rejects the built-in provider ids as reserved. So getBaseModel throws, buildOption's catch { return null } swallows it, and the option is dropped with no error in the console or the server log.

The catalogue is the server's statement of which models this deployment may use. This honours it for any entry rather than for one hardcoded provider name. The change is strictly more permissive and affects only models the server already advertised.

Reproduce: register a custom provider, add its model id to the scope's picker list, and observe it missing from the composer while runtime-config lists it.

Verification

Both fixes applied to a running instance: the PUT returns 200, the provider persists, the model resolves through resolveModel, passes modelSupportedByHarness, appears in selectableCatalogForHarness, and the model is then selectable in the composer and serves turns.

No screenshot is attached. The surface in question is a model name appearing in the composer's picker, and a capture of a working instance would show organization data.

npx tsc --noEmit is clean on both changes.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Every custom-provider request returned 404. The store is built in wiring and
declared on both the app options and ApiDeps, but the routes read ctx.deps from
ServerOptions, and index.ts never passed it there, so ctx.deps.customProviders
was always undefined and the guard at the top of each handler rejected the
request. The feature is unreachable in any deployment.

refreshCustomProviders goes with it: the handlers call it after a successful
upsert to rehydrate the in-process registry, so without it a saved provider
would not resolve until the next restart.
A model served by a custom provider never appears in the web UI picker.
getBaseModel clones a template only when the catalogue entry's provider is the
literal string "openrouter"; a custom provider's slug cannot be that, because the
built-in provider ids are reserved. So it throws, and buildOption's catch
discards the option silently: the model is absent from the picker with no error,
while the server has already returned it in modelCatalog and modelsByHarness.

The catalogue is the server's statement of which models this deployment may use.
Honour it for any entry rather than for one hardcoded provider name.
if (template) return cloneModel(template, id, clone.name);
}
if (fallback?.provider === "openrouter") {
if (fallback) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes the model selectable, but it also clones all of openrouter/auto’s transport identity. I reproduced the full runtime-options → transcript path with an Anthropic-compatible custom model: the selected model and reconstructed assistant message had provider: "openrouter", api: "openai-completions", and the OpenRouter base URL. /api/turn sends only model.id, so core routing remains correct, but the client transcript metadata and provider-derived defaults are wrong. Please carry the custom provider/protocol metadata into the client model (and cover both custom protocols in the web tests) rather than retaining the OpenRouter template identity.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, and the diagnosis was exactly right — thanks for reproducing the whole path rather than just the symptom.

Fixed in 87757b5. The protocol and endpoint already existed server-side on CustomRuntimeModel; they were simply dropped at the catalogue boundary, which is why the client had nothing better to go on than the template. customModelCatalog() now carries api and baseUrl, ModelCatalogEntry and the runtime-config payload pass them through, and getBaseModel picks the template matching the declared protocol and then overrides provider, api and baseUrl with the model's own.

So an Anthropic-protocol custom model now clones an Anthropic template and reports its own provider slug and endpoint, rather than presenting as OpenRouter over openai-completions.

An entry with no declared protocol keeps the previous openai-completions assumption, so built-in dynamic OpenRouter models are unchanged.

Tests: both custom protocols plus the unspecified-protocol default in plugins/web-ui/test/pi-models.test.ts, including an explicit assertion that an anthropic-messages model is never attributed to an openai-completions transport. The catalogue round-trip test in test/custom-providers.test.ts now asserts the protocol fields it carries.

Cloning openrouter/auto made a custom model selectable but gave it the
template's transport identity: an Anthropic-protocol custom model arrived in the
client as provider "openrouter" with api "openai-completions" and OpenRouter's
base URL. Core routing was unaffected, since /api/turn sends only model.id, but
the transcript metadata and any provider-derived default read from the model
were wrong.

The protocol and endpoint already exist server-side on CustomRuntimeModel and
were being dropped at the catalogue boundary. They now travel with the entry, so
the client clones the template matching the declared protocol and overrides
provider, api and baseUrl with the model's own. An entry with no declared
protocol keeps the previous OpenAI-completions assumption.

Tests cover both custom protocols and the unspecified-protocol default, and
assert an Anthropic-protocol model is never attributed to an OpenAI-completions
transport. The catalogue round-trip test now asserts the protocol fields it
carries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants