feat(providers): add OrcaRouter provider support - #182
Conversation
Add a named `orcarouter` provider that mirrors the existing MiniMax wiring: a fixed base URL (`https://api.orcarouter.ai/v1`) on an OpenAI-compatible client, a namespaced default model (`openai/gpt-4o-mini`), and fail-closed embeddings since the gateway's embeddings endpoint is unverified. - src/providers/orcarouter.ts: OrcaRouterProvider extends OpenAIProvider - provider factory: SUPPORTED_PROVIDERS + getOrcaRouterProvider() (requires ORCAROUTER_API_KEY) - provider guard: ORCAROUTER_API_KEY key-var entry - docs: providers.mdx tab, environment-variables.mdx reference, README row - tests: factory + fail-closed embeddings, mirroring MiniMax Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Marc-oss-hub <sjh668899@outlook.com>
ethanj
left a comment
There was a problem hiding this comment.
Thanks for this, and welcome. The provider wiring is careful work. You found every registration point, including the two in docs/ where MiniMax isn't even listed, and mirroring the MiniMax pattern was the right call. I also appreciate you disclosing that you work on OrcaRouter.
I verified the central premise and it holds. GET /v1/models returns 192 models, 190 of them namespaced, and bare gpt-4o is absent, so the generic openai provider genuinely would fail out of the box. A dedicated provider is justified rather than redundant.
Two things to fix before I can merge.
1. The docs change breaks the MDX build.
The OrcaRouter tab was inserted before the GitHub Copilot tab's </Tab> rather than after it, so the file now has six <Tab> opens and five closes. Copilot is left unclosed when </Tabs> is reached. Compiling the file confirms it:
main: COMPILES OK
this branch: Unexpected closing tag `</Tabs>`, expected corresponding
closing tag for `<Tab>` (177:3-177:31)
Fix is placement only: emit </Tab> after the Copilot <Warning>, before <Tab title="OrcaRouter">.
2. The embeddings rationale isn't accurate.
The comment in src/utils/embedding-provider.ts now says orcarouter throws "because their APIs expose no embeddings endpoint", but OrcaRouter does expose one:
POST /v1/embeddings -> 401 (exists, needs auth)
POST /v1/definitely-not-a-real-path -> 404 (control: unknown routes 404)
and /v1/models lists openai/text-embedding-3-small, -3-large, -ada-002, plus two Gemini embedding models.
Failing closed for a first cut is fine by me. Wiring embeddings up means touching EMBEDDING_CAPABLE_PROVIDERS, EMBEDDING_CREDENTIALS, EMBEDDING_BACKENDS and EMBEDDING_ENDPOINT_VARS, plus EMBEDDING_MODELS and EMBED_BATCH_SIZES in constants.ts, and that's reasonably a separate PR. But could you change the stated reason to "not wired up in llmwiki yet" rather than "the API has no endpoint"? Same for the "unverified" wording in providers.mdx. That comment is what the next person reads when deciding whether OrcaRouter embeddings are possible, and the current version closes the question wrongly.
(If you'd rather just wire them up here, that works too.)
Two smaller things, neither blocking:
src/utils/provider.ts's header comment now reads "(Anthropic, OpenAI, Ollama, MiniMax, or OrcaRouter)" but the file also imports Copilot and the Agent SDK provider. It was already incomplete before you touched it. Since you're editing that line anyway, either complete it or drop the parenthetical.openai/gpt-4o-miniis the only "mini" default in the repo; every other provider defaults to a flagship. Compile runs structured tool-call extraction where model capability shows up directly in output quality, so I'd like a sentence on why mini is the right default, or pick something stronger. Happy to defer to you here since you know the gateway.
I've approved the CI run so you'll get checks on the next push. Let me know if any of this is unclear or you'd like a hand with the embeddings wiring.
…hich ran Closes the first two items of #186. A prompt modifier is a setting that changes what the page prompt ASKS FOR without changing the committed prompt wording — today only the output language, set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled project reported "Nothing to compile" and left every page in the previous language. The selection now travels in state.json and a flipped modifier invalidates the pages it would have changed. Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical whether or not a modifier was active, so it could not separate two such pages. The digest is one GLOBAL fact, so anything that narrows what a run recompiles can otherwise advance it past work that never happened, leaving a page current and permanently stale. Two such paths exist and both are handled: - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a subset by design and must not record the selection as true of the project. It flushes source state but leaves the digest, so the sources it filtered out keep the only signal that says they are stale. The refreshed pages pay a second regeneration on the next full compile — the safe direction. - `markUnchangedPendingSources` demotes a promoted source back to `unchanged` when a pending review candidate carries the same source hash. It ran on the line after the promotion, so with any candidate present the feature was undone entirely and a modifier flip made zero model calls. A candidate produced under a different selection is not a duplicate of this run's work, so deduplication is skipped when the selection changed. An ABSENT digest reads as "none selected" rather than as its own third state. Reading absence as "no difference" was tried first and is wrong: the no-op compile path never flushes state, so a project with nothing to compile would never record a first digest, and flipping a modifier on exactly the settled project this exists for would stay silent forever. Reading it as "none" costs an untouched project nothing and costs one recompile to a project already running under --lang when it upgrades — the same trade the embedding store makes for an index that predates fingerprints. The digest is recorded at the single durable flush rather than at load, so a compile that dies mid-run leaves the previous digest on disk and the re-run still sees the difference. `promptModifierPairs` is the one canonical rendering that both the digest and the frontmatter stamp consume, so a page's recorded modifiers and the state digest cannot disagree about what the same selection means. Items 3 and 4 of #186 are not included: they need files from #182 and #184, which are unmerged. When #183 lands, --no-sources-section becomes one more entry in activePromptModifiers and inherits the invalidation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
…hich ran Closes the first two items of #186. A prompt modifier is a setting that changes what the page prompt ASKS FOR without changing the committed prompt wording — today only the output language, set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled project reported "Nothing to compile" and left every page in the previous language. The selection now travels in state.json and a flipped modifier invalidates the pages it would have changed. Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical whether or not a modifier was active, so it could not separate two such pages. The digest is one GLOBAL fact, so anything that narrows what a run recompiles can otherwise advance it past work that never happened, leaving a page current and permanently stale. Two such paths exist and both are handled: - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a subset by design and must not record the selection as true of the project. It flushes source state but leaves the digest, so the sources it filtered out keep the only signal that says they are stale. The refreshed pages pay a second regeneration on the next full compile — the safe direction. - `markUnchangedPendingSources` demotes a promoted source back to `unchanged` when a pending review candidate carries the same source hash. It ran on the line after the promotion, so with any candidate present the feature was undone entirely and a modifier flip made zero model calls. A candidate produced under a different selection is not a duplicate of this run's work, so deduplication is skipped when the selection changed. An ABSENT digest reads as "none selected" rather than as its own third state. Reading absence as "no difference" was tried first and is wrong: the no-op compile path never flushes state, so a project with nothing to compile would never record a first digest, and flipping a modifier on exactly the settled project this exists for would stay silent forever. Reading it as "none" costs an untouched project nothing and costs one recompile to a project already running under --lang when it upgrades — the same trade the embedding store makes for an index that predates fingerprints. The digest is recorded at the single durable flush rather than at load, so a compile that dies mid-run leaves the previous digest on disk and the re-run still sees the difference. `promptModifierPairs` is the one canonical rendering that both the digest and the frontmatter stamp consume, so a page's recorded modifiers and the state digest cannot disagree about what the same selection means. Items 3 and 4 of #186 are not included: they need files from #182 and #184, which are unmerged. When #183 lands, --no-sources-section becomes one more entry in activePromptModifiers and inherits the invalidation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
…hich ran Closes the first two items of #186. A prompt modifier is a setting that changes what the page prompt ASKS FOR without changing the committed prompt wording — today only the output language, set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled project reported "Nothing to compile" and left every page in the previous language. The selection now travels in state.json and a flipped modifier invalidates the pages it would have changed. Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical whether or not a modifier was active, so it could not separate two such pages. The digest is one GLOBAL fact, so anything that narrows what a run recompiles can otherwise advance it past work that never happened, leaving a page current and permanently stale. Two such paths exist and both are handled: - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a subset by design and must not record the selection as true of the project. It flushes source state but leaves the digest, so the sources it filtered out keep the only signal that says they are stale. The refreshed pages pay a second regeneration on the next full compile — the safe direction. - `markUnchangedPendingSources` demotes a promoted source back to `unchanged` when a pending review candidate carries the same source hash, so with any candidate present a modifier flip made zero model calls. Each candidate now records the digest it was GENERATED under and dedup demotes only when the hash and the selection both match. Comparing against the PROJECT's digest instead is not sufficient, and the difference is only visible in one direction: review mode never flushes state, so a project whose only compiles were `--review` has no recorded digest, and an absent digest means "none selected" — exactly what clearing a modifier requests. Japanese to Spanish passes either way; Japanese to default does not. An ABSENT digest reads as "none selected" rather than as its own third state. Reading absence as "no difference" was tried first and is wrong: the no-op compile path never flushes state, so a project with nothing to compile would never record a first digest, and flipping a modifier on exactly the settled project this exists for would stay silent forever. Reading it as "none" costs an untouched project nothing and costs one recompile to a project already running under --lang when it upgrades — the same trade the embedding store makes for an index that predates fingerprints. The digest is recorded at the single durable flush rather than at load, so a compile that dies mid-run leaves the previous digest on disk and the re-run still sees the difference. `promptModifierPairs` is the one canonical rendering that both the digest and the frontmatter stamp consume, so a page's recorded modifiers and the state digest cannot disagree about what the same selection means. Items 3 and 4 of #186 are not included: they need files from #182 and #184, which are unmerged. When #183 lands, --no-sources-section becomes one more entry in activePromptModifiers and inherits the invalidation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
…hich ran (#188) Closes the first two items of #186. A prompt modifier is a setting that changes what the page prompt ASKS FOR without changing the committed prompt wording — today only the output language, set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled project reported "Nothing to compile" and left every page in the previous language. The selection now travels in state.json and a flipped modifier invalidates the pages it would have changed. Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical whether or not a modifier was active, so it could not separate two such pages. The digest is one GLOBAL fact, so anything that narrows what a run recompiles can otherwise advance it past work that never happened, leaving a page current and permanently stale. Two such paths exist and both are handled: - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a subset by design and must not record the selection as true of the project. It flushes source state but leaves the digest, so the sources it filtered out keep the only signal that says they are stale. The refreshed pages pay a second regeneration on the next full compile — the safe direction. - `markUnchangedPendingSources` demotes a promoted source back to `unchanged` when a pending review candidate carries the same source hash, so with any candidate present a modifier flip made zero model calls. Each candidate now records the digest it was GENERATED under and dedup demotes only when the hash and the selection both match. Comparing against the PROJECT's digest instead is not sufficient, and the difference is only visible in one direction: review mode never flushes state, so a project whose only compiles were `--review` has no recorded digest, and an absent digest means "none selected" — exactly what clearing a modifier requests. Japanese to Spanish passes either way; Japanese to default does not. An ABSENT digest reads as "none selected" rather than as its own third state. Reading absence as "no difference" was tried first and is wrong: the no-op compile path never flushes state, so a project with nothing to compile would never record a first digest, and flipping a modifier on exactly the settled project this exists for would stay silent forever. Reading it as "none" costs an untouched project nothing and costs one recompile to a project already running under --lang when it upgrades — the same trade the embedding store makes for an index that predates fingerprints. The digest is recorded at the single durable flush rather than at load, so a compile that dies mid-run leaves the previous digest on disk and the re-run still sees the difference. `promptModifierPairs` is the one canonical rendering that both the digest and the frontmatter stamp consume, so a page's recorded modifiers and the state digest cannot disagree about what the same selection means. Items 3 and 4 of #186 are not included: they need files from #182 and #184, which are unmerged. When #183 lands, --no-sources-section becomes one more entry in activePromptModifiers and inherits the invalidation.
Summary
Adds OrcaRouter as a first-class provider in llmwiki. OrcaRouter is an OpenAI-compatible AI gateway whose API requires namespaced model ids (e.g.
openai/gpt-4o-mini), so the provider's default model isopenai/gpt-4o-mini.What this changes
src/providers/orcarouter.ts— newOrcaRouterProviderextendingOpenAIProvider, with the fixed base URLhttps://api.orcarouter.ai/v1and fail-closedembed/embedBatch(mirrors the existing MiniMax wiring).src/utils/provider.ts—orcarouteradded toSUPPORTED_PROVIDERSand thebuildProviderfactory;getOrcaRouterProvider()readsORCAROUTER_API_KEYand throws a clear error when it is missing.src/utils/constants.ts—PROVIDER_MODELS.orcarouter = "openai/gpt-4o-mini".src/utils/provider-guard.ts—ORCAROUTER_API_KEYentry in the credential key-var map.src/utils/embedding-provider.ts— comment updated to noteorcarouteris embedding-incapable (fail-closed).test/provider-factory.test.ts,test/embed-batch-providers.test.ts,test/embedding-provider.test.ts— factory construction, missing-key error, fail-closed embeddings, and embedding-capability coverage, mirroring the MiniMax tests.docs/configuration/providers.mdx,docs/configuration/environment-variables.mdx,README.md— OrcaRouter provider tab, env-var reference, and provider table row.Why a separate provider rather than the OpenAI-compatible escape hatch
OrcaRouter is OpenAI-compatible, but the generic
openaiprovider defaults to a bare model id (gpt-4o), which OrcaRouter rejects — its API requires a namespaced id such asopenai/gpt-4o-mini. A dedicatedorcarouterprovider ships the correct namespaced default and mirrors the existingMiniMaxprovider pattern (a hosted OpenAI-compatible gateway with its own base URL, key env var, and fail-closed embeddings), so users get a correct, discoverable setup instead of remembering to pairOPENAI_BASE_URLwith a namespacedLLMWIKI_MODEL.How to use it
Model override (any namespaced id):
export LLMWIKI_MODEL=openai/gpt-4o-miniVerification
npx tsc --noEmit) passes;npm run buildsucceeds.provider-factory.test.ts,embed-batch-providers.test.ts,embedding-provider.test.ts,embedding-provider-guard.test.ts,embedding-provider-model.test.ts— 57 tests.EPERM, POSIX path separators, harness auth env) — reproduced on pristinemainviagit stash.fallow auditon the changed files reports no issues.getProvider()→OrcaRouterProvider.complete()with a real key againsthttps://api.orcarouter.ai/v1returned a valid completion.Security: no secrets, credentials, or private data are introduced, transmitted, or exposed by this change.
I'm an engineer on the OrcaRouter team.