Skip to content

feat(providers): add OrcaRouter provider support - #182

Open
Marc-oss-hub wants to merge 1 commit into
atomicstrata:mainfrom
Marc-oss-hub:feat/orcarouter-provider
Open

feat(providers): add OrcaRouter provider support#182
Marc-oss-hub wants to merge 1 commit into
atomicstrata:mainfrom
Marc-oss-hub:feat/orcarouter-provider

Conversation

@Marc-oss-hub

Copy link
Copy Markdown

Summary

Adds OrcaRouter as a first-class provider in llmwiki. OrcaRouter is an OpenAI-compatible AI gateway whose API requires namespaced model ids (e.g. openai/gpt-4o-mini), so the provider's default model is openai/gpt-4o-mini.

What this changes

  • src/providers/orcarouter.ts — new OrcaRouterProvider extending OpenAIProvider, with the fixed base URL https://api.orcarouter.ai/v1 and fail-closed embed/embedBatch (mirrors the existing MiniMax wiring).
  • src/utils/provider.tsorcarouter added to SUPPORTED_PROVIDERS and the buildProvider factory; getOrcaRouterProvider() reads ORCAROUTER_API_KEY and throws a clear error when it is missing.
  • src/utils/constants.tsPROVIDER_MODELS.orcarouter = "openai/gpt-4o-mini".
  • src/utils/provider-guard.tsORCAROUTER_API_KEY entry in the credential key-var map.
  • src/utils/embedding-provider.ts — comment updated to note orcarouter is embedding-incapable (fail-closed).
  • test/provider-factory.test.ts, test/embed-batch-providers.test.ts, test/embedding-provider.test.ts — factory construction, missing-key error, fail-closed embeddings, and embedding-capability coverage, mirroring the MiniMax tests.
  • docs/configuration/providers.mdx, docs/configuration/environment-variables.mdx, README.md — OrcaRouter provider tab, env-var reference, and provider table row.

Why a separate provider rather than the OpenAI-compatible escape hatch

OrcaRouter is OpenAI-compatible, but the generic openai provider defaults to a bare model id (gpt-4o), which OrcaRouter rejects — its API requires a namespaced id such as openai/gpt-4o-mini. A dedicated orcarouter provider ships the correct namespaced default and mirrors the existing MiniMax provider pattern (a hosted OpenAI-compatible gateway with its own base URL, key env var, and fail-closed embeddings), so users get a correct, discoverable setup instead of remembering to pair OPENAI_BASE_URL with a namespaced LLMWIKI_MODEL.

How to use it

export LLMWIKI_PROVIDER=orcarouter
export ORCAROUTER_API_KEY=sk-orca-...
llmwiki compile

Model override (any namespaced id):

export LLMWIKI_MODEL=openai/gpt-4o-mini

Verification

  • Type-check (npx tsc --noEmit) passes; npm run build succeeds.
  • Provider tests pass: provider-factory.test.ts, embed-batch-providers.test.ts, embedding-provider.test.ts, embedding-provider-guard.test.ts, embedding-provider-model.test.ts — 57 tests.
  • Full suite: 4626 passed. The 227 failures are pre-existing Windows-environment issues (symlink EPERM, POSIX path separators, harness auth env) — reproduced on pristine main via git stash.
  • fallow audit on the changed files reports no issues.
  • L3 live test: getProvider()OrcaRouterProvider.complete() with a real key against https://api.orcarouter.ai/v1 returned a valid completion.

Security: no secrets, credentials, or private data are introduced, transmitted, or exposed by this change.

I'm an engineer on the OrcaRouter team.

Add a named `orcarouter` provider that mirrors the existing MiniMax
wiring: a fixed base URL (`https://api.orcarouter.ai/v1`) on an
OpenAI-compatible client, a namespaced default model
(`openai/gpt-4o-mini`), and fail-closed embeddings since the gateway's
embeddings endpoint is unverified.

- src/providers/orcarouter.ts: OrcaRouterProvider extends OpenAIProvider
- provider factory: SUPPORTED_PROVIDERS + getOrcaRouterProvider()
  (requires ORCAROUTER_API_KEY)
- provider guard: ORCAROUTER_API_KEY key-var entry
- docs: providers.mdx tab, environment-variables.mdx reference, README row
- tests: factory + fail-closed embeddings, mirroring MiniMax

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Marc-oss-hub <sjh668899@outlook.com>

@ethanj ethanj left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, and welcome. The provider wiring is careful work. You found every registration point, including the two in docs/ where MiniMax isn't even listed, and mirroring the MiniMax pattern was the right call. I also appreciate you disclosing that you work on OrcaRouter.

I verified the central premise and it holds. GET /v1/models returns 192 models, 190 of them namespaced, and bare gpt-4o is absent, so the generic openai provider genuinely would fail out of the box. A dedicated provider is justified rather than redundant.

Two things to fix before I can merge.

1. The docs change breaks the MDX build.

The OrcaRouter tab was inserted before the GitHub Copilot tab's </Tab> rather than after it, so the file now has six <Tab> opens and five closes. Copilot is left unclosed when </Tabs> is reached. Compiling the file confirms it:

main:        COMPILES OK
this branch: Unexpected closing tag `</Tabs>`, expected corresponding
             closing tag for `<Tab>` (177:3-177:31)

Fix is placement only: emit </Tab> after the Copilot <Warning>, before <Tab title="OrcaRouter">.

2. The embeddings rationale isn't accurate.

The comment in src/utils/embedding-provider.ts now says orcarouter throws "because their APIs expose no embeddings endpoint", but OrcaRouter does expose one:

POST /v1/embeddings                  -> 401   (exists, needs auth)
POST /v1/definitely-not-a-real-path  -> 404   (control: unknown routes 404)

and /v1/models lists openai/text-embedding-3-small, -3-large, -ada-002, plus two Gemini embedding models.

Failing closed for a first cut is fine by me. Wiring embeddings up means touching EMBEDDING_CAPABLE_PROVIDERS, EMBEDDING_CREDENTIALS, EMBEDDING_BACKENDS and EMBEDDING_ENDPOINT_VARS, plus EMBEDDING_MODELS and EMBED_BATCH_SIZES in constants.ts, and that's reasonably a separate PR. But could you change the stated reason to "not wired up in llmwiki yet" rather than "the API has no endpoint"? Same for the "unverified" wording in providers.mdx. That comment is what the next person reads when deciding whether OrcaRouter embeddings are possible, and the current version closes the question wrongly.

(If you'd rather just wire them up here, that works too.)

Two smaller things, neither blocking:

  • src/utils/provider.ts's header comment now reads "(Anthropic, OpenAI, Ollama, MiniMax, or OrcaRouter)" but the file also imports Copilot and the Agent SDK provider. It was already incomplete before you touched it. Since you're editing that line anyway, either complete it or drop the parenthetical.
  • openai/gpt-4o-mini is the only "mini" default in the repo; every other provider defaults to a flagship. Compile runs structured tool-call extraction where model capability shows up directly in output quality, so I'd like a sentence on why mini is the right default, or pick something stronger. Happy to defer to you here since you know the gateway.

I've approved the CI run so you'll get checks on the next push. Let me know if any of this is unclear or you'd like a hand with the embeddings wiring.

ethanj pushed a commit that referenced this pull request Aug 21, 2026
…hich ran

Closes the first two items of #186.

A prompt modifier is a setting that changes what the page prompt ASKS FOR
without changing the committed prompt wording — today only the output language,
set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely
by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled
project reported "Nothing to compile" and left every page in the previous
language. The selection now travels in state.json and a flipped modifier
invalidates the pages it would have changed.

Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the
JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical
whether or not a modifier was active, so it could not separate two such pages.

The digest is one GLOBAL fact, so anything that narrows what a run recompiles
can otherwise advance it past work that never happened, leaving a page current
and permanently stale. Two such paths exist and both are handled:

  - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a
    subset by design and must not record the selection as true of the project.
    It flushes source state but leaves the digest, so the sources it filtered out
    keep the only signal that says they are stale. The refreshed pages pay a
    second regeneration on the next full compile — the safe direction.

  - `markUnchangedPendingSources` demotes a promoted source back to `unchanged`
    when a pending review candidate carries the same source hash. It ran on the
    line after the promotion, so with any candidate present the feature was
    undone entirely and a modifier flip made zero model calls. A candidate
    produced under a different selection is not a duplicate of this run's work,
    so deduplication is skipped when the selection changed.

An ABSENT digest reads as "none selected" rather than as its own third state.
Reading absence as "no difference" was tried first and is wrong: the no-op
compile path never flushes state, so a project with nothing to compile would
never record a first digest, and flipping a modifier on exactly the settled
project this exists for would stay silent forever. Reading it as "none" costs an
untouched project nothing and costs one recompile to a project already running
under --lang when it upgrades — the same trade the embedding store makes for an
index that predates fingerprints.

The digest is recorded at the single durable flush rather than at load, so a
compile that dies mid-run leaves the previous digest on disk and the re-run
still sees the difference.

`promptModifierPairs` is the one canonical rendering that both the digest and
the frontmatter stamp consume, so a page's recorded modifiers and the state
digest cannot disagree about what the same selection means.

Items 3 and 4 of #186 are not included: they need files from #182 and #184,
which are unmerged. When #183 lands, --no-sources-section becomes one more
entry in activePromptModifiers and inherits the invalidation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
ethanj pushed a commit that referenced this pull request Aug 21, 2026
…hich ran

Closes the first two items of #186.

A prompt modifier is a setting that changes what the page prompt ASKS FOR
without changing the committed prompt wording — today only the output language,
set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely
by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled
project reported "Nothing to compile" and left every page in the previous
language. The selection now travels in state.json and a flipped modifier
invalidates the pages it would have changed.

Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the
JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical
whether or not a modifier was active, so it could not separate two such pages.

The digest is one GLOBAL fact, so anything that narrows what a run recompiles
can otherwise advance it past work that never happened, leaving a page current
and permanently stale. Two such paths exist and both are handled:

  - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a
    subset by design and must not record the selection as true of the project.
    It flushes source state but leaves the digest, so the sources it filtered out
    keep the only signal that says they are stale. The refreshed pages pay a
    second regeneration on the next full compile — the safe direction.

  - `markUnchangedPendingSources` demotes a promoted source back to `unchanged`
    when a pending review candidate carries the same source hash. It ran on the
    line after the promotion, so with any candidate present the feature was
    undone entirely and a modifier flip made zero model calls. A candidate
    produced under a different selection is not a duplicate of this run's work,
    so deduplication is skipped when the selection changed.

An ABSENT digest reads as "none selected" rather than as its own third state.
Reading absence as "no difference" was tried first and is wrong: the no-op
compile path never flushes state, so a project with nothing to compile would
never record a first digest, and flipping a modifier on exactly the settled
project this exists for would stay silent forever. Reading it as "none" costs an
untouched project nothing and costs one recompile to a project already running
under --lang when it upgrades — the same trade the embedding store makes for an
index that predates fingerprints.

The digest is recorded at the single durable flush rather than at load, so a
compile that dies mid-run leaves the previous digest on disk and the re-run
still sees the difference.

`promptModifierPairs` is the one canonical rendering that both the digest and
the frontmatter stamp consume, so a page's recorded modifiers and the state
digest cannot disagree about what the same selection means.

Items 3 and 4 of #186 are not included: they need files from #182 and #184,
which are unmerged. When #183 lands, --no-sources-section becomes one more
entry in activePromptModifiers and inherits the invalidation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
ethanj pushed a commit that referenced this pull request Aug 21, 2026
…hich ran

Closes the first two items of #186.

A prompt modifier is a setting that changes what the page prompt ASKS FOR
without changing the committed prompt wording — today only the output language,
set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely
by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled
project reported "Nothing to compile" and left every page in the previous
language. The selection now travels in state.json and a flipped modifier
invalidates the pages it would have changed.

Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the
JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical
whether or not a modifier was active, so it could not separate two such pages.

The digest is one GLOBAL fact, so anything that narrows what a run recompiles
can otherwise advance it past work that never happened, leaving a page current
and permanently stale. Two such paths exist and both are handled:

  - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a
    subset by design and must not record the selection as true of the project.
    It flushes source state but leaves the digest, so the sources it filtered out
    keep the only signal that says they are stale. The refreshed pages pay a
    second regeneration on the next full compile — the safe direction.

  - `markUnchangedPendingSources` demotes a promoted source back to `unchanged`
    when a pending review candidate carries the same source hash, so with any
    candidate present a modifier flip made zero model calls. Each candidate now
    records the digest it was GENERATED under and dedup demotes only when the
    hash and the selection both match.

    Comparing against the PROJECT's digest instead is not sufficient, and the
    difference is only visible in one direction: review mode never flushes
    state, so a project whose only compiles were `--review` has no recorded
    digest, and an absent digest means "none selected" — exactly what clearing a
    modifier requests. Japanese to Spanish passes either way; Japanese to
    default does not.

An ABSENT digest reads as "none selected" rather than as its own third state.
Reading absence as "no difference" was tried first and is wrong: the no-op
compile path never flushes state, so a project with nothing to compile would
never record a first digest, and flipping a modifier on exactly the settled
project this exists for would stay silent forever. Reading it as "none" costs an
untouched project nothing and costs one recompile to a project already running
under --lang when it upgrades — the same trade the embedding store makes for an
index that predates fingerprints.

The digest is recorded at the single durable flush rather than at load, so a
compile that dies mid-run leaves the previous digest on disk and the re-run
still sees the difference.

`promptModifierPairs` is the one canonical rendering that both the digest and
the frontmatter stamp consume, so a page's recorded modifiers and the state
digest cannot disagree about what the same selection means.

Items 3 and 4 of #186 are not included: they need files from #182 and #184,
which are unmerged. When #183 lands, --no-sources-section becomes one more
entry in activePromptModifiers and inherits the invalidation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
ethanj added a commit that referenced this pull request Aug 21, 2026
…hich ran (#188)

Closes the first two items of #186.

A prompt modifier is a setting that changes what the page prompt ASKS FOR
without changing the committed prompt wording — today only the output language,
set by --lang or LLMWIKI_OUTPUT_LANG. `detectChanges` classifies a source purely
by the SHA-256 of its bytes, so `llmwiki compile --lang Japanese` over a settled
project reported "Nothing to compile" and left every page in the previous
language. The selection now travels in state.json and a flipped modifier
invalidates the pages it would have changed.

Pages also carry a `promptModifiers` frontmatter entry, surfaced per page in the
JSON export. PROMPT_VERSION names the prompt IMPLEMENTATION and is identical
whether or not a modifier was active, so it could not separate two such pages.

The digest is one GLOBAL fact, so anything that narrows what a run recompiles
can otherwise advance it past work that never happened, leaving a page current
and permanently stale. Two such paths exist and both are handled:

  - A SCOPED run (`refresh --stale`, which supplies a changeFilter) recompiles a
    subset by design and must not record the selection as true of the project.
    It flushes source state but leaves the digest, so the sources it filtered out
    keep the only signal that says they are stale. The refreshed pages pay a
    second regeneration on the next full compile — the safe direction.

  - `markUnchangedPendingSources` demotes a promoted source back to `unchanged`
    when a pending review candidate carries the same source hash, so with any
    candidate present a modifier flip made zero model calls. Each candidate now
    records the digest it was GENERATED under and dedup demotes only when the
    hash and the selection both match.

    Comparing against the PROJECT's digest instead is not sufficient, and the
    difference is only visible in one direction: review mode never flushes
    state, so a project whose only compiles were `--review` has no recorded
    digest, and an absent digest means "none selected" — exactly what clearing a
    modifier requests. Japanese to Spanish passes either way; Japanese to
    default does not.

An ABSENT digest reads as "none selected" rather than as its own third state.
Reading absence as "no difference" was tried first and is wrong: the no-op
compile path never flushes state, so a project with nothing to compile would
never record a first digest, and flipping a modifier on exactly the settled
project this exists for would stay silent forever. Reading it as "none" costs an
untouched project nothing and costs one recompile to a project already running
under --lang when it upgrades — the same trade the embedding store makes for an
index that predates fingerprints.

The digest is recorded at the single durable flush rather than at load, so a
compile that dies mid-run leaves the previous digest on disk and the re-run
still sees the difference.

`promptModifierPairs` is the one canonical rendering that both the digest and
the frontmatter stamp consume, so a page's recorded modifiers and the state
digest cannot disagree about what the same selection means.

Items 3 and 4 of #186 are not included: they need files from #182 and #184,
which are unmerged. When #183 lands, --no-sources-section becomes one more
entry in activePromptModifiers and inherits the invalidation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants