Skip to content

feat(llms): add OrcaRouter as a named LLM provider - #2275

Open
XiaoHuo888-hue wants to merge 1 commit into
Open-Source-Legal:mainfrom
XiaoHuo888-hue:add-orcarouter-provider
Open

feat(llms): add OrcaRouter as a named LLM provider#2275
XiaoHuo888-hue wants to merge 1 commit into
Open-Source-Legal:mainfrom
XiaoHuo888-hue:add-orcarouter-provider

Conversation

@XiaoHuo888-hue

Copy link
Copy Markdown

Summary

Adds OrcaRouter as a first-class LLM provider. OrcaRouter is an OpenAI-compatible model routing gateway that fronts dozens of hosted models behind one endpoint — pick a router alias like orcarouter/auto or a specific hosted model. It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every prompt/response and governing every tool call on a default-deny basis, with no application code changes.

Changes

  • New provider component opencontractserver/pipeline/llm_providers/orcarouter_provider.py — mirrors the OpenAI provider. Declares ORCAROUTER_API_KEY (secret) and a base_url (optional, defaulting to https://api.orcarouter.ai/v1) so credentials are configurable live in System Settings → Pipeline Components, exactly like the other providers. The registry auto-discovers it and it surfaces in the System Settings LLM picker and Corpus.preferred_llm / AgentConfiguration.preferred_llm.
  • opencontractserver/llms/model_factory.py — pydantic-ai has no native orcarouter: provider prefix, so a bare "orcarouter:..." spec string would raise "Unknown model" at agent construction. The factory now always builds a concrete OpenAI-compatible model for this provider (DB credentials win; otherwise ORCAROUTER_API_KEY + the default endpoint), keeping the env-fallback contract safe.
  • Teststest_llm_model_factory.py: registry discovery + model construction (no-DB-creds still builds a concrete model; DB base_url/api_key override; invalid DB base_url falls back to default). test_llm_runtime_config.py: provider registered + schema.
  • Docs — model-spec table + API-keys section in docs/architecture/llms/README.md; ORCAROUTER_API_KEY added to the production sample env.
  • Changelog fragment changelog.d/orcarouter-provider.added.md.

Test plan

  • pre-commit run --all-files passes on the touched files (black, isort, flake8, mypy, pyupgrade, trailing-whitespace, end-of-file-fixer); the changelog fragment validates via python3 scripts/collate_changelog.py --check (the pre-commit wrapper needs bare python in PATH, which the CI runner provides).
  • Standalone verification exercises the same code paths: registry discovery, settings schema, and model construction for orcarouter: specs.
  • Live test: drove the factory-built model end-to-end against https://api.orcarouter.ai/v1/chat/completions with a real key — HTTP 200, returned ORCA-LIVE-OK.

Checklist

  • Tests pass locally for any code this PR touches
  • pre-commit run --all-files passes (black, isort, flake8, prettier)
  • A changelog fragment was added under changelog.d/
  • No new dependency added — reuses the existing pydantic-ai-slim[openai] / openai stack

Contributor License Agreement

By submitting this pull request, you agree to license your contribution under the project's Contributor License Agreement.

Disclosure: I'm an engineer on the OrcaRouter team.

Adds an orcarouter: provider to the pipeline LLM provider registry
(opencontractserver/pipeline/llm_providers/orcarouter_provider.py)
mirroring the OpenAI provider pattern. OrcaRouter is an OpenAI-compatible
model routing gateway; model specs like orcarouter:orcarouter/auto reuse
the existing pydantic-ai OpenAI client path.

pydantic-ai has no native orcarouter: prefix, so
opencontractserver/llms/model_factory.py now always constructs a concrete
OpenAI-compatible model for this provider instead of returning a bare spec
string (which would raise 'Unknown model'). DB-configured credentials win;
otherwise ORCAROUTER_API_KEY and the default endpoint
https://api.orcarouter.ai/v1 are used.

Docs: model-spec table + API-keys section in docs/architecture/llms/README.md,
ORCAROUTER_API_KEY in the production sample env. Changelog fragment added.

Signed-off-by: XiaoHuo888-hue <jinhao.song@myflashcloud.com>
@github-actions

Copy link
Copy Markdown
Contributor


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


XiaoHuo888-hue seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant