Add Atlas Cloud LLM provider - #167
Conversation
ethanj
left a comment
There was a problem hiding this comment.
Review
Thanks for this, and welcome — this is a nicely built first contribution. You clearly went and read how the existing providers work before writing anything, and it shows: AtlasCloudProvider mirrors the minimax.ts/copilot.ts shape closely, the fail-closed embed()/embedBatch() overrides follow the established convention rather than silently inheriting OpenAI's, and the alias handling is threaded through every site that reads a provider string — including resolveModel() in src/eval/citation-support.ts, which was reading it raw before. That last one is a genuine drive-by fix. The test coverage for alias routing and missing-key behavior is good, and I appreciated the clean env-var precedence with no fallback to another provider's key.
A few things to sort out before this lands. The first is the one that matters.
1. Blocking — the default model isn't verified against the tool-calling contract compile depends on
Atlas inherits OpenAIProvider, which sends tools and tool_choice: "required" on completions:
llm-wiki-compiler/src/providers/openai.ts
Lines 143 to 146 in 72305a3
Concept extraction and several other primary compile paths depend on getting tool arguments back — a model that ignores tools or can't satisfy tool_choice: "required" doesn't degrade gracefully here, it fails the workflow. The default is set to qwen/qwen3.5-flash:
llm-wiki-compiler/src/utils/constants.ts
Lines 97 to 99 in 72305a3
The new tests confirm the provider constructs and that the model string resolves, but none of them exercises a real structured completion, so the tool contract is currently unvalidated for this default. Could you confirm that model returns tool calls against Atlas's endpoint — ideally with a contract test that asserts tool arguments come back, not just that the model id resolves? If it turns out it doesn't support tool calling, the fix is to pick a tool-capable default or add an Atlas-specific structured-output path. Happy to be wrong here; I just couldn't verify it from the published model page either way, and it's the one thing that would make the provider look installed but not actually work.
2. Blocking — user-facing provider and env vars are missing from the docs
docs/AGENTS.md:18 requires that any user-facing feature change update the Mintlify docs in the same PR, and line 19 names providers and environment variables explicitly. Three surfaces need updating:
docs/configuration/environment-variables.mdx:17— theLLMWIKI_PROVIDERtable enumerates the valid values and omitsatlascloud, which makes it factually wrong rather than merely incompletedocs/configuration/providers.mdx— needs an Atlas Cloud setup tab, and it's worth calling out the embeddings limitation there so users know semantic search needs a different providerREADME.md— provider list and credentials table
llm-wiki-compiler/src/utils/constants.ts
Lines 104 to 114 in 72305a3
For precedent, the two most recent provider additions (#55 and #81) both did this in the same PR.
3. Worth fixing — semantic search hard-fails instead of degrading
EMBEDDING_MODELS has no atlascloud entry, so model resolution falls through to the Anthropic default:
llm-wiki-compiler/src/utils/embeddings-store.ts
Lines 302 to 307 in 72305a3
The effect: with an existing voyage-3-lite store on disk, the freshness gate resolves Atlas to voyage-3-lite, decides the store is current, and starts the semantic path — which then hits your embed() override and throws, instead of falling back to lexical search.
llm-wiki-compiler/src/providers/atlascloud.ts
Lines 38 to 50 in 72305a3
Important caveat, and this is not on you: MiniMax and Copilot are in exactly the same position — absent from EMBEDDING_MODELS, throwing on embed(). So this is a pre-existing gap that your provider joins rather than one you introduced, and it may well be better fixed as a separate change covering all three (either an explicit "no embeddings" marker that makes the store gate degrade immediately, or catching embedding-unavailable at the semantic-selection call site). Your call whether to take it here or file it — I don't want to widen the scope of a first PR for something two existing providers already do.
4. Minor — provider list now lives in three places
The unknown-provider error previously read from Object.keys(PROVIDER_KEY_VARS), the same object doing the credential lookup, so the advertised list couldn't drift from the enforced one. It now reads SUPPORTED_PROVIDER_INPUTS, leaving three separately-maintained lists (constants.ts, SUPPORTED_PROVIDERS in provider.ts, PROVIDER_KEY_VARS in provider-guard.ts). All three agree today so nothing is broken — but the next provider that updates two of three gets a guard that rejects what the factory can build. Worth collapsing to one source while it's cheap.
5. Minor — the file header you edited is still stale
llm-wiki-compiler/src/utils/provider.ts
Lines 5 to 7 in 72305a3
You added Atlas Cloud to this list, but it still omits the Copilot and Agent-SDK providers, both implemented in that same file. Easy to finish off while you're in there.
Item 1 is the real gate; 2 is a repo rule; 3 is genuinely optional for this PR. Nice work on the parts that are done — happy to talk any of it through if something's unclear or if you'd rather split some of it into a follow-up.
|
Following up on this after a month, and rather than leave it sitting I have picked it up and finished it in #190. You are credited as co-author there: the provider, the alias handling and the original tests are your work, and they rebased onto current Two things I changed, and one is worth knowing about. The default model. I was able to check the tool-calling question I raised, and the answer was no. Atlas Cloud's catalogue marks tool support per model, and Docs, which were the other blocking item, plus the provider-list consolidation I mentioned in point 4. Thank you for this. The alias threading through Closing this in favour of #190. |
Picks up #167, which had been open a month without a push, rebased onto current main and finished against the review. `LLMWIKI_PROVIDER=atlascloud` (aliases `atlas-cloud`, `atlas`) routes chat and tool calls through the Atlas Cloud gateway, an OpenAI-compatible API across models from several publishers. Credentials come from ATLASCLOUD_API_KEY or ATLAS_CLOUD_API_KEY; ATLASCLOUD_BASE_URL overrides the endpoint. The default model changed from what the PR shipped. Atlas Cloud's catalogue marks tool support per model, and `qwen/qwen3.5-flash` carries no `supported_features` entry at all while 35 of its 136 models advertise `["json_mode","structured_outputs","tools"]`. Compile extracts concepts through `toolCall` with `tool_choice: "required"`, so that default would have failed on the first extraction request rather than degrading. `qwen/qwen3.5-35b-a3b` is the smallest catalogue model that does advertise tools, keeping the cheap-and-fast intent of the original choice. A dedicated provider is warranted rather than the OpenAI-compatible escape hatch: all 136 catalogue ids are namespaced by publisher and bare `gpt-4o` is absent, so `LLMWIKI_PROVIDER=openai` with `OPENAI_BASE_URL` fails on its own default model. `SUPPORTED_PROVIDERS` is now derived from `SUPPORTED_PROVIDER_INPUTS` instead of being a second hand-written list. The two drifting apart yields a guard that rejects what the factory can build, and the PR had grown the count from two lists to three. The stale module header in provider.ts, which had omitted Copilot and the Agent SDK since they were added, is corrected in the same pass. All three fail-closed providers told the user to change LLMWIKI_PROVIDER when embeddings are unavailable, which costs them the chat provider they chose. LLMWIKI_EMBEDDING_PROVIDER exists (#154) precisely so they do not have to, and it is what the docs prescribe. Atlas Cloud inherited the wrong text verbatim from MiniMax, and a Copilot test asserted the wrong advice was correct, named "error message mentions switching to the openai provider" — which is how it survived #154 shipping the variable that made it wrong. All three messages now point at the embedding override, Copilot's names OPENAI_EMBEDDINGS_API_KEY rather than the main key, and the test asserts the correct remedy plus the absence of the old one. Docs cover the three surfaces the repo requires for a user-facing provider: the LLMWIKI_PROVIDER enumeration and a credentials section in environment-variables.mdx, a setup tab in providers.mdx, and the README provider table. Embeddings are not wired up and fail closed, matching MiniMax and Copilot. Those three share an absent EMBEDDING_MODELS entry, which is better fixed once across all of them than for this provider alone. Co-authored-by: binyangzhu000-sudo <224954946+binyangzhu000-sudo@users.noreply.github.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUdoRq1DJq23aJuhK9QK7X
Picks up #167, which had been open a month without a push, rebased onto current main and finished against the review. `LLMWIKI_PROVIDER=atlascloud` (aliases `atlas-cloud`, `atlas`) routes chat and tool calls through the Atlas Cloud gateway, an OpenAI-compatible API across models from several publishers. Credentials come from ATLASCLOUD_API_KEY or ATLAS_CLOUD_API_KEY; ATLASCLOUD_BASE_URL overrides the endpoint. The default model changed from what the PR shipped. Atlas Cloud's catalogue marks tool support per model, and `qwen/qwen3.5-flash` carries no `supported_features` entry at all while 35 of its 136 models advertise `["json_mode","structured_outputs","tools"]`. Compile extracts concepts through `toolCall` with `tool_choice: "required"`, so that default would have failed on the first extraction request rather than degrading. `qwen/qwen3.5-35b-a3b` is the smallest catalogue model that does advertise tools, keeping the cheap-and-fast intent of the original choice. A dedicated provider is warranted rather than the OpenAI-compatible escape hatch: all 136 catalogue ids are namespaced by publisher and bare `gpt-4o` is absent, so `LLMWIKI_PROVIDER=openai` with `OPENAI_BASE_URL` fails on its own default model. `SUPPORTED_PROVIDERS` is now derived from `SUPPORTED_PROVIDER_INPUTS` instead of being a second hand-written list. The two drifting apart yields a guard that rejects what the factory can build, and the PR had grown the count from two lists to three. The stale module header in provider.ts, which had omitted Copilot and the Agent SDK since they were added, is corrected in the same pass. All three fail-closed providers told the user to change LLMWIKI_PROVIDER when embeddings are unavailable, which costs them the chat provider they chose. LLMWIKI_EMBEDDING_PROVIDER exists (#154) precisely so they do not have to, and it is what the docs prescribe. Atlas Cloud inherited the wrong text verbatim from MiniMax, and a Copilot test asserted the wrong advice was correct, named "error message mentions switching to the openai provider" — which is how it survived #154 shipping the variable that made it wrong. All three messages now point at the embedding override, Copilot's names OPENAI_EMBEDDINGS_API_KEY rather than the main key, and the test asserts the correct remedy plus the absence of the old one. Docs cover the three surfaces the repo requires for a user-facing provider: the LLMWIKI_PROVIDER enumeration and a credentials section in environment-variables.mdx, a setup tab in providers.mdx, and the README provider table. Embeddings are not wired up and fail closed, matching MiniMax and Copilot. Those three share an absent EMBEDDING_MODELS entry, which is better fixed once across all of them than for this provider alone. Co-authored-by: binyangzhu000-sudo <224954946+binyangzhu000-sudo@users.noreply.github.com>
Summary
Validation
Notes