feat(embeddings): allow a separate embedding provider

[PipDscvr]

Closes #154.

LLMWIKI_EMBEDDING_PROVIDER selects the backend that serves embeddings, independently of LLMWIKI_PROVIDER. @knew-inventai's configuration from the issue now works as written:

LLMWIKI_PROVIDER=claude-agent
LLMWIKI_EMBEDDING_PROVIDER=openai
OPENAI_EMBEDDINGS_BASE_URL=http://localhost:8000/v1
LLMWIKI_EMBEDDING_MODEL=<local-vllm-embedding-model>

Shape

getProvider() had seven callers, and they already split cleanly: six embedding, one chat (llm.ts:61). The seam existed; this PR names it.

• buildProvider(name) is extracted from getProvider()'s switch, so the embedding factory can construct any backend without duplicating construction.
• A new src/utils/embedding-provider.ts owns the embedding-capable set, the credential rule, and getEmbeddingProvider().
• The six embedding callers, plus model and batch-size resolution, follow the embedding provider.

LLMProvider is left alone. Splitting it into separate chat and embedding interfaces is arguably the cleaner end state, but it would touch all six provider classes and the SDK types for what is fundamentally one environment variable.

With LLMWIKI_EMBEDDING_PROVIDER unset, every path returns what it returns today: same provider object, same model, same batch size, same soft-credential handling, no store rebuild.

Two things worth reviewing

The embedding credential is not the chat credential. anthropic/claude-agent embeddings go to Voyage, so the check is VOYAGE_API_KEY, not ANTHROPIC_API_KEY. Reusing PROVIDER_KEY_VARS from provider-guard.ts would have validated the wrong variable entirely.

LLMWIKI_EMBEDDING_MODEL still does not apply to anthropic/claude-agent, even when named explicitly. An earlier draft of this branch made it apply, which seemed like the obvious generalisation. It is wrong: VoyageEmbeddingProvider.embed() calls voyageEmbedDefault with no model argument, and neither provider has embedding-model plumbing, so the model name would be recorded in the store while voyage-3-lite was actually embedded. Since that field is the store's invalidation key, a user with a leftover LLMWIKI_EMBEDDING_MODEL would have been billed for a full re-embed and left with a store describing vectors that never existed. The rule is therefore unchanged from main: the variable applies when the effective embedding provider is openai or ollama.

Also fixed here

migrateEmbeddingStore's rebuild path carried the previous vector dimension forward. Switching to a provider whose vectors have a different length — voyage-3-lite is 512, text-embedding-3-small is 1536 — failed assertVectorValid on every subsequent compile and never recovered, since the write aborts before the store is persisted. The rebuilt store now takes its dimension from the vectors the re-embed actually produces.

This bug predates the PR: an openai user switching between different-dimension models hits it today. But this feature makes provider-switching a documented workflow, so shipping without it would have meant documenting an example that bricks any existing wiki. Happy to split it into its own PR if you would rather review it separately.

What this does not do

minimax and copilot expose no embeddings API and remain non-capable; naming one now fails with a clear error listing the valid values, instead of an opaque failure from the provider's embed().

To be precise about that, since an earlier draft of this description overclaimed it: this is a message-quality improvement, not a timing one. embeddings-refresh.ts catches embedding failures and routes them through handleSafeEmbeddingFailure, which warns and continues unless LLMWIKI_EMBED_STRICT is set. Naming a bad embedding provider does not abort a compile that would otherwise have succeeded.

Tests

• test/embedding-provider.test.ts — factory resolution, the capable-set gate, and the credential rule in both directions. The issue's own configuration is asserted end to end, including that OPENAI_EMBEDDINGS_BASE_URL actually reached the constructed client rather than merely that some OpenAIProvider came back.
• test/embedding-provider-model.test.ts — one case per row of the model-resolution matrix, not a spot check. The rule has exactly one row that differs from main, and the rows that must not move are the ones that would silently re-embed existing users' wikis.
• test/embeddings-migrate.test.ts — a rebuild triggered by a model change yields dimensions: 0; against the previous code it returned the stale value.

How to test

npm test
npx vitest run test/embedding-provider.test.ts test/embedding-provider-model.test.ts test/embeddings-migrate.test.ts

To exercise the split without a local model server, point the embedding endpoint at anything OpenAI-compatible:

export LLMWIKI_PROVIDER=claude-agent
export LLMWIKI_EMBEDDING_PROVIDER=openai
export OPENAI_EMBEDDINGS_BASE_URL=http://localhost:8000/v1
export LLMWIKI_EMBEDDING_MODEL=<your-embedding-model>
llmwiki query "some question"

The property most worth checking is the one that protects existing users: unset LLMWIKI_EMBEDDING_PROVIDER and confirm the next compile does not re-embed. If the stored model changed, something moved on the default path.

unset LLMWIKI_EMBEDDING_PROVIDER
llmwiki compile        # should report no embedding work

Note on #167

EMBEDDING_CAPABLE_PROVIDERS is deliberately an explicit set rather than derived from EMBEDDING_MODELS, so that adding a model entry does not silently confer embedding capability. The cost is that it is now the fourth provider list in the codebase, alongside SUPPORTED_PROVIDERS, PROVIDER_KEY_VARS, and the EMBEDDING_MODELS/EMBED_BATCH_SIZES/EMBED_BATCH_CAPS trio. Worth knowing for the Atlas Cloud provider in #167, which will need to touch several of them — and that PR also edits src/utils/provider.ts, whose switch this branch extracts. The rebase is small either way; happy to go second.

@knew-inventai — does this cover your vLLM setup? I have verified the resolution logic and the credential rule, but not against a real vLLM instance.

[ethanj]

This is a well-built change and the framing in the description is the right one — getProvider() really did have seven callers splitting cleanly into six embedding and one chat, and naming that seam rather than bolting on a special case is what makes the rest of it fall out cleanly. A few things I checked specifically and found solid: all six embedding call sites actually moved (no stragglers still on getProvider()), the guard and the factory share findEmbeddingProviderProblem/EMBEDDING_CAPABLE_PROVIDERS rather than maintaining parallel lists that could drift, the two Anthropic-side providers correctly collapse to one Voyage backend so moving between them doesn't force a rebuild, and refusing to honor LLMWIKI_EMBEDDING_MODEL for those two is a subtle call that's right — recording a model name that was never sent would tag the store with a lie. The dimensions: 0 rebuild fix and the empty-store fix are real bugs caught in passing.

Two blockers, then three smaller items.

1. Blocking — the guard accepts a credential the runtime then ignores

OPENAI_EMBEDDINGS_API_KEY satisfies the OpenAI embedding credential check on its own:

llm-wiki-compiler/src/utils/embedding-provider.ts

Lines 54 to 58 in 0a58115

	const EMBEDDING_CREDENTIALS: Record<string, { keyVars: readonly string[]; endpointVar: string | null }> = {
	anthropic: { keyVars: ["VOYAGE_API_KEY"], endpointVar: null },
	"claude-agent": { keyVars: ["VOYAGE_API_KEY"], endpointVar: null },
	openai: { keyVars: ["OPENAI_EMBEDDINGS_API_KEY", "OPENAI_API_KEY"], endpointVar: "OPENAI_EMBEDDINGS_BASE_URL" },
	ollama: { keyVars: [], endpointVar: null },

But the constructor only builds a separate embeddings client when embeddingsBaseURL is set. Without it, embeddingsClient aliases the chat client, built from OPENAI_API_KEY ?? "llmwiki-unset", and resolveEmbeddingsKey is never reached:

llm-wiki-compiler/src/providers/openai.ts

Lines 134 to 142 in 0a58115

	});
	this.embeddingsClient = options.embeddingsBaseURL
	? new OpenAI({
	apiKey: this.resolveEmbeddingsKey(options, resolvedKey),
	baseURL: options.embeddingsBaseURL,
	timeout,
	})
	: this.client;
	}

So with a non-OpenAI chat provider, LLMWIKI_EMBEDDING_PROVIDER=openai, and only the dedicated key set, startup validation passes and embedding then authenticates with the placeholder. Confirmed by constructing the provider directly — the client comes back holding the literal llmwiki-unset. That's the exact failure mode this PR set out to remove, just relocated: it fails late, as a 401, instead of at the guard.

Fix direction: construct the dedicated client when either the URL or the key is configured, not the URL alone.

2. Blocking — legacy stores skip provider and endpoint invalidation, then get laundered

Any store written before this lands has no fingerprint and falls back to comparing the model name alone:

llm-wiki-compiler/src/utils/embeddings-store.ts

Lines 376 to 381 in 0a58115

	*/
	export function storeMatchesActiveEmbedding(store: Record<string, unknown> | null | undefined): boolean {
	if (!store) return false;
	if (typeof store.fingerprint === "string") return store.fingerprint === resolveEmbeddingFingerprint();
	return store.model === resolveEmbeddingModel();
	}

That check decides whether the old vectors survive:

llm-wiki-compiler/src/utils/embeddings.ts

Lines 88 to 95 in 0a58115

	// A store whose vectors came from a DIFFERENT embedding configuration — other
	// provider, other endpoint, other model — cannot have any of them preserved,
	// so hand the migration `null` and let its existing rebuild path own the case.
	// Gating here rather than inside the migration keeps that function pure: it
	// takes the active identity as data and never reads the environment.
	const preservable = storeMatchesActiveEmbedding(parsedOld?.store) ? parsedOld : null;
	const { store: migrated, reembedPageIds } = migrateEmbeddingStore(preservable, collected, model);

So the CHANGELOG's own motivating scenario — hosted OpenAI, then repointed at a local OpenAI-compatible endpoint with the model tag unchanged — reports as matching on every store that exists today. The vectors are preserved and queried with locally-generated query vectors, and nothing warns.

The part that makes this more than a delayed rebuild: a partial update mixes old and new vectors and then stamps the result with the new fingerprint, so the mixed store is trusted permanently from that point on. The weak check doesn't just postpone the fix, it launders the bad state into a fingerprinted one that all the new machinery will vouch for.

I follow the reasoning in the docstring — forcing every existing project into a full re-embed on upgrade is a genuinely bad default, and I don't think you should do that. But the model-only fallback should force a rebuild when an explicit embedding-provider or endpoint override is active, since that's precisely the case where the model name is known not to be sufficient.

3. Worth fixing — endpoint credentials are persisted and printed in full

resolveEmbeddingEndpoint returns the environment value verbatim, and the fingerprint joins it in unhashed:

llm-wiki-compiler/src/utils/embeddings-store.ts

Lines 355 to 366 in 0a58115

	export function resolveEmbeddingFingerprint(): string {
	const providerName = getActiveEmbeddingProviderName();
	// Keyed on the BACKEND, not the provider name: anthropic and claude-agent both
	// embed via Voyage, so moving between them must not trigger a rebuild.
	// NUL-separated: no env value can contain one, so no pair of distinct
	// configurations can collide by concatenation.
	return [
	resolveEmbeddingBackend(providerName),
	resolveEmbeddingModel(),
	resolveEmbeddingEndpoint(providerName),
	].join("\0");
	}

That string is a persisted store field, so an endpoint carrying userinfo or a credential in the query lands in .llmwiki/embeddings.json in cleartext. The forwarding warning prints the whole URL too:

llm-wiki-compiler/src/providers/openai.ts

Lines 83 to 94 in 0a58115

	function warnForwardedKey(embeddingsBaseURL: string): void {
	if (isLoopbackEndpoint(embeddingsBaseURL) || warnedForwardedKeyHosts.has(embeddingsBaseURL)) return;
	warnedForwardedKeyHosts.add(embeddingsBaseURL);
	output.status(
	"!",
	output.warn(
	`Sending OPENAI_API_KEY to the embeddings endpoint ${embeddingsBaseURL}. ` +
	`Set OPENAI_EMBEDDINGS_API_KEY to use a different credential there.` +
	(embeddingsBaseURL.startsWith("http://") ? " That endpoint is plaintext http." : ""),
	),
	);
	}

Hashing the fingerprint tuple costs nothing — it's only ever compared for equality, never read back — and would fix the persistence half outright. Redacting userinfo and query values in the warning covers the other half.

4. Minor — three docstrings still say "model" after the widening

You generalized invalidation from a model-name comparison to a full fingerprint, renamed isStaleModel to isStaleConfiguration, and updated the user-facing message to "different embedding configuration" — but the surrounding docs describe the old, narrower behavior:

llm-wiki-compiler/src/utils/embeddings-load.ts

Lines 116 to 118 in 0a58115

	* not v3 (older version — the writer has not flipped yet), failed the search
	* gate (corrupt vectors / over-cap), is stale-model, or is absent.
	*/

llm-wiki-compiler/src/utils/embeddings-search.ts

Lines 105 to 107 in 0a58115

	* Read the embedding store, returning null when it is missing, empty (per the
	* caller's predicate), or built with a stale model. Centralises the "is this
	* store usable for semantic lookup right now?" check.

llm-wiki-compiler/src/utils/embeddings-store.ts

Lines 4 to 6 in 0a58115

	* Owns the on-disk JSON contract for .llmwiki/embeddings.json (types, version,
	* atomic read/write) and the active-model resolution used to tag and validate
	* a store. No retrieval or embedding logic lives here — this is the base module

5. Minor — docs coverage is uneven across parallel sections

The docs work here is substantial and the LLMWIKI_PROVIDER table fix is a nice catch. It's just inconsistent between sections that describe the same situation: the two Anthropic-side sections and the OpenAI-Compatible section all gained LLMWIKI_EMBEDDING_PROVIDER, while the Ollama tab's table didn't, and the Copilot tab still advises switching the whole provider to openai — which is the workaround this PR replaces. OPENAI_EMBEDDINGS_API_KEY is covered in prose but missing from the OpenAI reference table, though its sibling table in providers.mdx has the row.

Items 1 and 2 are what I'd like resolved before this lands; 3 is small and worth taking while you're in here. Happy to talk through the rebuild-on-override rule in 2 if you'd rather scope it differently.

[PipDscvr]

All five items are addressed in 04d6293. Both blockers reproduced exactly as described before being fixed.

1. Blocking — the guard accepted a credential the runtime ignored

Confirmed and fixed. Constructing new OpenAIProvider("gpt-4o", { embeddingsApiKey: "sk-embed" }) returned a client holding llmwiki-unset, and sk-cloud when a chat key was set — the dedicated key was discarded in both directions.

The condition was the endpoint alone. It is now either signal, extracted into buildEmbeddingsClient so the constructor stays within the complexity budget:

private buildEmbeddingsClient(options: OpenAIProviderOptions, chatKey: string, timeout: number): OpenAI {
  if (!options.embeddingsBaseURL && !options.embeddingsApiKey) return this.client;
  return new OpenAI({
    apiKey: this.resolveEmbeddingsKey(options, chatKey),
    baseURL: options.embeddingsBaseURL ?? options.baseURL ?? null,
    timeout,
  });
}

The base URL falls back to the chat one, since a dedicated key with no endpoint means only the credential differs. resolveEmbeddingsKey now guards the warning on embeddingsBaseURL rather than asserting it non-null — a key with no separate endpoint is forwarded nowhere, so there is nothing to warn about and the ! is gone.

Four tests in openai-embeddings-credential.test.ts cover the key-only path: the dedicated key wins with and without a chat key, the endpoint is inherited, and no warning fires.

2. Blocking — legacy stores skipped invalidation, then got laundered

Fixed as you suggested. storeMatchesActiveEmbedding keeps the model-name fallback, but only while nothing overrides the backend or its endpoint:

if (typeof store.fingerprint === "string") return store.fingerprint === resolveEmbeddingFingerprint();
if (hasEmbeddingConfigurationOverride()) return false;
return store.model === resolveEmbeddingModel();

hasEmbeddingConfigurationOverride() lives in embedding-provider.ts, which already owns the env knowledge — true for an explicit LLMWIKI_EMBEDDING_PROVIDER or a non-empty resolveEmbeddingEndpoint() for the effective provider. It introduces no new throw surface: the fallback path already called getActiveEmbeddingProviderName() via resolveEmbeddingModel().

This resolves the laundering too. Under an override the legacy store is rejected, so preservable is null, the migration takes its rebuild path, and the fingerprint gets stamped onto vectors that were all produced by the active configuration. There is no longer a mixed store to launder.

One consequence worth your call. This bites more than the repoint-after-upgrade case. Anyone whose legacy store has always been written under a standing override — every ollama user, since OLLAMA_HOST is effectively required, and any openai user with OPENAI_BASE_URL set — pays one full re-embed on upgrade, even though nothing about their setup changed. We cannot distinguish those from a genuine repoint, because the legacy store records nothing to distinguish them by. I took the bounded one-time cost over silent mixing, per your reasoning, and documented it in both the CHANGELOG and the environment-variables page. If you would rather scope it to an explicit LLMWIKI_EMBEDDING_PROVIDER only, and accept that the endpoint-repoint case stays broken for legacy stores until their next write, that is a one-line change.

This also surfaced a real fixture assumption: context-integration.test.ts seeded an unstamped store while aimock points OPENAI_BASE_URL at its own server. That is now precisely the rejected configuration, so the seed stamps the fingerprint the subprocess resolves — which is what the compiler actually writes.

3. Endpoint credentials persisted and printed

Both halves fixed.

The fingerprint tuple is SHA-256 hashed. It is only ever compared for equality, so nothing downstream changes, and model stays on the store in cleartext for diagnostics. Tests assert that an endpoint carrying user:hunter2@ and ?api-key=sk-secret leaves no trace of the host or either secret in the stored value, and — the part worth pinning — that hashing does not collapse two endpoints differing only in their credential.

redactUrlCredentials strips userinfo and replaces query values while keeping parameter names, so the warning still identifies the endpoint. Dedup still keys on the raw URL: two endpoints differing only in credential are different destinations and each deserves its own warning.

4. Stale docstrings

All three updated to describe configuration-level invalidation: embeddings-load.ts, embeddings-search.ts, and the embeddings-store.ts file header.

5. Docs coverage

• LLMWIKI_EMBEDDING_PROVIDER added to the Ollama variables table.
• The Copilot tab no longer advises switching the whole provider. It now shows keeping Copilot for chat and routing embeddings elsewhere, which is the thing this PR adds.
• OPENAI_EMBEDDINGS_API_KEY added to the OpenAI table in environment-variables.mdx — that was the table missing it; the providers.mdx sibling already had the row.
• The invalidation section and CHANGELOG now state the legacy-store rule from item 2.

Verification

npx tsc --noEmit clean, npm run build succeeds, fallow reports no issues, and the full suite passes at 4395 passed / 3 skipped / 0 failed.

One note on reproducing that locally: .husky/pre-push runs the suite, and six credential-assertion tests fail if your shell exports LLMWIKI_PROVIDER or LLMWIKI_MODEL — they assert the CLI fails with no provider configured. Unset both and it is green. Unrelated to this branch; it reproduces on main.

[ethanj]

Approving — all three land well, and the legacy-store one lands better than what I asked for.

Scoping the rebuild to "an override is active" keeps the property that mattered on both sides: the laundering path is closed, and someone upgrading with no override set still doesn't pay for a full re-embed. That was the tension in the original finding and you resolved it in the right direction rather than picking one side. Hashing the fingerprint is a nice consequence too — it's only ever compared for equality, so opacity was free, and keeping model in cleartext for diagnostics is the detail that makes it practical.

Keying the embeddings client on endpoint or credential is right, and falling back to the chat base URL when only the key differs is the case I'd have expected to be missed. Good tests around all of it — the endpoint-only override with no named provider, and preferring the dedicated key over a chat key that is set, are both cases I hadn't raised.