feat(model-vector): dedupe reject-phrase embedding across workers via a required store - #2514
Merged
jfallows merged 3 commits intoSep 3, 2026
Merged
Conversation
… an optional store VectorModelHandlerImpl is a per-worker factory that eagerly embeds every configured reject phrase in its constructor, once per attach. Since every namespace binding is replicated to and attached independently on every EngineWorker, an N-worker engine fires N identical, redundant embed() calls against the configured embedding provider for the exact same input, all at once, at attach time -- unnecessary load against whatever service backs the embedding, worst case a synchronized burst against a provider with limited concurrency. Add an optional `store` reference (a new StoredConfig, alongside VaultedConfig/ GuardedConfig/CatalogedConfig/EmbeddedConfig) so a worker can check a shared cache before embedding, take a short-lived lock to embed for real only when the cache is empty, and have every other worker (or, with a distributed store, replica) poll the cache with capped backoff until the winner's result lands -- including recovering if the lock owner fails without ever writing a result, since the lock simply expires and the next poll wins it instead. Configuring no store preserves the current per-worker-independent behavior exactly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
model-vector is incubator-only and has never shipped in a release, so there is no backward-compatibility reason to keep the store reference optional. Require it in the schema, drop the now-unreachable no-store code path from VectorModelHandlerImpl, and update every fixture/test that predates the property to configure one -- consolidating the config/spec IT fixtures back down to a single reject.yaml now that every scenario needs a store anyway. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…property store became a required vector model property in this same PR; this example predates it and never got one, so its zilla.yaml no longer validated -- confirmed as the tcp.echo.embedding CI failure. Add a memory store (in-process only, matching this example's single-instance, no-external-dependency scope) and reference it from the model. Verified locally: the stack becomes healthy and all three test.sh scenarios (accepted message, two differently-worded rejections) pass against a freshly built zilla:develop-SNAPSHOT image. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
jfallows
deleted the
feature/model-vector-store-backed-reject-embedding
branch
September 3, 2026 23:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
VectorModelHandlerImplis a per-worker factory that eagerly embeds every configuredrejectphrase in its constructor, once per attach. Since every namespace binding is replicated to and attached independently on everyEngineWorker, an N-worker engine fires N identical, redundantembed()calls against the configured embedding provider for the exact same input, all at once, at attach time.storereference on thevectormodel config (a newStoredConfig, alongsideVaultedConfig/GuardedConfig/CatalogedConfig/EmbeddedConfig) so a worker checks a shared cache first, takes a short-lived lock to embed for real only on a cache miss, and every other worker (or, with a distributed store, replica) polls the cache with capped backoff until the winner's result lands. If the lock owner fails without writing a result, the lock simply expires and the next poll wins it instead. A failed embed is never cached, so a transient error doesn't permanently poison the result.storeis required, not optional:model-vectoris incubator-only and has never shipped in a release, so there's no backward-compatibility reason to make the dedup opt-in.Test plan
config/engine.conf,incubator/model-vector.conf,incubator/model-vector.spec,incubator/model-vectorall green (./mvnw clean verify), plus a full./mvnw clean install -DskipTestsreactor build to confirm nothing else regressed from the new sharedStoredConfigclass orVectorModelConfig's changed constructor.VectorModelHandlerImplTestsimulate two workers sharing one store (asstore-memoryshares its backing map across everyEngineWorkerin one process): only one of them ever callsEmbeddingHandler.embed(), the loser picks up the winner's cached result via its own retry/backoff, and a failed embed releases the lock for the next attempt without caching the failure.VectorModelITscenarios load a realzilla.yamlreferencing atype: teststore end-to-end through the engine's own config-loading/name-resolution path, not just direct unit construction.SchemaTestconfirms the fixture config validates now thatstoreis a required property.