feat(core): content-hash embedding cache to avoid re-embedding identical chunks - #410
Open
strichter wants to merge 6 commits into
Open
feat(core): content-hash embedding cache to avoid re-embedding identical chunks#410strichter wants to merge 6 commits into
strichter wants to merge 6 commits into
Conversation
…cal chunks Embeddings are a pure function of (model, text), so the same code chunk produces the same vector every time. But collection names — and the merkle snapshots that drive incremental sync — are keyed by the absolute codebase path, so indexing the same repository at a second path (a git worktree, a re-clone, or a CI checkout) re-embeds every file from scratch. On a CPU embedder that dominates indexing time. This adds an opt-out content-addressed cache that maps (modelIdentifier, chunk text) -> vector, so each unique chunk is embedded once and reused across every collection on the machine. Per-codebase collections stay separate, so different branches/worktrees remain independently searchable; only the embedding work is shared. - `FileSystemEmbeddingCache`: zero-dependency, sharded, atomic-write cache under `~/.context/embedding-cache` (configurable via EMBEDDING_CACHE_DIR), mirroring how FileSynchronizer persists under `~/.context`. - `CachedEmbedding`: transparent decorator over any Embedding provider; splits a batch into cache hits and misses, only sends misses to the provider, preserves input order, and stays faithful to a misbehaving provider (never reshapes a mismatched batch). - `Embedding.getModelIdentifier()`: fully-qualifies the model so two models can never share cache entries; overridden by each provider to include the model name. - Wired into Context; enabled by default, opt out with EMBEDDING_CACHE=false. - Documented the new env vars in the MCP config help and startup summary. - Tests: new cached-embedding suite; existing embedding-error tests bypass the cache; a jest setup redirects the cache to a temp dir so the suite never touches the developer's real ~/.context. Measured: re-indexing an identical tree at a new path drops from ~69s to ~3s (nomic-embed-text on CPU), with search results unchanged.
The Ollama embedder called client.embed() with no timeout, no AbortSignal, and no retry. A single request that never returns — e.g. embedding a content-hashed, single-line minified bundle under viz/storybook-static/assets/ on a CPU embedder — hangs indexing forever with no self-recovery. - Inject a timeout fetch into the Ollama client so every embed request is bounded by OLLAMA_EMBED_TIMEOUT_MS (default 120s); a hung request now rejects (and is caught upstream, aborting the pass in bounded time) instead of wedging. - Add a bounded retry for transient failures only; a timeout is deterministic for a given input, so it is never retried. - Add storybook-static to DEFAULT_IGNORE_PATTERNS: its bundles are plain .js with content-hash names, so the *.min.js / *.bundle.js patterns miss them — exclude the whole tree by directory so the pathological input never reaches the embedder. Follow-up (upstream): drop only the offending file on timeout instead of aborting the pass, so one bad file doesn't end an otherwise-good index. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
strichter
force-pushed
the
feat/content-hash-embedding-cache
branch
from
July 24, 2026 04:05
e9c0630 to
b566a41
Compare
Index a git worktree's branch as a sparse copy-on-write overlay on top of
the shared base ("main") index, at file granularity:
- One canonical Milvus collection per repository (keyed by the git common
dir), shared across every linked worktree — so branches overlay a single
base rather than each re-indexing a full private copy.
- A `branch` scalar field on every chunk (folded into the primary-key hash
so identical base/branch chunks never collide), threaded through both
schemas, both insert maps, search output_fields, and the result mappers.
- On a non-base branch, indexing is restricted to the files the branch has
touched (git diff vs base + untracked); untouched files resolve from base.
- Force-reindex and clearIndex on a branch delete only that branch's rows —
never drop the shared collection (which would take out the base index and
every sibling worktree).
- Search is a two-pass "branch shadows base" merge: the branch's rows win;
base rows survive only for files the branch never touched.
- Non-git paths and CODEINDEX_BRANCH_OVERLAY=false preserve the prior
single-namespace behavior exactly.
- Synchronizer map re-keyed by resolved worktree path (not collection name),
since the collection is now shared across worktrees.
Base branch configurable via CODEINDEX_BASE_BRANCH (default "main").
Validated end-to-end against live Milvus + Ollama (touched-only indexing,
per-file shadow, base fallback, base rows preserved, cross-worktree sharing).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…se drift (#2) Two freshness holes made the index answer confidently from rows that no longer match the working tree. **The overlay went stale between explicit index runs.** A branch's touched files were only re-indexed when someone ran `codeindex index`, so an edit made after that run was invisible to search. `semanticSearch` now brings the overlay up to date first: re-derive the touched set from git (milliseconds), stat each candidate, and re-chunk only the files whose (mtime, size) actually moved — unchanged chunks hit the content-hash embedding cache, so a one-line edit costs one file's work. A file that leaves the touched set has its branch rows dropped so the base index shows through again. Deriving the set from git on every search, rather than accumulating filesystem events, is what makes this correct across `git checkout`, rebase, stash and crashes — the cases where a watcher desyncs silently. It is bounded at LAZY_REFRESH_MAX_FILES: a long-lived branch can differ from base by thousands of files (measured: 1,510 on a real worktree), and that is an indexing job, not a freshness top-up. Past the bound the refresh declines and the search says so instead of turning into a full index run. A refresh failure never costs the caller their search. **Base drift was invisible.** Every file a branch has *not* touched answers from the shared base index, which only moves when someone re-indexes the base branch. On a live machine that index was 1,308 commits and 2,863 files behind origin/main with nothing saying so. Base-branch index runs now stamp the commit they reflect, and a search whose base has drifted past BASE_STALENESS_WARN_COMMITS says how far behind it is and what to run. Both notices ride on the search result, not a log: silent staleness is what turns a stale index into wrong answers someone acts on. Tests: 11 new cases over a real temp git repo covering first-index, no-op, single-file delta, branch switch, row-drop on revert, the bound at and past its edge, drift detection and quiet-when-fresh, the opt-out, and refresh-failure passthrough.
A stopped or crashed Milvus took the whole MCP server with it. The Milvus
gRPC client rejects with `14 UNAVAILABLE: ... ECONNREFUSED`, nothing catches
it, and Node promotes an unhandled rejection to a fatal uncaught exception.
The rejection settles moments AFTER the MCP handshake has already advertised
the tools, so the client is left holding a dead stdio server with no error to
surface: `search_code` simply disappears for the rest of the session and
agents fall back to grep without ever noticing they lost semantic search.
Reproduced by pointing the server at a closed port: before this change the
process exits 1 and advertises no tools; after it, all four tools stay
available and each answers with `isError` instead.
- index.ts: contain unhandled rejections at the process level. Deliberately
does not trap `uncaughtException` -- a synchronous throw can leave genuinely
inconsistent state, while the failure this guards against is asynchronous.
- sync.ts: the periodic sync called `handleSyncIndex()` fire-and-forget with
no `.catch()`, unlike the initial sync and the trigger watcher, which both
already guard it. Give it the same catch, and keep the interval handle so
the loop can be stopped -- it was a local that nothing retained.
- utils.ts/handlers.ts: report an unreachable backend as such. The old text
("check if the codebase has been indexed first") pointed at the one thing
that is not wrong; a reader who follows it re-indexes and fails identically.
* fix(mcp): rank implementation above the tests that exercise it
A test restates the query's vocabulary almost literally -- the error strings,
the option names, the call itself -- while the implementation encodes the
mechanism and often names none of it. Pure embedding similarity therefore
rewards the test, and the effect is not a mild reordering: for
"retry with exponential backoff when the API returns a rate limit error"
every returned result was a test call site, and the decorator that actually
implements it never appeared at all. Two factors compound. Similarity ranks
the restatement first, and the search asked the vector store for exactly the
caller's limit, so once tests filled those slots the implementation was not
in the list to be promoted.
Search wider than asked (4x, still capped at 50), re-rank, then trim. Test,
fixture and mock paths take a score penalty; they are demoted, never dropped,
because sometimes the test is the answer -- a decisively more relevant test
still wins, and CLAUDE_CONTEXT_TEST_RANK_PENALTY=1 turns the whole thing off.
Results carry a [test] tag so a reader scanning locations does not have to
infer the role from the path.
Path matching is segment-exact rather than substring, so shipped packages that
merely contain the word -- src/.../testing/, contest/, latest/ -- keep their
rank.
Measured on the query above: before, 3 results, all tests, no implementation.
After, the implementation ranks first and two more implementation sites
surface. A query that already ranked well is unchanged in its top 3.
* fix(mcp): skip the test penalty when the query asks for tests
Benchmarking the penalty on a set of test-seeking queries ("where do we test
entity resolution merging", "fixtures used by the extraction tests") showed the
one real cost of demotion: tests ranked first for 4/6 of them before, 3/6 after,
and one query lost tests from its top 5 entirely.
That cost is avoidable rather than inherent. When the query itself names tests,
specs, fixtures, mocks or coverage, demoting them is not a heuristic misfire --
it is the opposite of what was asked. Skip the penalty for those queries.
Restores test-seeking queries to the unpenalised baseline (4/6 first, 0/6
losing tests) while the implementation-seeking set keeps its full gain (11/12
ranked first, up from 8/12; no query left without an implementation hit).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Collection names — and the merkle snapshots that drive incremental sync — are keyed by the absolute codebase path (
md5(path)). So indexing the same repository at a second path re-embeds every file from scratch:Embeddings are a pure function of
(model, text), so this is pure waste — and on a CPU embedder (e.g. Ollamanomic-embed-text) embedding dominates indexing time, turning "index my other worktree" into minutes/hours.This is closely related to #271 (sharing across clones). Rather than collapse everyone onto one shared collection — which loses per-branch correctness and thrashes when multiple branches are indexed at once — this shares only the embedding work, keeping per-codebase collections independent.
What this adds
An opt-out, content-addressed embedding cache mapping
(modelIdentifier, chunk text) -> vector. Each unique chunk is embedded once ever and reused across every collection on the machine; different branches/worktrees stay independently searchable.FileSystemEmbeddingCache— zero-dependency, sharded, atomic-write (temp + rename, safe for concurrent worktree indexers) cache under~/.context/embedding-cache(configurable viaEMBEDDING_CACHE_DIR), mirroring howFileSynchronizerpersists under~/.context.CachedEmbedding— transparent decorator over anyEmbeddingprovider: splits each batch into cache hits/misses, only sends misses to the provider, preserves input order, and stays faithful to a misbehaving provider (never reshapes a mismatched/empty batch, so the caller's length validation still fires).Embedding.getModelIdentifier()— fully qualifies the model so two models can never share entries; overridden by each provider (Ollama,OpenAI,Gemini,VoyageAI) to include the model name.Context; enabled by default, opt out withEMBEDDING_CACHE=false.--helpand startup summary.Correctness
Caching never changes results — an embedding is deterministic for a given
(model, text), and the cache key mixes a fully-qualified model identifier with the exact text (NUL-separated). Two different models cannot return each other's vectors.Tests
cached-embedding.test.ts: unique-once, partial-batch misses, order preservation across hits/misses, model isolation, single-embed()round-trip.embedding-errortests bypass the cache (they assert on embedder calls); a jestsetupFilesredirects the cache to a temp dir so the suite never touches the developer's real~/.context.pnpm --filter @zilliz/claude-context-core test→ 34 passed.pnpm buildsucceeds for core + mcp.Measured
Re-indexing an identical tree at a new path drops from ~69s to ~3s (
nomic-embed-texton CPU), search results unchanged.🤖 Generated with Claude Code