Live-verify server/managed backend adapters; fix 8 bugs - #14
Merged
Conversation
Exercised the 9 previously-unverified backend adapters (issue #11) against real engines: Docker containers for pgvector / redis / elasticsearch / weaviate / mongodb, the embedded Milvus Lite engine for milvus, and a loadable-extension Python for sqlite_vec. All 344 contract tests now pass across 13 backends. Adapter bugs fixed: - sqlite_vec: vec0 tables reject INSERT OR REPLACE (delete+insert instead); a KNN query through a JOIN drops the required `k` constraint (subquery form); an over-broad `except OperationalError` silently swallowed query errors; a NULL distance for a zero-norm vector crashed scoring. - milvus: used tempfile.mkstemp (creates a file) but milvus-lite >=3.0 treats the path as a directory -> FileExistsError. - pgvector: the vector(N) type modifier cannot be a bound parameter; executemany is a cursor (not connection) method in psycopg 3; the query vector needs an explicit ::vector cast. - elasticsearch: lazily-created collections were invisible to list / contains / get (the index is now created eagerly, the dense_vector field added on first write); _keys sorted on the disallowed _id field (now uses helpers.scan). - weaviate: collections.get() never raises in v4, so _class_exists always reported True -> create / get / delete were broken (now collections.exists()). - mongodb: _raw_to_document was undefined (NameError in _read); the Atlas vector search index is now created and awaited automatically on the first search, and metadata filtering moved client-side (Atlas can only pre-filter on index-declared fields). Test harness: - tests/docker-compose.yml: one container per server backend. - tests/conftest.py: sweeps embedded + server backends; server backends are TCP-probed and skipped when unreachable, so CI stays green without them. - tests/README.md: how to run the server-backend suite. Also: de-degenerate three test inputs that used all-zero vectors (undefined cosine similarity, which Elasticsearch rejects); add milvus-lite to the milvus extra so the embedded engine works out of the box. Refs #11
This was referenced May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #13. Refs #11.
Live verification of the 9 backend adapters that #11 shipped
correct-by-construction but never ran against a real engine. Verified the 7
reachable without a cloud account;
pineconeandturbopufferremaindeferred (need accounts).
Result: all 344 contract tests pass across 13 backends. 8 real bugs found.
Bugs fixed
INSERT OR REPLACE→ delete+insert; KNN through a JOIN drops the requiredkconstraint → subquery form; over-broadexcept OperationalErrorswallowed query errors → explicit_tables_exist; NULL distance for a zero-norm vector crashed scoringtempfile.mkstempcreates a file, but milvus-lite ≥3.0 wants a directory path →FileExistsErrorvector(N)type modifier can't be a bound parameter;executemanyis a cursor method in psycopg 3, not a connection method; query vector needs an explicit::vectorcastlist/in/get→ index created eagerly,dense_vectorfield added on first write;_keyssorted on the disallowed_idfield →helpers.scancollections.get()never raises in v4, so_class_existsalways returnedTrue→ create/get/delete broken →collections.exists()_raw_to_documentwas undefined (NameErrorin_read); the Atlas vector search index is now auto-created + awaited on first search; metadata filtering moved client-side (Atlas can only pre-filter on index-declared fields)Test harness (reusable)
tests/docker-compose.yml— one container per server backend.tests/conftest.py— theclientfixture now sweeps embedded and serverbackends; server backends are TCP-probed and skipped when unreachable, so
CI stays green without containers.
tests/README.md— how to run it.Notable design decision
The mongodb adapter previously required the Atlas vector search index to be
created out-of-band. It now creates and awaits the index automatically on the
first search — making mongodb consistent with every other adapter (all
auto-create their index) and honoring the facade's "simple things simple"
principle. Needs an Atlas-capable deployment (Atlas /
mongodb-atlas-local).Test-input change
Three contract tests used all-zero vectors as "valid" inputs. A zero-magnitude
vector has undefined cosine similarity and Elasticsearch rejects it outright;
the inputs are now non-zero (assertions are filter/dimension-driven, so intent
is preserved). This makes the cross-backend contract honest — vd does not
promise zero-vector portability.
Verification environment
Run in an isolated venv (Python 3.11, all backend client libs) to keep the
heavyweight DB clients out of the main environment. milvus is verified against
the embedded Milvus Lite engine — same adapter code path as a server, only the
client constructor differs.