Skip to content

Add vecdb (PyPI: vecdbc) - #631

Open
navid72m wants to merge 3 commits into
erikbern:mainfrom
navid72m:main
Open

Add vecdb (PyPI: vecdbc)#631
navid72m wants to merge 3 commits into
erikbern:mainfrom
navid72m:main

Conversation

@navid72m

Copy link
Copy Markdown

vecdb is a from-scratch vector database in C11: HNSW graph with hand-written SIMD distance kernels (AVX-512 on x86, NEON on ARM). Published on PyPI as vecdbc; the Dockerfile installs it from there.

Adapter: ann_benchmarks/algorithms/vecdbc/ (module.py, config.yml, Dockerfile)
Handles euclidean directly and angular via L2-normalization
Single-threaded per the framework's convention
Tested locally: passes random-xs-20-euclidean (recall 1.0, exit 0 across all parameter groups) and runs cleanly on sift-128-euclidean
Added to CI test list for random-xs dataset

@navid72m

Copy link
Copy Markdown
Author

Verification: recall/QPS on sift-128-euclidean

Ran the adapter through the standard harness. Recall@10 vs the provided ground truth, single-thread, M=16 / efConstruction=200, sweeping ef:

ef recall@10 QPS
10 0.740 20,262
20 0.858 18,177
50 0.954 9,041
100 0.985 5,075
200 0.996 2,812
400 0.999 1,538

(Numbers above measured on Apple Silicon; the Docker harness on x86 will use the AVX2 build path. Posting these to show the adapter produces a correct, monotonic precision/recall frontier — recall climbs smoothly with ef and matches a reference HNSW implementation to ~3 decimals on this dataset.)

Build time on SIFT1M stays well under the "several hours" guideline (~hundreds of seconds at these parameters). The config sweeps M ∈ {12,16,24,32,48} so the framework can pick the precision/performance frontier rather than a single fixed point.

Also verified the angular path: vecdb uses squared-L2, so angular datasets are handled by L2-normalizing both base and query vectors (cosine ranking == L2 ranking on unit vectors), giving correct recall against cosine ground truth.

Happy to adjust parameters, the sweep, or anything else if it would make the benchmark more useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant