This repository contains the results and configuration of the five-domain retrieval study comparing keyword search and vector search performance. The results reflect derived MRR scores and indexing times without redistributing the original underlying textual passages (to adhere to license restrictions such as MS MARCO). For the full context and conclusions, read the study's blog post.
- CodeXGLUE
- MS MARCO
- SQuAD
- HotpotQA
- SciQ
- 10k documents ingested per corpus.
- 1k queries evaluated per corpus.
- Evaluated using MRR@10.
- Keyword Search: Tantivy 0.22.0 default analyzer
- Vector Search: ChromaDB ≥0.4.0 default index;
all-MiniLM-L6-v2, 384-dim - Chunking: No chunking
- Hardware: Apple M4, 16GB
- Execution Date: Jan 14 2026