Add mxbai dense + bm25 hybrid search to the backend

[jkupchanko]

Reworks the search backend to dense + hybrid retrieval and makes the collection reproducible from the repo.

What changed

• Dense vectors move from all-MiniLM (384-d) to mxbai-embed-large-v1 (1024-d).
• Adds a Qdrant/bm25 sparse vector; hybrid fuses dense + sparse with RRF. No reranker.
• Both vectors embed server-side through Cloud inference, so ingest and query run the identical models. The hand-rolled sparse encoder is gone.
• init_collection_startups.py builds the collection the search path expects: named dense and sparse vectors, modifier=IDF on the sparse index, and the text index on document. Renames the payload once at ingest to document/logo_url/homepage_url.

API

• GET /api/search?q=&mode=semantic|keyword|hybrid, hybrid by default.
• Back-compat: the old neural flag still works. neural=true is semantic, neural=false is keyword.

Notes

• mxbai query prefix: added, and measured. On the Cloud-inference path it's a no-op (identical scores, 5/5 top-5 overlap across five queries) because Cloud already applies mxbai's query prompt server-side. Kept for the self-hosted path.
• qdrant-client bumped to 1.19.0 with a regenerated lock. 1.14.2 can't talk to the current server for inference. Dropped the fastembed extra since the app embeds via Cloud inference.

Reproduce

docker run -p 6333:6333 qdrant/qdrant
python -m qdrant_demo.init_collection_startups
curl 'localhost:8000/api/search?q=ai&mode=hybrid'

[kanungle]

these results don't seem that good. Ranking feels weak. Can we also show more than 10 results?

[kanungle]

Why was prefer_grpc removed?

[jkupchanko]

Hey Neil, good catch. Ran a bunch of tests to pin down where the ranking was falling apart. It wasn't the index, that's returning the right matches. It was the embedding model (all-MiniLM), too lightweight to order 3M results well. So I kept it but added a re-ranker on top (ms-marco-MiniLM-L-6-v2) that re-scores the top matches. Threw in a small stats line too so you can see the models and latency on each search, and bumped results from 10 to 20.

[kanungle]

Please uplift this demo to include:

1. a higher dimension embedding model
2. no reranker
3. hybrid search option (maybe with scoreboosting)

Add context in the UI that this is a 'simple' comparison of different vector search methods

[jkupchanko]

The model is now mxbai at 1024D, the reranker is gone, and hybrid is in as a toggle. Re-embedded all 3M, and in a before/after the hybrid wins 15 of 18 queries and runs faster (~210ms).

[kanungle]

demo looks good but there are merge conflicts

[jkupchanko]

Sorry about the conflicts. Fixed now, it's backend-only so it drops cleanly onto the current frontend. mxbai + hybrid, no reranker, and the existing neural= param still works, so nothing on the frontend needs to change.

[kanungle]

Heads up, this needs another pass before it lands.

I was clicking around the live preview and noticed hybrid search latency was high for a dataset this size. That sent me into the code to find out why. Full write-up below, with additional findings from claude code as well:

On the latency, since that is what started this. CLOUD_INFERENCE defaults to "0" in config.py, so unless the deploy sets it, every query embeds locally with mixedbread-ai/mxbai-embed-large-v1. That is 335M parameters on CPU, per request, replacing the 22M-parameter MiniLM this repo ran before. The Dockerfile also dropped the fastembed prewarm line without putting anything in its place, so the first request after a cold start pulls roughly 1.2GB before it can answer anything. The 60 second client timeout added in neural_searcher.py reads as a workaround for that rather than a fix.

Why it matters: this is the startup search demo. Slow first impressions are the one thing it cannot afford.

Suggested fix: default CLOUD_INFERENCE to 1 so the query embeds server side, or restore a prewarm step in the Dockerfile for the new model. Then drop the timeout back to something that fails loudly instead of hanging for a minute.

The structural problem underneath all of this. My understanding is that you generated the embeddings locally and uploaded them to Qdrant Cloud, and the app now queries that collection. Mixing the two is fine in itself, both sides run the same mxbai weights with the same pooling and normalization, so the vectors line up. The problem is that the script that built startups_hybrid is not in this PR and not in the repo. The read path here was rewritten against a collection nobody else can rebuild. Please correct me if I have the ingest story wrong, because a couple of the items below depend on it.

---

Must fix (blocks shipping)

• The script that built the collection is not in the repo. init_collection_startups.py is untouched by this PR. It still builds a collection with a single unnamed dense vector, no sparse_vectors_config, and payload key document. Meanwhile neural_searcher.py:32,40,43 queries using="dense" and using="sparse" against COLLECTION_NAME, now defaulting to startups_hybrid.

  Why: run this repo's own init script and both semantic and hybrid search return 400, "vector name dense does not exist". The sparse leg has no vector to hit at all. A local one-off upload is fine for getting the preview standing up, but it means the demo is only reproducible on your machine, and the next person who re-indexes breaks it with no way to recover.
  
  Suggested fix: fold whatever you ran locally into `init_collection_startups.py` in this same PR. Named `dense` vector, a `sparse_vectors_config` entry for `sparse`, and the same encoder writing documents that the searcher uses for queries.
  
  Reproduce:
  ```
  docker run -p 6333:6333 qdrant/qdrant
  python -m qdrant_demo.init_collection_startups
  curl 'localhost:8000/api/search?q=ai&mode=hybrid'
  ```
  

• The TEXT_FIELD_NAME flip breaks keyword search. config.py changes it from "document" to "description", but init_collection_startups.py renames description to document in the payload and creates the text index on that same constant.

  Why: against any collection built by this repo the keyword filter now targets a field with no index and no data, so it returns zero results. If a record ever did come back, `text_searcher.py:15` does `record["description"]` and raises `KeyError`. One constant is doing two jobs, payload key on write and query key on read, so flipping it silently changes what future collections index too.
  
  Suggested fix: keep `TEXT_FIELD_NAME` consistent with whatever init actually writes. If the payload key should become `description`, change the rename in `read_points()` in the same commit and re-index.
  

• A stopword-only query builds a malformed sparse vector. to_sparse("the a of it") returns ([], []). I ran it to confirm. service.py:38 only guards against an empty string, so an empty SparseVector reaches the prefetch.

  Why: real users type short queries. This is a 500 on input that should return zero results.
  
  Suggested fix: if `indices` comes back empty, fall back to the dense-only path instead of building the prefetch. Related: the `len(w) > 1` filter drops single-character tokens, so `"C++ IDE"` tokenizes to `["ide"]` and loses the language. On a startups dataset that is the term people search for.
  

• IDF is never configured anywhere. The sparse.py docstring asserts "Qdrant applies IDF at query time (sparse index modifier=IDF)", but nothing in the repo sets modifier=Modifier.IDF. If your local upload script set it, that is another thing living outside version control.

  Why: index-time values carry the term-frequency component only, by design. Without the modifier the keyword leg ranks on raw term frequency, so long documents that repeat common words win. This one fails silently, which is worse than the 400s above, because the demo looks like it works and just returns bad results.
  
  Suggested fix: `sparse_vectors_config={"sparse": models.SparseVectorParams(modifier=models.Modifier.IDF)}` at collection creation, or move to the option below and get it for free.
  

Should fix

• mxbai is missing its query prefix. fastembed's own registry entry for this model reads "Prefixes for queries/documents: necessary", but fastembed never applies one. query_embed falls through to the base implementation, which is plain embed with no prefix.

  Why: mxbai was tuned for retrieval with `"Represent this sentence for searching relevant passages: "` on the query and nothing on the document. Right now neither side gets a prefix, so nothing is mismatched, you are just leaving retrieval quality on the table with a model chosen specifically for its quality. To be clear this is not a correctness break, and mixing locally-generated document embeddings with Cloud-inference queries is fine on its own, both paths run the same model with the same pooling and normalization.
  
  Suggested fix: prepend the prefix to the query text only, in `_dense()`. Documents stay as they are, so no re-indexing is needed. Worth measuring the recall difference on a few queries and putting the number in the PR description, since that is the kind of result this demo should be showing off.
  

• The hand-rolled sparse encoder should not exist. fastembed already ships Qdrant/bm25, reachable through the same models.Document(text=..., model="Qdrant/bm25") pattern this PR already uses for the dense side.

  Why: real IDF, stemming, and language-aware stopwords, with document and query encoding guaranteed consistent by construction rather than by a comment promising they are. That consistency guarantee matters more than usual here, given that ingest and query are running from different scripts. The custom version also masks crc32 to 31 bits despite the comment saying u32, and hashed indices cannot be mapped back to terms when a result looks wrong.
  
  Suggested fix: swap both sides to `Qdrant/bm25` and delete `sparse.py`. That resolves the IDF item above as well.
  

• The Dockerfile client bump is unnecessary and leaves the container out of sync with the lockfile. pip install -U "qdrant-client==1.18.0" runs on top of a pyproject.toml pin of 1.14.2.

  Why: the stated reason is wrong. I checked the 1.14.2 wheel and it already carries `cloud_inference` and the prefetch and fusion API. So the bump buys nothing, and now the container runs a version no lockfile records.
  
  Suggested fix: drop the line, or bump the pin in `pyproject.toml` and regenerate `poetry.lock` properly.
  

• The back-compat mapping is not back-compat. service.py:37 maps neural=true to hybrid, and unset also defaults to hybrid.

  Why: `neural=true` previously meant dense-only. The old frontend can no longer request the mode it is asking for, and it gets a different one without knowing. The PR description says "Semantic and keyword search both work", but the mode actually served by default is hybrid, which that claim does not cover.
  
  Suggested fix: map `neural=true` to `semantic` and make hybrid opt-in through `mode=hybrid`. If hybrid should be the new default, say so in the description and update the frontend to ask for it by name.
  

• score is not comparable across modes. Dense returns cosine, roughly 0 to 1. Hybrid returns RRF, roughly 1/60 or 0.016.

  Why: same field name, two scales. Any UI rendering it as a relevance number looks broken the moment someone toggles modes.
  
  Suggested fix: either normalize before returning, or add the scale to the stats block so the frontend knows what it is displaying.
  

• Error handling leaks internals and is inconsistent. service.py:46 returns str(e)[:300] to the browser, which exposes the Qdrant URL and connection detail. service.py:59 swallows the same class of failure into a 200 with an error key.

  Why: a public demo should not hand out its backend topology, and two endpoints in the same file should not disagree about what failure looks like.
  
  Suggested fix: log the detail server side, return a generic message to the client, and pick one convention for both endpoints.
  

---

Short version: the read path was rewritten and the write path stayed on your laptop. Get the real ingest script into init_collection_startups.py and most of the Must-fix list goes with it. Happy to look again once that is in.

[jkupchanko]

Fixed the issues from your review, main one being the reproducible ingest script.

[jkupchanko]

Oh okay, I just finished everything. Sorry it took me a bit, I got held up on two things. The mxbai prefix turned out to be a no-op on cloud, since Qdrant already applies the query prompt server-side (measured it, recall came back identical). And the locked client 1.14.2 can't actually talk to the current server for inference, it fails parsing the response, so that one needed a real bump.

Here's what I did:

1. The ingest script is in the repo now and builds the collection the search expects, so your reproduce steps work.
2. Moved sparse to Qdrant bm25 and deleted the hand-rolled one, so ingest and query use the same model. Covers the IDF point too.
3. Field names are all on document now, renamed once at load, so keyword search works.
4. Stopword and punctuation queries don't crash, and C++ survives in hybrid.
5. Cloud inference defaults on, so no local download at boot, and neural=true is semantic again.
6. Bumped qdrant-client to 1.19 in pyproject and regenerated the lock instead of the pip line, dropped the fastembed extra since we don't embed locally anymore. Had to widen the python range a bit so the lock would regenerate, container still runs 3.11.
7. Lowered the timeout to 15 seconds. And errors come back generic now instead of leaking the url.

Score also carries a type now, cosine vs rrf, so the UI knows the scale.

[kanungle]

@jkupchanko where's the link to the demo? I can't see it in the PR anymore

[kanungle]

This looks good. Can we remove the latency indicator completely since this isn't a performance demo and may be critiqued?

[jkupchanko]

Ah yeah, here it is: https://qdrant-startup-search-production.up.railway.app

That's the live one, running the 3M hybrid setup from this PR. Toggle between semantic, keyword, and hybrid up top.

[jkupchanko]

And here's the main public one if that's easier: https://qdrant-demo-qvpw.vercel.app/

Same backend, just the hosted frontend.