Reproducer:
- Check out latest main branches of cuvs and cuvs-lucene
- upload dataset alpha-123 (please ping me for access -- note requires NVIDIA access -- note alpha-123 is not the real name of dataset)
- run with automatic heuristics
This should reproduce the green line with poor search latency performance.
My hypothesis: I suspect the intrinsic dimension of the manifold within 1536d dataset isn't large enough to produce a preponderance of hub-vectors to account for the fact that we are running the benchmark on a baselayer-only graph.
What is apparent from extra tests: when I bypass the automatic heuristics with my own custom parameters, I get even worse search latency performance than the performance achieved by the automatic heuristics when I set cagraHnswLayers=1, but then the search-latency performance gradually increases until we basically match the expected search-latency at cagraHnswLayers=6.
Question: why can't we just increase the number of cagraHnswLayers while using the automatic heuristics?
Answer: we hit the "not enough KNN neighbors" RAFT errors when we try to increase the number of cagraHnswLayers above 1 (note: I typically see that behavior when nProbes is too low or nLists is too high)
Note: FAISS implements both of their CAGRA to HNSW conversions differently from cuvs-lucene, so this might only be an issue with cuvs-lucene?
Reproducer:
This should reproduce the green line with poor search latency performance.
My hypothesis: I suspect the intrinsic dimension of the manifold within 1536d dataset isn't large enough to produce a preponderance of hub-vectors to account for the fact that we are running the benchmark on a baselayer-only graph.
What is apparent from extra tests: when I bypass the automatic heuristics with my own custom parameters, I get even worse search latency performance than the performance achieved by the automatic heuristics when I set cagraHnswLayers=1, but then the search-latency performance gradually increases until we basically match the expected search-latency at cagraHnswLayers=6.
Question: why can't we just increase the number of cagraHnswLayers while using the automatic heuristics?
Answer: we hit the "not enough KNN neighbors" RAFT errors when we try to increase the number of cagraHnswLayers above 1 (note: I typically see that behavior when nProbes is too low or nLists is too high)
Note: FAISS implements both of their CAGRA to HNSW conversions differently from cuvs-lucene, so this might only be an issue with cuvs-lucene?