⚡ Bolt: Optimize Vector DB Search (120x faster) - #11
⚡ Bolt: Optimize Vector DB Search (120x faster)#11google-labs-jules[bot] wants to merge 1 commit into
Conversation
Optimizes the `HNSWIndex` search method in `memory/lightweight_vector_db.py` by replacing the Python iterative loop with vectorized NumPy operations. Changes: - Introduces `_matrix` cache to store vectors in a contiguous NumPy array. - Implements `_update_matrix` to lazily rebuild the cache when data changes. - Replaces loop-based cosine distance calculation with `np.dot` matrix multiplication. - Pre-calculates vector norms to avoid redundant computations. Performance Impact (10k vectors): - Cold Search: Reduced from ~107ms to ~77ms (includes cache build). - Hot Search: Reduced from ~96ms to ~0.8ms (~120x faster). - Mixed Add/Search: Reduced from ~9.25s to ~1.91s (~4.8x faster).
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
⚡ Bolt Optimization Report
💡 What:
Replaced the iterative O(N) Python loop in
HNSWIndex.searchwith vectorized NumPy matrix operations. Implemented a caching mechanism (_matrixand_matrix_norms) that updates lazily when new vectors are added.🎯 Why:
The original implementation calculated cosine distance one-by-one in a Python loop, which is extremely slow for large datasets. This created a performance bottleneck in memory retrieval operations.
📊 Impact:
Based on benchmarks with 10,000 vectors:
🔬 Measurement:
Verified using a reproduction script that populated 10k vectors and performed search operations.
All existing integration tests (
test_ananta_quick.py) passed.PR created automatically by Jules for task 970480832259386966 started by @Rohith-Shimori