Repository navigation

Reduce Pipnn Memory Consumption #1030

Closed

Assignees

Labels

Milestone

weiyaoluo (SeliMeli)

opened

Currently Pipnn consume more memory than Vamana, we need to explore options to reduce its mem footprint:

reduce mem consumption by review/profiling/fix code issues in current implementation.
use split build (partition + extract knn -> write knn graph on disk -> hashprune/final prune)
like vamana, chunk build dataset and merge

weiyaoluo (SeliMeli)
added
enhancementNew feature or request
on May 7, 2026
weiyaoluo (SeliMeli)
added this to DiskANN backlogon May 7, 2026
weiyaoluo (SeliMeli)
moved this to Todo in DiskANN backlogon May 7, 2026
Wei Wu (wuw92)
added a sub-issue on May 8, 2026
- Pipnn partition not follow c_min parameter and create too many small cluster #1035
Wei Wu (wuw92)
removed a sub-issue on May 8, 2026
- Pipnn partition not follow c_min parameter and create too many small cluster #1035
Alex Razumov (arrayka)
added this to the 2026-05 milestone on May 8, 2026
Alex Razumov (arrayka)
added a parent issue on May 8, 2026
- Support PiPNN builds #878
Alex Razumov (arrayka)
assigned
weiyaoluo (SeliMeli)
on May 8, 2026
SeliMeli commented on Aug 24, 2026
weiyaoluo (SeliMeli)
on Aug 24, 2026
ContributorAuthor
pipnn mem consumption is now match vamana fp build
weiyaoluo (SeliMeli)
closed this as completed on Aug 24, 2026
github-project-automation
moved this from Todo to Done in DiskANN backlogon Aug 24, 2026

to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

weiyaoluo (SeliMeli)

Labels

Type

No type

Projects

DiskANN backlog
Status
Done

Milestone

2026-05

Relationships

None yet

Development

No branches or pull requests