Skip to content

[diskann-garnet] Support configurable start points and allow importing start points - #1464

Open
Jack Moffitt (metajack) wants to merge 2 commits into
push-ttwwvqlmswwsfrom
push-qtrvttpzkznv
Open

Jack Moffitt (metajack) wants to merge 2 commits into
push-ttwwvqlmswwsfrom
push-qtrvttpzkznv

Conversation

@metajack

@metajack Jack Moffitt (metajack) commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Iceberg indices set an arbitrary start point id, so Garnet reserving id 0 made index import awkward.

This PR removes the hardcoded use of id 0 for the start point, and removes tracking its state from the FSM. The default start point is now u32::MAX. It also adds the ability for Garnet to control the start point id and import start point terms.

@metajack Jack Moffitt (metajack) changed the title push qtrvttpzkznv [diskann-garnet] Support configurable start points and allow importing start points Oct 6, 2026
@metajack
Jack Moffitt (metajack) marked this pull request as ready for review October 6, 2026 19:41
@metajack
Jack Moffitt (metajack) added this pull request to stack #1466 October 6, 2026 19:44
@metajack
Jack Moffitt (metajack) requested review from a team and a balanced review from Copilot October 7, 2026 21:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Import finalization can incorrectly reject valid external IDs that match the start-point ID bytes.

1 open finding
What changed in this PR

Adds configurable Garnet start-point IDs and supports importing start-point terms while excluding them from FSM accounting.

Changes:

  • Threads configurable start-point IDs through the FFI and provider.
  • Reserves the start-point ID outside the free-space map.
  • Adds import validation, tests, documentation, and a major version bump.
File Description
diskann-garnet/​src/​provider.rs Implements configurable and importable start points.
diskann-garnet/​src/​fsm.rs Excludes the reserved ID from allocation and tracking.
diskann-garnet/​src/​lib.rs Adds the start-point ID to the FFI.
diskann-garnet/​src/​test_utils.rs Updates provider test construction.
diskann-garnet/​src/​ffi_tests.rs Tests configurable and imported start points.
diskann-garnet/​src/​ffi_recall_tests.rs Updates recall tests for the new FFI.
diskann-garnet/​docs/​ffi-design.rs Documents the new FFI parameter.
diskann-garnet/​docs/​data-design.md Documents start-point storage semantics.
diskann-garnet/​Cargo.toml Bumps the crate to 7.0.0.
diskann-garnet/​diskann-garnet.nuspec Bumps the package to 7.0.0.
Cargo.lock Records the new crate version.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

.write_iid(&context.term(Term::Neighbors), 0, &ns)
{
return (true, Some(false));
for term in [Term::Attributes, Term::ExtMap, Term::IntMap] {
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Oct 7, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.68173% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.04%. Comparing base (dd739d1) to head (626a787).

Files with missing lines Patch % Lines
diskann-garnet/src/provider.rs 98.28% 7 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@                  Coverage Diff                  @@
##           push-ttwwvqlmswws    #1464      +/-   ##
=====================================================
+ Coverage              90.98%   92.04%   +1.05%     
=====================================================
  Files                    586      586              
  Lines                 119020   119379     +359     
=====================================================
+ Hits                  108294   109883    +1589     
+ Misses                 10726     9496    -1230     
Flag Coverage Δ
miri 92.04% <98.68%> (+1.05%) ⬆️
unittests 91.98% <98.68%> (+1.30%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
diskann-garnet/src/fsm.rs 98.58% <100.00%> (+0.17%) ⬆️
diskann-garnet/src/lib.rs 93.81% <100.00%> (+0.11%) ⬆️
diskann-garnet/src/test_utils.rs 98.69% <ø> (ø)
diskann-garnet/src/provider.rs 90.00% <98.28%> (+1.75%) ⬆️

... and 47 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants