Skip to content

Remove DataProvider::to_external_id - #1541

Merged
Mark Hildebrand (hildebrandmw) merged 1 commit into
mainfrom
mhildebr/remove-to-external-id
Oct 29, 2026
Merged

Mark Hildebrand (hildebrandmw) merged 1 commit into
mainfrom
mhildebr/remove-to-external-id

Conversation

@hildebrandmw

Copy link
Copy Markdown
Contributor

DataProvider::to_external_id is apparently not used anywhere in the core algorithm. There are currently two users of internal to external ID translation:

  • diskann-garnert: Just uses it for post-processing and works just fine with to_external_id as an inherent method on GarnetProvider.
  • diskann-inmem: Does do ID translation, but already uses a more batch-friendly version during its post-processing.

Removing this (outside of removing code that doesn't apparently have a need) helps the diskann-inmem work as we don't need to provide an extremely slow fallback if ID translation is moved behind the epoch protected boundary.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the removal of the unused DataProvider::to_external_id trait method.

No open issues 🎉

What changed in this PR

Removes the unused DataProvider::to_external_id trait method, leaving internal-to-external ID translation to provider-specific post-processing. This simplifies the core interface and avoids requiring a slow fallback for future epoch-protected translation in diskann-inmem.

Changes:

  • Remove the trait method and its implementations across providers.
  • Preserve Garnet’s translation logic as a private inherent method.
  • Remove test assertions for the deleted API.
File Description
diskann/​src/​provider.rs Remove to_external_id from the core trait.
diskann/​src/​graph/​test/​provider.rs Remove the test provider’s implementation and related assertions.
diskann/​src/​flat/​test/​provider.rs Remove the test provider’s implementation.
diskann-providers/​src/​model/​graph/​provider/​async_/​inmem/​provider.rs Remove identity translation and its test assertion.
diskann-label-filter/​src/​encoded_attribute_provider/​document_provider.rs Remove forwarding of the deleted method.
diskann-inmem/​src/​provider.rs Remove single-ID reverse translation and related assertions.
diskann-garnet/​src/​provider.rs Move reverse translation into an inherent method.
diskann-disk/​src/​search/​provider/​disk_provider.rs Remove identity translation and its test assertion.
diskann-bftree/​src/​provider.rs Remove identity translation and its test assertion.

Review effort: Balanced

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.02%. Comparing base (413be59) to head (e3bc17e).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1541      +/-   ##
==========================================
+ Coverage   91.01%   91.02%   +0.01%     
==========================================
  Files         602      602              
  Lines      124144   124080      -64     
==========================================
- Hits       112994   112949      -45     
+ Misses      11150    11131      -19     
Flag Coverage Δ
miri 91.02% <100.00%> (+0.01%) ⬆️
unittests 90.73% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
diskann-bftree/src/provider.rs 93.95% <ø> (-0.03%) ⬇️
diskann-disk/src/search/provider/disk_provider.rs 95.90% <ø> (-0.04%) ⬇️
diskann-garnet/src/provider.rs 90.18% <100.00%> (+0.08%) ⬆️
diskann-inmem/src/provider.rs 96.19% <ø> (-0.10%) ⬇️
...rc/encoded_attribute_provider/document_provider.rs 0.00% <ø> (ø)
.../src/model/graph/provider/async_/inmem/provider.rs 92.36% <ø> (-0.21%) ⬇️
diskann/src/flat/test/provider.rs 71.21% <ø> (+2.59%) ⬆️
diskann/src/graph/test/provider.rs 95.34% <ø> (-0.05%) ⬇️
diskann/src/provider.rs 98.18% <ø> (ø)

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hildebrandmw
Mark Hildebrand (hildebrandmw) merged commit 7aed6c2 into main Oct 29, 2026
32 checks passed
@hildebrandmw
Mark Hildebrand (hildebrandmw) deleted the mhildebr/remove-to-external-id branch October 29, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants