Repository navigation
[CI] Run SDE tests for diskann with all features - #1440
Merged
Merged
Conversation
weiyaoluo (SeliMeli)
requested review from
a team
and
a lite review from Copilot
September 24, 2026 05:16
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
No unresolved review issues were identified.
Review effort: Lite
Findings: None
What changed in this PR
Extends SDE CI coverage to test diskann with all features enabled.
Changes:
- Adds
diskannto both SDE test jobs. - Enables
--all-featuresfor Nehalem and Sapphire Rapids runs.
| File | Description |
|---|---|
.github/workflows/ci.yml |
Expands SDE coverage for both jobs. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1440 +/- ##
=======================================
Coverage 90.58% 90.58%
=======================================
Files 567 567
Lines 112317 112317
=======================================
+ Hits 101737 101743 +6
+ Misses 10580 10574 -6
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Mark Hildebrand (hildebrandmw)
approved these changes
Sep 24, 2026
Harsha Simhadri (harsha-simhadri)
approved these changes
Sep 25, 2026
Mark Hildebrand (hildebrandmw)
added a commit
that referenced
this pull request
Oct 5, 2026
# DiskANN v0.60.0 ## Major Changes ### `diskann-utils`: Matrix API changes There have been significant changes to the matrix APIs in `diskann-utils`. The previous contents of `diskann_utils::views` have been moved to `diskann_utils::views::rowmajor`. The following type changes have been made | v0.59 | v0.60 | Remarks | |-|-|-| | `views::Matrix<T>` | `views::rowmajor::Owned<T>` | Size changed from 32 bytes to 24 bytes | | `views::MatrixView<'a, T>` | `views::rowmajor::Ref<'a, T>` | Size changed from 32 bytes to 24 bytes | | `views::MatrixViewMut<'a, T>` | `views::rowmajor::Mut<'a, T>` | Size changed from 32 bytes to 24 bytes | | `MatrixBase` inherent methods | `rowmajor::Matrix` and `rowmajor::MatrixMut` trait methods | | The `rowmajor::Matrix` and `rowmajor::MatrixMut` trait methods need to be imported to access most matrix functionality. In addition, the following methods have been renamed and changed: | v0.59 | v0.60 | Remark | |-|-|-| | `row_iter` | `rows` | No longer panics when `ncols == 0` | | `row_iter_mut` | `rows_mut` | No longer panics when `ncols == 0` | | `par_row_iter` | `par_rows` | No longer panics when `ncols == 0` | | `par_row_iter_mut` | `par_rows_mut` | | | `get_row_unchecked` | `row_unchecked` | | | `get_row_unchecked_mut` | `row_unchecked_mut` | | | `as_mut_view` | `as_view_mut` | | | `try_from` constructors | `try_from_data` | | In addition, * The `Index` and `IndexMut` implementations have been removed. Use `Matrix::element` and `Matrix::element_mut` instead. * The `Init` and `Generator` style constructors for `Matrix::new` have been removed. Instead, `rowmajor::Owned::from_fn` and `rowmajor::Owned::from_element` can be used instead. The former's closure has been changed to take an argument `rowmajor::RowCol` indicated the row and column being initialized. Both of these constructors take now take the initializer as the last argument. * Matrix constructors validate against integer overflow and `isize::MAX` allocations via a `rowmajor::Layout<T>` layout. Constructors panic if these validations fail. `try_*` variations of these constructors which return `rowmajor::LayoutError` instead. ### `diskann-utils`: Strided API changes * Structs `StridedBase`, `StridedView`, and `MutStridedView` were collapsed into a single, immutable `Strided<'a, T>`. * `strided::linear_length` was removed, use `strided::Layout::linear_length` instead. * `strided::TryFromErrorLight` was removed. * `strided::TryFromError` changed from a struct to an enum, and `into_inner`/`as_static` were removed. ### `diskann` `InsertStrategy` is no longer a `SearchStrategy` subtrait. Implementations must now provide ```rust type SearchAccessor; type SearchAccessorError; fn insert_search_accessor(...) -> Result<Self::SearchAccessor, Self::SearchAccessorError>; ``` Previously, `insert_search_accessor` had a provided implementation forwarding to `SearchStrategy::search_accessor`. That default is now gone. This change was done to decouple insert element types from search element types. ### `diskann-disk` The publicly re-exported types below were removed in favor of standard buffered I/O: ``` diskann_disk::storage::CachedReader diskann_disk::storage::CachedWriter ``` Consumers should migrate to `std::io::BufReader`/`BufWriter` and the relevant storage-provider reader/writer. `READ_WRITE_BLOCK_SIZE` also changed from `u64` to `usize`. ### `diskann-quantization` These public traits now require `Debug`: ``` spherical::iface::Quantizer spherical::iface::DynQueryComputer spherical::iface::DynDistanceComputer ``` ### `diskann-inmem` (experimental) The old `layers` abstraction was removed and replaced by the `Representation`/`Store`/`Slots` architecture. And support for a spherically quantized backed was added. # Full Change Log * [diskann-inmem] Prepare code for quantization and beyond by @hildebrandmw in #1352 * Add Neon inner product U8xU1, U8xU2 and U8xU4 kernels by @pfoxARM in #1372 * [CI] Check all workspace features and private docs by @wuw92 in #1400 * [CI] Check Rust license headers by @wuw92 in #1401 * [disk] Replace custom cached I/O with standard buffered readers and writers by @wuw92 in #1396 * [dikkann_wide] Integer SIMDMinMax by @suri-kumkaran in #1407 * Add Neon Squared L2 kernels for USlice2 and USlice4 by @pfoxARM in #1398 * Bump the github-actions group with 3 updates by @dependabot[bot] in #1406 * [CI] Check workspace direct dependency versions by @wuw92 in #1410 * Train fresh disk-search PQ codebooks for each index build by @xwj-ox in #1299 * [diskann-quantization] Make most things in `diskann-quantization/spherical/iface.rs` debuggable by @hildebrandmw in #1414 * [diskann] Make `InsertStrategy` independent of `SearchStrategy` by @hildebrandmw in #1393 * Use #[expect] to detect unnecessary lint suppressions by @wuw92 in #1409 * [diskann-garnet] Store attributes during insert, not after by @metajack in #1412 * [diskann-wide] Fix `alias!` by @hildebrandmw in #1417 * [diskann-utils] Simplify `StridedView`. by @hildebrandmw in #1376 * [diskann-wide] More Neon Zip/Unzip impls by @hildebrandmw in #1405 * [diskann-garnet] Deserialize max id for quantization on load by @metajack in #1419 * More ergonomic `spherical::iface::Quantizer` constructors by @hildebrandmw in #1418 * [diskann-bftree] Clamp number of concurrency test parallel workers by @hildebrandmw in #1433 * Tweak behavior of adaptive L by @magdalendobson in #1354 * Bump the github-actions group with 2 updates by @dependabot[bot] in #1436 * Reform Range Search by @magdalendobson in #1337 * PiPNN 2/6: add numerical kernels by @SeliMeli in #1287 * Matrix overhaul 1 of N - Data Accessors by @hildebrandmw in #1415 * Add `assert_contains!` helper by @hildebrandmw in #1421 * Matrix overhaul 2 of N - `Matrix` invariants by @hildebrandmw in #1420 * [CI] Run SDE tests for diskann with all features by @SeliMeli in #1440 * [diskann-wide] Add Neon `vabdq_u8` and `vabdq_s8`. by @hildebrandmw in #1437 * [diskann-inmem] Add support for spherical quantization by @hildebrandmw in #1432 * [benchmark-runner] Bump edition to 2024 by @hildebrandmw in #1444 * [diskann-wide] Add byte-vector reinterpretations for u32x8 and u64x8 by @partychen in #1445 * Propagate invalid matrix layout errors in production paths by @arrayka in #1453 * Add sparse vector distance kernels (f32/f16 L2, inner product, cosine) by @mana-agarwal in #1434 * Bump the github-actions group with 3 updates by @dependabot[bot] in #1454 * [diskann-utils] Matrix Update 3 of N (constructors) by @hildebrandmw in #1451 * [diskann-utils] Matrix Update 4 of 4 by @hildebrandmw in #1457 ## New Contributors * @mana-agarwal made their first contribution in #1434 **Full Changelog**: v0.59.0...v0.60.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
Follow-up to the #1287 review: #1287 (comment)
What does this implement/fix? Briefly explain your changes.
Both SDE jobs (
sde-baseline-testson Nehalem andsde-avx512-testson Sapphire Rapids) now also testdiskannand pass--all-features.pipnnenabled, the PiPNN kernel tests run under SDE. Under-sprthey also cover the V4 (AVX-512) top-k path. Runner CPUs vary, so no other CI job is guaranteed to reach that path.--all-featuresalso runs the non-defaultdiskann-quantizationfeatures (linalg,flatbuffers,codegen) under SDE.Any other comments?
Local check with SDE 10.8.0 and the same commands and environment as CI: both jobs pass. Wall time from a cold target directory was 213 s (Nehalem) and 297 s (Sapphire Rapids), including compilation. The slowest test binary is the
diskann-quantizationlib tests under-spr(80 s).