Skip to content

feat: publish index artifacts through transactions - #620

Merged
jiaqizho merged 6 commits into
milvus-io:mainfrom
yuruiz:feature/transaction-index-files
Aug 12, 2026
Merged

feat: publish index artifacts through transactions#620
jiaqizho merged 6 commits into
milvus-io:mainfrom
yuruiz:feature/transaction-index-files

Conversation

@yuruiz

@yuruiz yuruiz commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • publish completed index metadata through Transaction::AddIndexInfo
  • persist typed artifact identity, index name, file keys, sizes, row count, and engine versions in manifest Index
  • expose the same metadata through LoonIndexInfo and C FFI
  • remove ManifestMinorVersion; index presence is represented by manifest indexes
  • document the manifest-first, minimal-etcd QueryCoord/QueryNode load design

Testing

  • builder-container clang-format dry run
  • builder-container build of milvus_test and Test_FFI
  • focused manifest / transaction / FFI GTests (5 passed)
  • Test_FFI (78 tests, 0 failed)

@sre-ci-robot
sre-ci-robot requested review from sunby and tedxu August 11, 2026 05:50
@sre-ci-robot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yuruiz
To complete the pull request process, please assign tedxu after the PR has been reviewed.
You can assign the PR to them by writing /assign @tedxu in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sre-ci-robot

Copy link
Copy Markdown
Collaborator

Welcome @yuruiz! It looks like this is your first PR to milvus-io/milvus-storage 🎉

Comment thread cpp/include/milvus-storage/transaction/transaction.h Outdated
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.00000% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.42%. Comparing base (1fd5edf) to head (5bafce4).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
cpp/src/ffi/bridge.cpp 86.40% 14 Missing ⚠️
cpp/src/ffi/manifest_c.cpp 90.16% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #620      +/-   ##
==========================================
+ Coverage   76.19%   76.42%   +0.23%     
==========================================
  Files         173      173              
  Lines       17388    17561     +173     
  Branches     2618     2655      +37     
==========================================
+ Hits        13248    13421     +173     
  Misses       4140     4140              
Flag Coverage Δ
cpp 79.01% <90.00%> (+0.22%) ⬆️
python 44.45% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread cpp/include/milvus-storage/ffi_c.h Outdated
Comment thread cpp/include/milvus-storage/manifest.h Outdated
Comment thread cpp/src/ffi/bridge.cpp Outdated
Comment thread cpp/include/milvus-storage/ffi_c.h
Comment thread cpp/src/transaction/transaction.cpp Outdated
Comment thread cpp/include/milvus-storage/manifest.h Outdated
Comment thread cpp/src/manifest.cpp Outdated
Comment thread cpp/include/milvus-storage/ffi_c.h
@yuruiz

yuruiz commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the latest review comments in commit 5bafce4:

  • kept Index::properties as std::unordered_map for the common key-lookup path
  • split legacy raw MILV index decoding by manifest version; v2-v5 use the four-field layout, while v6+ uses the typed layout
  • added compatibility comments and kept LoonIndexInfo limited to completed artifact metadata; IsDeleted/IndexState remain transient build-task state

Validation: legacy v2/v3 tests passed and Test_FFI passed 78/78.

@jiaqizho

Copy link
Copy Markdown
Collaborator

/lgtm

@jiaqizho
jiaqizho added this pull request to the merge queue Aug 12, 2026
Merged via the queue into milvus-io:main with commit dac5781 Aug 12, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants