Md/s3#1939
Draft
aschwartz12 wants to merge 3 commits into
Draft
Conversation
Introduce the metadata-exchange backend layer without wiring it in yet: the nixlMetadataBackend contract, the nixlMetadataContext interface (the agent-side operations a backend needs), and the self-contained P2P (socket) backend that implements the contract against the context. Nothing constructs these yet; the agent still uses its inline path. The manager that owns and dispatches to backends, and the cutover that removes the inline path, come in the following commit. Splitting the introduction out keeps that cutover reviewable on its own.
Wire the backend layer in and remove the inline path. nixlAgentData now owns a nixlMDManager (built unconditionally) that owns the worker thread and dispatches each call to a backend by precedence: a peer address selects P2P, otherwise the configured centralized store (ETCD). Add the self-contained ETCD backend (owns its nixlEtcdClient). nixlAgent's exchange methods (sendLocalMD, sendLocalPartialMD, fetchRemoteMD, invalidateLocalMD) become thin wrappers over the manager; pure-cache operations are consolidated on nixlAgentData, which implements nixlMetadataContext for the backends. nixl_listener.cpp is deleted: its worker loop moves into the manager, its socket helpers into the P2P backend, and its etcd client into the ETCD backend. Public API and observable P2P/ETCD behavior are unchanged; gtest log expectations are updated to the new backend messages.
Add md_manager.cpp covering the manager's routing and the P2P and ETCD backends (send/fetch/invalidate by peer address and by name). ETCD cases are gated on a live NIXL_ETCD_ENDPOINTS endpoint.
|
👋 Hi aschwartz12! Thank you for contributing to ai-dynamo/nixl. Your PR reviewers will review your contribution then trigger the CI to test your changes. 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR3 - test(metadata): nixlMDManager gtests for P2P and ETCD
Branch
md/s3(8cfa5674), stacked on PR2 (md/s2). Third slice of the4-PR metadata-manager stack. Adds gtest coverage for the manager and its P2P
and ETCD backends. No production code changes.
What?
Adds
test/gtest/md_manager.cpp(~272 lines) and registers it intest/gtest/meson.build. Covers:skipped without
NIXL_ETCD_ENDPOINTS)Why?
Keeps the large cutover in PR2 reviewable without also carrying new test code.
Reviewers can validate manager behavior in a dedicated, small PR after the
production cutover lands.
How?
MDManagerFixtureexercises the manager end-to-end over real sockets (P2P) or alive etcd endpoint (ETCD). Tests use the same public agent API
(
sendLocalMD,fetchRemoteMD, etc.) that production callers use.Scope - Tests only. No public API change. No production code change. TCPStore
tests land in PR4.
Size
~273 lines added across 2 files. Under the 1000-line target.
Test plan
NIXL_ETCD_ENDPOINTS=host:port ./builddir-etcd/test/gtest/nixl_gtest --gtest_filter=MDManagerFixture*Stack
md/s2)md/s4) adds TCPStore backend and its gtests