Md/s1#1937
Draft
aschwartz12 wants to merge 1 commit into
Draft
Conversation
Introduce the metadata-exchange backend layer without wiring it in yet: the nixlMetadataBackend contract, the nixlMetadataContext interface (the agent-side operations a backend needs), and the self-contained P2P (socket) backend that implements the contract against the context. Nothing constructs these yet; the agent still uses its inline path. The manager that owns and dispatches to backends, and the cutover that removes the inline path, come in the following commit. Splitting the introduction out keeps that cutover reviewable on its own.
|
👋 Hi aschwartz12! Thank you for contributing to ai-dynamo/nixl. Your PR reviewers will review your contribution then trigger the CI to test your changes. 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR1 - refactor(metadata): backend contract and P2P backend
Branch
md/s1(bc4202cc), stacked on upstreamnixl/main. First slice ofthe 4-PR metadata-manager stack. Introduces the internal
nixlMetadataBackendcontract, a self-contained P2P backend implementation, and the
nixlMetadataContextinterface onnixlAgentData— without wiring any of itinto the agent yet.
On main, P2P metadata exchange is implemented inline in
nixlAgenton top of ashared comm-worker thread, with socket helpers living in
nixl_listener.cpp.This commit adds the new abstraction layer alongside the existing path. Runtime
behavior is unchanged: the agent still uses the inline comm-thread path.
What?
Adds three new source files and a partial
agent_data.hupdate:nixl_metadata_backend.h— backend contract (prepare*methods,needsWorker,serviceEvents)nixl_p2p_metadata_backend.{cpp,h}— self-contained P2P sockets and listenernixlMetadataContextinterface inagent_data.h— the port backends use toreach serialization and cache operations
No manager, no ETCD backend, no agent cutover, no
nixl_listener.cppdeletion.Why?
Establishes the transport boundary before the cutover. Reviewers can read the
contract and the P2P backend in isolation (~640 added lines) without the larger
manager/ETCD/agent rewrite that follows in PR2.
The P2P socket helpers move out of
nixl_listener.cppinto the backend class.They live in an anonymous namespace in the old file, so this slice compiles
cleanly alongside the existing listener.
How?
Contract -
nixlMetadataBackenddefines fourprepare*methods(
prepareSendLocal,prepareSendLocalPartial,prepareFetchRemote,prepareInvalidateLocal). Each runs on the caller thread, validates andserializes, returns a synchronous status, and hands back transport work as a
nixlWorkerTaskwrapped innixlPreparedOp.P2P backend -
nixlP2pMetadataBackendowns its sockets and listener logic(previously in
nixl_listener.cpp). It is self-contained but not yetinstantiated; PR2 wires it through the manager.
Context port -
nixlMetadataContextis declared inagent_data.hwith themethods backends need (
getLocalMD,getLocalPartialMD,loadRemoteMD,getName,getConfig,invalidateRemoteMD).nixlAgentDatadoes not yetimplement it; that lands in PR2.
Scope - Additive only. No public API change. No behavior change. The comm thread
and
nixl_listener.cppremain in place until PR2.Size
~639 lines added across 5 files. Under the 1000-line target.
Test plan
builddir) and the ETCD config (builddir-etcd).Stack
nixl/mainmd/s2) wires the manager, adds ETCD, flips the agent, deletesnixl_listener.cpp