add shared KV backend abstraction layer

[hxieustc]

Summary

This PR adds src/plugins/kv/ as the canonical home for all KV-family backend plugins and introduces nixlKVBackendBase, a shared abstract base class that factors out the boilerplate that existing KV plugins use (PR #1791 (REDIS) and PR #1903 (WQSKV) independently duplicated).

Neither plugin is moved or modified by this PR — it purely establishes the abstraction so both can be adapted on top of it.

---

Motivation

PR #1791 and PR #1903 each implement a NIXL backend that stores descriptor data in a key-value store. Despite targeting different vendor libraries (hiredis vs. WDS KVCache), they independently arrived at identical implementations for:

• Key resolution policy (metaInfo if non-empty, else std::to_string(devId))
• Capability flags (supportsRemote/Local/Notif)
• No-op lifecycle stubs (connect, disconnect, unloadMD, loadLocalMD)
• Plugin entrypoint structure (nixlBackendPluginCreator<T>)
• Async transfer contract (postXfer dispatches → checkXfer polls → releaseReqH blocks)

Without a shared base, every future KV backend will re-copy this boilerplate and risk diverging silently over time.

---

What's in this PR

src/plugins/kv/kv_req_handle.h — nixlKVReqH + nixlKVXferCallback

Shared async request handle using an atomic pending counter, a first_error CAS field, and a condition variable. nixlKVXferCallback is a C-style completion callback compatible with both C++ async APIs and vendor C callback conventions (hiredis, wds_kvcache_put, etc.).

src/plugins/kv/kv_backend.h — nixlKVBackendBase : nixlBackendEngine

Abstract base class providing shared implementations of:

• registerMem / deregisterMem — metaInfo-or-devId key resolution, devIdToKey_ map management
• checkXfer / releaseReqH — poll/block on nixlKVReqH pending counter
• connect, disconnect, unloadMD, loadLocalMD — no-op stubs
• getSupportedMems() — defaults to {DRAM_SEG}; subclasses override to add OBJ_SEG etc.
• resolveKey() — two-step lookup (metadata pointer cast → devIdToKey_ fallback)
• nixlKVMetadata — shared inner metadata type (subclassable for backend-specific fields)

Leaves prepXfer, postXfer, queryMem as pure virtuals.

src/plugins/kv/kv_backend.cpp — concrete implementations of all shared methods.

src/plugins/kv/meson.build — builds nixl_kv_base as a static library and exposes nixl_kv_base_dep for plugin subdirectories to link against. Plugin subdir() stubs for redis/ and wqskv/ are present but commented out, ready to be activated as each PR migrates.

src/plugins/meson.build — adds subdir('kv') unconditionally (base infrastructure, not an optional plugin).

---

How each existing PR adapts

PR #1903 (WQSKV): mechanical — WQSKV already uses the atomic counter + CV pattern. Change the base class, remove duplicated fields/methods, replace nixlWqskvBackendReqH with nixlKVReqH and wqskv_xfer_cb with nixlKVXferCallback. All 23 helper unit tests are unaffected.

PR #1791 (REDIS): requires one behavioral change — postXfer currently uses std::promise/std::future per descriptor, which doesn't map onto nixlKVReqH. The hiredis completion callback should call nixlKVXferCallback instead of promise.set_value(status). checkXfer and releaseReqH are then deleted (inherited). The iRedisClient interface is retained for testability with updated signatures.

---

File layout (target state after both PRs adapt)

src/plugins/kv/
├── kv_backend.h                      ← nixlKVBackendBase
├── kv_backend.cpp                 ← nixlKVBackendBase
├── kv_req_handle.h                 ← nixlKVReqH + nixlKVXferCallback
├── meson.build
├── redis/                                   ← nixlRedisKVEngine : nixlKVBackendBase
└── wqskv/                               ← nixlWQSKVEngine : nixlKVBackendBase

Future KV backends follow the same pattern: add a subdirectory under src/plugins/kv/, subclass nixlKVBackendBase, implement prepXfer / postXfer / queryMem.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi hxieustc! Thank you for contributing to ai-dynamo/nixl.

Your PR reviewers will review your contribution then trigger the CI to test your changes.

🚀