v0.23.4

Highlights

• Fixed a production transfer-lock leak on multi-node clusters: RDMA peer sessions were keyed by the remote QPN, which is only unique within one remote HCA. Peers booted from the same image collide, and a collision let one peer's handshake silently overwrite — and one peer's invalidation destroy — another peer's sessions, wedging the connection permanently and leaking transfer locks (blocks pinned for the full 120 s timeout) on every subsequent fetch. Connections now own their per-NIC sessions and remote-memory snapshots directly, making the collision unrepresentable (#399).
• P/D serving: the router now forwards the first token from P, and P saves partial tail blocks, improving prefill/decode handoff (#395).

Reliability and transfer behavior (#399)

• Requester-side transfer-lock release is now RAII-guarded: the session is released on every exit path — completion, handled error, panic unwind, or a dropped future — with a WARN identifying any abnormal path. Previously a panic in the fetch task silently leaked the holder-side lock until GC.
• Session worker threads no longer leak: workers held a strong reference cycle that kept the thread, QP, and CQ alive forever after a connection was invalidated or a handshake aborted. Workers now hold a weak reference and exit once the session is dropped; commands accepted before an invalidation now fail with an explicit "session invalidated before batch execution" error instead of a generic channel-closed.
• RDMA fetch staging chunks are now sized min(remaining bytes on that NUMA, 256 MiB). The previous fixed 256 MiB floor made small fetches fail outright on pinned pools smaller than the cap; the cap on large fetches (reclaim-amplification bound from the previous release) is preserved.
• The accept side of the RDMA handshake now aborts cleanly on completion failure instead of leaving the client permanently stuck in "connecting".
• Validated end to end with the p2p_rdma integration test on an H200 node with 400G IB: it fails on the previous release (chunk allocation error) and passes on this one, with all session workers exiting cleanly on teardown.

Breaking metric change

• pegaflow_rdma_qps now reports the actual number of active RC queue pairs, matching its description. It previously counted connection entries (peers × NICs), so dashboards will show a step up by a factor of qps_per_peer.

P/D and connector

• NIXL pega-pull control-plane polling reduced (#388).

Full Changelog: v0.23.3...v0.23.4

v0.23.3

Highlights

• Added PegaNixlPullConnector, which keeps the vLLM NIXL control plane while using PegaFlow RDMA v1 for pull-mode KV reads. The H200 P/D benchmark completed 64/64 requests at 9.04 req/s with 232.21 ms mean TTFT.
• Added a MetaServer flush barrier and an embeddable P2P transfer service for applications that host PegaEngine in-process. The validated H200 P/D run discovered and transferred 33/33 blocks (74.2 MiB) over RDMA with token-identical greedy output.

Reliability and transfer behavior

• RDMA-prefetched blocks are now registered with MetaServer only when they remain resident after local cache admission.
• RDMA prefetch now uses block-local staging and preserves an already staged prefix when a later allocation fails. In the measured H200 pressure A/B, reclaim amplification fell from 2.56x to 2.07x (-19.5%) and request throughput changed from 3.16 to 3.49 req/s. The tradeoff was higher mean TTFT (4.52 s to 5.07 s), higher mean TPOT (30.0 ms to 45.3 ms), and lower mean logged RDMA throughput (28.9 GiB/s to 18.0 GiB/s).
• Hardened the RC backend with overlap-safe memory registration, predecessor-based MR lookup, shorter global lock scope, and error returns for inconsistent session state. Real-link A/B results on an 8x400G IB node stayed within 3% noise; this release does not claim a measured throughput win from that refactor.

Release build and tooling

• Release-profile A/B measurements improved the 32K query/prefetch lease benchmark from 3.7152 ms to 3.5507 ms (-4.43%) and the 61-layer 32K save-submit benchmark from 482.18 ms to 443.73 ms (-7.98%). Thin LTO and one codegen unit are now enabled for release artifacts. Clean build time increased from 27.05 s to 48.03 s (+77.6%), while the benchmark binary decreased from 12.03 MB to 8.60 MB (-28.5%).
• Updated local development, CI, and release builds to Rust 1.97.0.
• Added PEGAFLOW_IB_DEVICE support to the P2P RDMA integration test.

Full Changelog: v0.23.2...v0.23.3

v0.23.2

fix(core): register RDMA-fetched blocks to MetaServer after prefetch …

v0.23.1

chore: bump version to 0.23.1

Co-authored-by: Cursor <cursoragent@cursor.com>

v0.23.0

What's Changed in v0.23.0

✨ Features

• MLA layer-split KV cache — page-first per-shard slots (#372)
  Page-first storage now covers MLA layer-split, where each tensor-parallel rank registers a disjoint subset of layers. Registrations are grouped by layer-set into shards (total_slots = num_shards): a layer-split instance has one shard per rank (each rank writes its own sub-page), while full-replica keeps a single shard with block-striped writes. Pipeline parallelism and DCP remain excluded. Validated end-to-end on an 8-GPU H200 node with GLM-5.1-FP8 (byte-identical warm-hit, per-shard load with zero failures, gsm8k 5-shot 0.95, warm-vs-cold bench A/B of 1.73x throughput / -47% P99 TTFT).
• Vendored vLLM NIXL connector (#366)
  Imports the vLLM NIXL KV connector into the tree under pegaflow/nixl_connector/ (push/pull schedulers and workers, TP mapping, metadata, stats), so the connector version is owned by PegaFlow instead of pulled from an external import.

🐛 Fixes

• core: preserve per-slot NUMA on block rebuild to prevent 0-slot re-serve (#370)
• nixl: use vendored connector imports (#371)
• metaserver: size the removal pipeline for eviction bursts (#369)
• connector: include pp_size and mla_layer_split in the namespace so layouts differing only in those dimensions no longer collide (#367)

🧹 Chores

• downgrade tokio to 1.48.0 (#364)

⚠️ Compatibility

Strict version handshake — client and server must both be 0.23.0 (exact CARGO_PKG_VERSION match at registration). SSD cache is ephemeral and wiped on restart.

Full diff: v0.22.10...v0.23.0

v0.22.10

Release of the pegaflow workspace / pegaflow-llm 0.22.10 — 12 commits since v0.22.9, centered on MLA KV-cache storage efficiency, model-aware transfer-backend selection, and cross-node redundancy observability.

English

✨ Features

• MLA KV page-first storage (#360) — store MLA KV cache page-first so per-block metadata collapses, cutting metadata overhead for MLA models.
• Per-layer MLA TP save distribution (#359) — spread MLA tensor-parallel save work across ranks by layer to balance save load.
• Model-aware KV transfer backend (#357) — the connector auto-selects the KV transfer backend per model; the server no longer needs a static backend setting.
• Metaserver block-redundancy metrics (#361) — new pegaflow_metaserver_block_redundancy{owners="1|2|3|>=4"} distribution plus pegaflow_metaserver_block_redundancy_avg gauge, surfacing the cross-node KV replication factor (how much effective cache capacity shrinks).
• P/D handshake wire schema (#345) — seal the prefill/decode handshake wire schema in pegaflow-pd-wire.
• Transfer benchmarks (#349, eb69309) — p2p RDMA fetch example plus native D2H/H2D transfer-path measurement.

🐛 Fixes

• Drop late duplicate saves (#358) — skip late duplicate saves of already-resident blocks, avoiding redundant work.

♻️ Refactors

• Restructure pd_connector for maintainability (#355).
• SealedBlock owns its RawBlock slots (#352).
• Use usize for block_ids, validated at the RPC boundary (#351).

🔧 Chore

• Bump version 0.22.9 → 0.22.10 (#362).

⚠️ Strict version handshake: client and server must match on CARGO_PKG_VERSION at registration — upgrade both sides together.

中文

✨ 新功能

• MLA KV page-first 存储 (#360) — MLA KV cache 按 page-first 布局存储,合并每块元数据,降低 MLA 模型的元数据开销。
• MLA TP save 按层跨 rank 分摊 (#359) — 把 MLA 张量并行的 save 工作按层分散到各 rank,均衡 save 负载。
• 按模型自动选 KV 传输 backend (#357) — connector 按模型自动选择 KV 传输 backend,server 不再需要静态指定。
• Metaserver 块冗余度指标 (#361) — 新增 pegaflow_metaserver_block_redundancy{owners="1|2|3|>=4"} 分布与 pegaflow_metaserver_block_redundancy_avg,反映跨节点 KV 副本数(即有效缓存容量缩水倍数)。
• P/D 握手 wire schema (#345) — 在 pegaflow-pd-wire 中固化 prefill/decode 握手协议。
• 传输 benchmark (#349, eb69309) — p2p RDMA fetch 示例 + 原生 D2H/H2D 传输路径测量。

🐛 修复

• 丢弃迟到的重复 save (#358) — 跳过对已驻留块的迟到重复 save,避免冗余工作。

♻️ 重构

• 重构 pd_connector 提升可维护性 (#355)。
• SealedBlock 自持 RawBlock slot (#352)。
• block_ids 改用 usize,在 RPC 边界校验 (#351)。

🔧 杂项

• 版本 0.22.9 → 0.22.10 (#362)。

⚠️ 严格版本握手:注册时 client 与 server 必须 CARGO_PKG_VERSION 完全一致——升级请两端同时进行。

Full Changelog: v0.22.9...v0.22.10

v0.22.9

What's Changed

• refactor(core): replace KVCacheRegistration with validated KVCacheLayout by @xiaguan in #340
• feat(pd): support mtp split connector by @GentleCold in #341
• fix(core)!: seal instance layer topology from registered layers by @xiaguan in #346
• feat(connector): register layer-split KV caches from cache config by @feifei-111 in #343
• chore: bump version to 0.22.9 by @xiaguan in #347

Full Changelog: v0.22.8...v0.22.9

v0.22.8

chore: bump version to 0.22.8 (#337)

## Summary
- bump workspace and Python package versions to 0.22.8
- refresh Cargo.lock package versions for workspace crates

## Tests
- cargo metadata --locked --no-deps
- uv run --no-project python -c "import pathlib, tomllib;
p=tomllib.loads(pathlib.Path('python/pyproject.toml').read_text());
assert p['project']['version']=='0.22.8'; assert
p['tool']['commitizen']['version']=='0.22.8'; print('python metadata
version ok')"
- pre-commit hooks during commit, including cargo test --release

v0.22.7

chore(release): prepare 0.22.7

v0.22.6

0.22.5 -> 0.22.6 对照

0.22.6 是 P/D 分离的第一个 stable 版本：这版把 P/D RDMA push 路径的性能稳定性、正确性保护、布局兼容和发布包完整性一起补齐。0.22.5 是上一版 release baseline；0.22.6 是推荐用于 P/D 分离的正式版本。

维度	0.22.5	0.22.6
P/D 分离	基础发布状态，P/D 路径仍在收敛	第一版 stable：P/D RDMA push、scheduler/worker 协议、prefill/decode 流程、proxy 和 native RDMA binding 都经过稳定化
性能	RDMA push 和布局映射还缺少本轮优化	支持 MLA cache layout；优化 layout mapping；补齐 connector metrics；提升 RDMA push 发送、完成通知、等待路径和链路利用率观测
正确性	对版本不匹配、零命中 query probe、RDMA-only query path 等边界保护不足	vLLM/server 版本不匹配 fail early；零命中 query probe 不再错误释放 lease；RDMA-only query path 加 cfg-gate；vLLM 启动失败更早暴露
发包	CUDA 13 wheel 使用 --no-default-features --features cuda-13，会关掉默认 rdma	CUDA 13 CI/release wheel 显式使用 --features cuda-13,rdma；cu12 和 cu13 包都带 RDMA
其他修复	SSD cache 单路径	SSD cache 支持多路径；清理未使用 Rust 依赖；测试门禁和 e2e cargo feature 对齐

0.22.5 -> 0.22.6 Comparison

0.22.6 is the first stable P/D disaggregation release. It closes the loop on P/D RDMA push performance, correctness guards, cache-layout compatibility, and package completeness. 0.22.5 is the previous release baseline; 0.22.6 is the recommended release for stable P/D disaggregation.

Area	0.22.5	0.22.6
P/D disaggregation	Baseline release state, with the P/D path still converging	First stable release: P/D RDMA push, scheduler/worker protocol, prefill/decode flow, proxy, and native RDMA binding have been stabilized
Performance	RDMA push and layout mapping had not received this round of optimization	Adds MLA cache layout support; optimizes layout mapping; adds connector metrics; improves RDMA push sending, completion signaling, wait behavior, and link-utilization visibility
Correctness	Edge cases such as version mismatch, zero-hit query probes, and RDMA-only query paths were under-protected	Fails early on vLLM/server version mismatch; avoids incorrect lease release for zero-hit query probes; cfg-gates RDMA-only query behavior; fails faster when vLLM startup dies
Packaging	CUDA 13 wheels used --no-default-features --features cuda-13, which disabled default rdma	CUDA 13 CI/release wheels now explicitly use --features cuda-13,rdma; both cu12 and cu13 packages include RDMA
Other fixes	Single SSD cache path	Multiple SSD cache paths; unused Rust dependencies removed; test gates and e2e cargo features aligned

Artifacts

• GitHub Release includes 10 wheels: Python 3.10-3.14 for both pegaflow-llm and pegaflow-llm-cu13.
• PyPI published pegaflow-llm==0.22.6 and pegaflow-llm-cu13==0.22.6.