[HiCache][Observability] Add fine-grained L1<->L2 KV transfer metrics (events / blocks / bytes / duration / bandwidth)#26974

Open

hxieustc wants to merge 28 commits into

sgl-project:mainfrom

hxieustc:add-data-xfer-fine-grain-statistics

Commits on Jun 26, 2026

add fine-grain L1-L2 data xfer statistics
hxieustc
committed
add fine-grain L1-L2 data xfer metrics to /metrics
hxieustc
committed
add fine-grain L1-L2 data xfer statistics for hi_mamba_radix_cache
hxieustc
committed
add fine-grain L1-L2 data xfer statistics for hybrid_cache
hxieustc
committed
add fine-grain L1-L2 data xfer statistics for unified_radix_cache
hxieustc
committed
show fine-grain transfer statistics in debug mode
hxieustc
committed
add fine-grain xfer statistics to disagg
hxieustc
committed
Fix HiCacheAck test consumers
hxieustc
committed
Skip HiCache transfer timing when disabled
hxieustc
committed
Record blocking HiCache write-back transfers
hxieustc
committed
Remove redundant HiCache transfer counters
hxieustc
committed
Fix HiCache transfer metric registration
hxieustc
committed
Tighten HiCache transfer metric accounting
hxieustc
committed
Fix HiCache write-through ack state handling
hxieustc
committed
Keep HiCache transfer totals with metrics
hxieustc
committed
fix(hicache): remove duplicate write_through_pending_id init in UnifiedTreeNode
hxieustc
committed
fix(hicache/metrics): wire enable_metrics to HybridCacheController and disagg offload path
hxieustc
committed
fix(hicache/metrics): fix byte estimation for hybrid sidecar pool transfers
hxieustc
committed
fix(hicache/mamba): add enable_metrics and extra_metric_labels params to attach_hybrid_pool_to_mamba_cache
hxieustc
committed
fix(hicache/dsa): propagate enable_metrics and extra_metric_labels to DSA attach path
hxieustc
committed
fix(hicache/metrics): pass resolved_pool_transfers to _make_hicache_ack in hybrid write/load paths
hxieustc
committed
fix(hicache/mamba): add enable_metrics and extra_metric_labels params to attach_hybrid_pool_to_mamba_cache
hxieustc
committed
fix(hicache/metrics): make L1/L2 transfer metrics emit real data on CUDA

ishandhanani
authored and
hxieustc
committed
test(hicache/metrics): add CPU unit tests for L1/L2 transfer metrics
hxieustc
committed
test(hicache/metrics): add Prometheus exposition tests
hxieustc
committed
test(hicache/metrics): fix registration and minor test adjustments
hxieustc
committed
test(hicache/metrics): trim to 9 essential tests
hxieustc
committed
fix: apply black formatting fixes
hxieustc
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[HiCache][Observability] Add fine-grained L1<->L2 KV transfer metrics (events / blocks / bytes / duration / bandwidth)#26974

[HiCache][Observability] Add fine-grained L1<->L2 KV transfer metrics (events / blocks / bytes / duration / bandwidth)#26974
hxieustc wants to merge 28 commits into
sgl-project:mainfrom
hxieustc:add-data-xfer-fine-grain-statistics

Commits on Jun 26, 2026

add fine-grain L1-L2 data xfer statistics

add fine-grain L1-L2 data xfer metrics to /metrics

add fine-grain L1-L2 data xfer statistics for hi_mamba_radix_cache

add fine-grain L1-L2 data xfer statistics for hybrid_cache

add fine-grain L1-L2 data xfer statistics for unified_radix_cache

show fine-grain transfer statistics in debug mode

add fine-grain xfer statistics to disagg

Fix HiCacheAck test consumers

Skip HiCache transfer timing when disabled

Record blocking HiCache write-back transfers

Remove redundant HiCache transfer counters

Fix HiCache transfer metric registration

Tighten HiCache transfer metric accounting

Fix HiCache write-through ack state handling

Keep HiCache transfer totals with metrics

fix(hicache): remove duplicate write_through_pending_id init in UnifiedTreeNode

fix(hicache/metrics): wire enable_metrics to HybridCacheController and disagg offload path

fix(hicache/metrics): fix byte estimation for hybrid sidecar pool transfers

fix(hicache/mamba): add enable_metrics and extra_metric_labels params to attach_hybrid_pool_to_mamba_cache

fix(hicache/dsa): propagate enable_metrics and extra_metric_labels to DSA attach path

fix(hicache/metrics): pass resolved_pool_transfers to _make_hicache_ack in hybrid write/load paths

fix(hicache/mamba): add enable_metrics and extra_metric_labels params to attach_hybrid_pool_to_mamba_cache

fix(hicache/metrics): make L1/L2 transfer metrics emit real data on CUDA

test(hicache/metrics): add CPU unit tests for L1/L2 transfer metrics

test(hicache/metrics): add Prometheus exposition tests

test(hicache/metrics): fix registration and minor test adjustments

test(hicache/metrics): trim to 9 essential tests

fix: apply black formatting fixes