chore(tests): align evict lmcache sizing#82
Merged
Conversation
- Route skip_l2 through TieredIOUringTransferLayer instead of a separate memory backend. - Disable SSD, io_uring, and L2 persistence resources inside iouring skip_l2 mode. - Migrate L1-only behavior tests and documentation to the iouring skip_l2 path.
- Stop expanding LMCache evict L1 sizing by the eviction watermark - Use the same derived L1 byte target for DaseR and LMCache benchmark runs - Update benchmark docs and unit tests for the equalized sizing policy
- Preserve non-overlapping L1 fragments during skip-l2 overwrites - Avoid partial-tail in-place writes past the cached L1 slice - Add an LMCache quiescence timeout to prevent benchmark hangs - Cover skip-l2 overwrite and LMCache wait edge cases
- Add the LMCache checkout ahead of DaseR in LMCache benchmark subprocess PYTHONPATH - Pass kv_connector_module_path for the LMCache MP connector - Cover LMCache benchmark connector and import path selection
- Add a benchmark shim connector name that avoids vLLM's built-in registry entry - Route the shim to the sibling LMCache checkout connector - Cover the non-conflicting connector name in benchmark tests
GentleCold
marked this pull request as ready for review
June 13, 2026 18:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
skip_l2into the iouring transfer path and remove the separate memory-only transfer layer.pinned_pool.pyin its current iouring package location.Why
The transfer stack should not maintain two independent mem-to-GPU paths. In no-evict runs,
skip_l2now means the iouring backend keeps the L1 path and skips SSD setup. In evict comparisons, LMCache and DaseR now receive matched L1 sizing so benchmark results are easier to interpret.Validation
pytest tests/transfer/test_iouring_skip_l2_transfer.py tests/transfer/test_tiered_iouring_transfer.py tests/server/test_ipc_server.py tests/transfer/test_cpu_imports.py -qpytest tests/server/test_main_cli.py tests/test_config.py tests/connector/test_daser_connector.py::test_worker_transfer_ready_allows_skip_l2_without_store_path tests/connector/test_daser_connector.py::test_runtime_config_ready_allows_skip_l2_without_store_path -qpytest tests/transfer tests/server tests/test_config.py tests/connector/test_daser_connector.py -qpytest tests/unit/test_benchmark_utils.py tests/unit/test_benchmark_unified_utils.py -qlmcache_l1_gb=112for derived L1111.09 GiB:/data/zwt/daser_bench/vllmbench_10000x1_correctness/run_20260614_002723Benchmark Notes
1472.55 ms, backend hit rate:0.85642872.59 ms, backend hit rate:1.07973.94 ms, DaseR8095.25 ms