refactor(daser): remove dead code, merge duplication, split large files#83
Merged
Conversation
- Delete unused _run_bg_loop and _bg_loop fallback in worker submit helpers - Delete no-op _clear_save_state and its two call sites - Delete write-only _ipc_async alias in DaserConnector - Delete unused IPCClientSync.match_and_alloc and IPCClientSync.init_transfer - Fix stale SHA256 docstring to xxh3_128 on alloc_chunk
- Delete unused _copy_pinned_to_dst from iouring layer - Drop unused nbytes parameter from _find_l1_locked and update call sites - Delete unused TransferMode enum and its export - Delete test-only NativeIOUring.read; rewrite test to use read_into - Delete unused PinnedMemoryBuffer.from_bytes and to_bytes
- Delete unused Gauge.dec from metrics - Delete test-only ModelGeometry.slot_size property; update config tests - Delete unused HTTPServerConfig task_separator/answer_separator fields - Delete pad_to_chunk_boundary alias; callers use pad_to_block_boundary - Delete unused VLLMClient.completion and its tests
- Correct chunk_key hash docstrings from SHA256 to xxh3_128 - Fix garbled MetadataStore.save docstring and use asdict for SlotEntry - Describe both retrieval and position implementations in base ABCs
- Raise ValueError when rotary_dim exceeds head_dim in non-table wrappers - Align with table-based functions that already raise on this condition - Previously the wrappers silently skipped rotation, masking misconfig - Add CPU-runnable contract tests for the shape guard and no-op paths
- Add coalesce_store_spans class attr, drain default no-op, stats and l1_bytes_used properties, and default grouped store/load implementations - Rename iouring backend self.stats to _stats backing the ABC stats property - Replace server getattr probing and private _l1_used access with ABC methods - Subclass TransferLayer in ipc server test doubles
- Extract _IPCClientBase holding socket path and retry/error contract - Add shared _raise_on_error helper and _MAX_ATTEMPTS retry constant - Both clients subclass the base; transport-specific call stays separate - Fix stale SHA256 docstrings to xxh3_128 on commit_chunk/evict_chunk
- Collapse three rope kernel cache getters into shared _compile_cached - Extract _validate_kv_and_tables for table rope entrypoints - Merge duplicate TileLang shape cache-key type aliases - Add _render_scalar shared by Counter.render and Gauge.render - Hoist insert/remove into RetrievalIndex base with _on_insert/_on_remove - Add cache-reuse mode constants in config and use across server/connector
… init - Add ServerCore._slots_for replacing repeated ceil(token_count/block_tokens) - Add _meta_matches shared by is_chunk_reusable and _has_store_owner - Delegate alloc_chunk to alloc_chunks to remove duplicated per-chunk body - Add VLLMClient._ensure_client replacing four inline lazy-init blocks
- Introduce _SaveFuture dataclass with release() replacing bare 3-tuples - Remove dead isinstance/len defensive branches in save-future reaping - Extract _init_server_transfer shared by both KV-cache register methods
- Move slice/cuda-ptr/pinned-copy/grouped-copy helpers to copy_ops module - Collapse _copy_src_to_pinned and _copy_src_to_pinned_at onto one helper - Keep thin layer method wrappers so lock semantics and test subclasses hold - copy_ops holds no tiering state and takes no locks
- Extract each op branch into a small _op_* coroutine - Build an op->handler dict and dispatch through it - Keep shared timing/metrics wrapper and derive status from the response
- Move committed/write-owner/evicted sets and commit-waiter futures into a ChunkLifecycle collaborator that keeps the transitions consistent - Replace scattered set mutations with mark_committed/mark_evicted/discard - Delegate wait_for_committed_chunks and is_chunk_reusable to the lifecycle - Drop now-unused asyncio import from core
- Compute block_dim once and use a single copy path for block-major and kv-major KV tensors in copy_staging_to_kv_cache - movedim(0, block_dim) unifies the slot-major to block-axis alignment
- Move O_DIRECT fd, io_uring ring pool, executor, and round-robin into L2IoEngine; the layer composes it as self._l2 (None when skip_l2) - Layer keeps thin _read_l2_into/_write_l2/_next_uring wrappers so the write-back glue and test subclass seams are preserved - Drop now-unused os/threading/concurrent.futures imports from the layer
- Move range-keyed pinned-host LRU cache into L1Cache: bisect index, LRU policy, pool, reserve/evict/drop/preserve, pool-waiter wakeups - Inject a pinned_predicate so L1Cache can skip closing slices still held by an in-flight L2 write, keeping L2 state out of the cache - Orchestrator composes L1Cache + optional L2IoEngine; skip_l2 collapses to 'self._l2 is None', removing scattered _skip_l2 and _fd guards - layer.py 1243 -> 715 lines; behavior preserved (32 transfer tests pass)
- Use a minimal _FakeTensor exposing only .shape instead of torch.zeros - The rotary_dim boundary guard runs before any tensor op, so the contract is verified without a real torch runtime (CI installs a torch stub) - Fixes unit-tests CI failure (module 'torch' has no attribute 'zeros')
- Update TransferLayer ABC to show formalized capability surface - Describe iouring layer as L1Cache + optional L2IoEngine + copy_ops - Add ChunkLifecycle and L1Cache/L2IoEngine to the component table - Show RetrievalIndex base insert/remove with _on_insert/_on_remove hooks - Fix stale IPCClientSync/Async op lists (lookup, commit_chunks)
- Add BenchmarkDataset name/add_cli_args/from_args hooks + register_dataset - Add DATASET_REGISTRY with add_dataset_cli_args and build_dataset helpers - run_bench.py and bench_load.py dispatch through the registry; the duplicated --dataset/--imdb/--longbench-dir/--datasets decls and the _load_samples if/else are gone - Adding a dataset now needs one registered subclass, zero runner edits - Document the pattern in benchmarks/README.md; add registry dispatch tests
- Delete unused count_prompt_tokens (count_prompt_payload_tokens is used) - Delete unused COMPARISON_GDS and BENCHMARK_SEED constants
- Move the inline vLLM bench path out of run_bench.py into utils/vllm_bench.py: prepare_config, run_load, command building, result normalization, correctness - run_bench.py now only forks on load generator and delegates; run_command and print_kv are injected so the module stays decoupled from the orchestrator - Replace hand-rolled _manifest_payload with dataclasses.asdict(manifest) - Delete dead _wait_with_message; fold _validate_bench_request_rate into vllm_bench.validate_args - run_bench.py 1241 -> 836 lines; update tests to the new module surface
- Drop _vllm_prefix_caching_enabled (set from vllm_config, never read) and the cache_config read that fed it; remove stale test setup lines - Drop _cross_layers_attn_backend (assigned in register, never read) - Remove unused ChunkLifecycle.is_write_owner and L1Cache/L2IoEngine capacity_bytes properties (and the L2 _l2_bytes attr they backed)
- _wait_daser_drained took settle_seconds only to del it; drop the param - run_daser_prefix forwarded settle_seconds solely to that dead param; remove it (the daser /drain wait is synchronous, no settle delay applies) - lmcache path keeps settle_seconds where _wait_lmcache_quiescent uses it
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cross-layer cleanup of
daser/: remove dead code, merge duplicated logic, formalize the transfer abstraction, and split the largest files into cohesive units. Behavior-preserving throughout — no functional changes intended.Net: 16 commits, ~-1.2k/+1.4k lines. The default unit/component suite stays green (328 passed) and the real-CUDA vLLM end-to-end path was validated after the high-risk changes.
What changed, by area
Dead code removed
_run_bg_loop,_bg_loop/_ipc_asyncaliases, no-op_clear_save_state, syncmatch_and_alloc/init_transfer_copy_pinned_to_dst,TransferModeenum, test-onlyNativeIOUring.read,PinnedMemoryBuffer.from_bytes/to_bytes, unused_find_l1_lockedparamGauge.dec,ModelGeometry.slot_sizeproperty,HTTPServerConfig.task_separator/answer_separator,pad_to_chunk_boundaryalias, unusedVLLMClient.completionAbstraction formalized
TransferLayerABC now declarescoalesce_store_spans,drain,stats,l1_bytes_used, and default grouped store/load. The IPC server no longer probes viagetattror reaches into the private_l1_used— it codes against the interface.Duplication merged
_compile_cached; shared_validate_kv_and_tablesCounter.render/Gauge.render→_render_scalarRetrievalIndexbase owns insert/remove with_on_insert/_on_removehooksServerCore._slots_for/_meta_matches;alloc_chunkdelegates toalloc_chunksVLLMClient._ensure_client;SaveFuturedataclass;_init_server_transferLarge files split (composition, not inheritance)
ServerCore→ extractedChunkLifecycle(commit/ownership/eviction state + commit waiters)_dispatch16-branch if-ladder → op→handler tableiouring/layer.py1243 → 715 lines, split intocopy_ops(stateless marshalling),l1_cache.L1Cache(range-keyed pinned LRU),l2_engine.L2IoEngine(io_uring positioned I/O). L1 and L2 stay decoupled via an injectedpinned_predicate; the write-back glue stays in the orchestrator.skip_l2collapses toself._l2 is None.One real fix
rope_apply:rotary_dim > head_dimnow raises instead of silently skipping rotation (which would corrupt KV). Added CPU-runnable contract tests.Testing
pytest -m "not integration and not slow": 328 passedtest_vllm_e2e: PASS (real CUDA, end-to-end store/load/promote/RoPE-reuse) — run after the ABC/lifecycle/dispatch changes and again after the iouring splitNotes
test_skip_save_blocks_chunk_allocfails on this machine due to a cuFile/GDS COMPAT-mode environment issue; it fails identically on cleanmaster, so it is not a regression from this PR.raysymlink in the venv + worktree module-path duplication); not introduced by this PR.Deferred follow-ups
ServerCore
DocumentService/TransferGatewayextraction, workerLoadStats, schedulerPendingState— lower-leverage or higher hot-path risk; left for a future PR.