feat: add mocker backend type for GPU-less router testing#166
Conversation
Adds a lightweight mocker backend that launches `dynamo.mocker` engines for testing router logic without requiring GPUs. Useful for priority queue benchmarking and CI environments.
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📝 WalkthroughWalkthroughThis PR introduces a new Mocker backend for srtctl that simulates GPU workers without real hardware. It adds MockerProtocol and MockerServerConfig classes with configuration, environment, and process management capabilities, integrating them into the existing backend system's serialization and deserialization pipeline. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@src/srtctl/backends/mocker.py`:
- Around line 1-32: Replace the locally redefined WorkerMode literal with the
canonical alias by adding "WorkerMode" to the guarded TYPE_CHECKING imports from
srtctl.core.topology and remove the local "WorkerMode = Literal[...]"
definition; specifically, update the TYPE_CHECKING block to import WorkerMode
and delete the WorkerMode symbol defined at the bottom of the file so all
references use the centralized type from srtctl.core.topology.
🧹 Nitpick comments (3)
src/srtctl/backends/mocker.py (3)
88-109: Consider usingmatchstatements for mode dispatch.Per coding guidelines (Python 3.10+ modern syntax including
matchstatements), bothget_config_for_modeandget_environment_for_modeare natural candidates formatch/caseinstead ofif/elifchains. This is a purely stylistic suggestion.♻️ Example refactor for
get_config_for_modedef get_config_for_mode(self, mode: WorkerMode) -> dict[str, Any]: """Get config dict for a worker mode.""" if not self.mocker_config: return {} - if mode == "prefill": - return dict(self.mocker_config.prefill or {}) - elif mode == "decode": - return dict(self.mocker_config.decode or {}) - elif mode == "agg": - return dict(self.mocker_config.aggregated or {}) - return {} + match mode: + case "prefill": + return dict(self.mocker_config.prefill or {}) + case "decode": + return dict(self.mocker_config.decode or {}) + case "agg": + return dict(self.mocker_config.aggregated or {}) + case _: + return {}As per coding guidelines: "Use Python 3.10+ modern syntax including
|unions andmatchstatements"
115-123: Priority order inget_served_model_namemay surprise users.The iteration order is
aggregated → prefill → decode, which means in disaggregated mode (whereaggregatedis likelyNone), the prefill config's model path takes precedence over decode. This is probably fine since they'd typically be the same model, but worth a brief inline comment documenting the precedence for future maintainers.
160-184:build_worker_commandignores several parameters silently.Parameters
endpoint_processes,runtime,frontend_type,profiling_enabled,nsys_prefix, anddump_config_pathare all unused. This is understandable for a mock backend, but a brief inline comment (e.g.,# Mocker ignores profiling/frontend/runtime — kept for BackendProtocol compatibility) would prevent future contributors from wondering if this is an oversight.
Adds a lightweight mocker backend that launches
dynamo.mockerengines for testing router logic without requiring GPUs. Useful for priority queue benchmarking and CI environments.Summary by CodeRabbit