Skip to content
This repository was archived by the owner on Apr 20, 2026. It is now read-only.

feat: add mocker backend type for GPU-less router testing#166

Open
PeaBrane wants to merge 3 commits into
ishandhanani:mainfrom
PeaBrane:rupei/add-mocker-backend
Open

feat: add mocker backend type for GPU-less router testing#166
PeaBrane wants to merge 3 commits into
ishandhanani:mainfrom
PeaBrane:rupei/add-mocker-backend

Conversation

@PeaBrane

@PeaBrane PeaBrane commented Feb 10, 2026

Copy link
Copy Markdown

Adds a lightweight mocker backend that launches dynamo.mocker engines for testing router logic without requiring GPUs. Useful for priority queue benchmarking and CI environments.

Summary by CodeRabbit

  • New Features
    • Introduced a new Mocker backend option for srtctl deployments and advanced configuration.
    • Supports comprehensive mode-specific settings and environment variable configuration for prefill, decode, and aggregated processing modes.
    • Mocker backend is now fully integrated with srtctl's configuration schema and available as a deployment backend option.
    • Enables flexible management of worker processes and deployment endpoints through customizable Mocker configurations.
    • Extended backend selection to include Mocker alongside existing supported backend types.

Adds a lightweight mocker backend that launches `dynamo.mocker` engines
for testing router logic without requiring GPUs. Useful for priority
queue benchmarking and CI environments.
@coderabbitai

coderabbitai Bot commented Feb 10, 2026

Copy link
Copy Markdown
Contributor

Warning

Rate limit exceeded

@PeaBrane has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 20 minutes and 30 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📝 Walkthrough

Walkthrough

This PR introduces a new Mocker backend for srtctl that simulates GPU workers without real hardware. It adds MockerProtocol and MockerServerConfig classes with configuration, environment, and process management capabilities, integrating them into the existing backend system's serialization and deserialization pipeline.

Changes

Cohort / File(s) Summary
Mocker Backend Implementation
src/srtctl/backends/mocker.py
Introduces MockerServerConfig dataclass for per-mode CLI configs and MockerProtocol implementing BackendProtocol. Includes methods for srun configuration, mode-specific config/environment selection, endpoint allocation, process conversion, and worker command building. Helper function _config_to_cli_args serializes nested dicts to CLI flags.
Backend System Integration
src/srtctl/backends/__init__.py, src/srtctl/core/schema.py
Exposes Mocker backend by importing and exporting MockerProtocol and MockerServerConfig. Updates BackendConfig union and all exports. Extends schema deserialization/serialization to recognize "mocker" backend type.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

A mocker hops in, all fluffy and spry,
Spinning up workers without GPU's sigh,
Config and endpoints now dance to its tune,
Simulating the server beneath the moon! 🐰✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: introduction of a mocker backend for GPU-less router testing, which is the primary focus of the entire changeset.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@src/srtctl/backends/mocker.py`:
- Around line 1-32: Replace the locally redefined WorkerMode literal with the
canonical alias by adding "WorkerMode" to the guarded TYPE_CHECKING imports from
srtctl.core.topology and remove the local "WorkerMode = Literal[...]"
definition; specifically, update the TYPE_CHECKING block to import WorkerMode
and delete the WorkerMode symbol defined at the bottom of the file so all
references use the centralized type from srtctl.core.topology.
🧹 Nitpick comments (3)
src/srtctl/backends/mocker.py (3)

88-109: Consider using match statements for mode dispatch.

Per coding guidelines (Python 3.10+ modern syntax including match statements), both get_config_for_mode and get_environment_for_mode are natural candidates for match/case instead of if/elif chains. This is a purely stylistic suggestion.

♻️ Example refactor for get_config_for_mode
     def get_config_for_mode(self, mode: WorkerMode) -> dict[str, Any]:
         """Get config dict for a worker mode."""
         if not self.mocker_config:
             return {}
 
-        if mode == "prefill":
-            return dict(self.mocker_config.prefill or {})
-        elif mode == "decode":
-            return dict(self.mocker_config.decode or {})
-        elif mode == "agg":
-            return dict(self.mocker_config.aggregated or {})
-        return {}
+        match mode:
+            case "prefill":
+                return dict(self.mocker_config.prefill or {})
+            case "decode":
+                return dict(self.mocker_config.decode or {})
+            case "agg":
+                return dict(self.mocker_config.aggregated or {})
+            case _:
+                return {}

As per coding guidelines: "Use Python 3.10+ modern syntax including | unions and match statements"


115-123: Priority order in get_served_model_name may surprise users.

The iteration order is aggregated → prefill → decode, which means in disaggregated mode (where aggregated is likely None), the prefill config's model path takes precedence over decode. This is probably fine since they'd typically be the same model, but worth a brief inline comment documenting the precedence for future maintainers.


160-184: build_worker_command ignores several parameters silently.

Parameters endpoint_processes, runtime, frontend_type, profiling_enabled, nsys_prefix, and dump_config_path are all unused. This is understandable for a mock backend, but a brief inline comment (e.g., # Mocker ignores profiling/frontend/runtime — kept for BackendProtocol compatibility) would prevent future contributors from wondering if this is an oversight.

Comment thread src/srtctl/backends/mocker.py Outdated
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant