Skip to content

Add multiprocess sim environment#47

Open
kyhavlov wants to merge 7 commits into
vladfi1:jaxfrom
kyhavlov:sim-multiprocess-env
Open

Add multiprocess sim environment#47
kyhavlov wants to merge 7 commits into
vladfi1:jaxfrom
kyhavlov:sim-multiprocess-env

Conversation

@kyhavlov

Copy link
Copy Markdown
Contributor

Adds a multiprocessing env wrapper for the JAX sim rollout worker to use. The parent process keeps policy inference and trajectory assembly, and worker processes each own a shard of SimBatchedEnvironment and write observations/resets into shared memory. Uses the same num_envs / inner_batch_size = worker count logic as the dolphin multiprocessing env code, and is enabled by async_envs=True. Doesn't support num_env_steps for now.

benchmarking in run_evaluator.py:

Settings:
4096 envs, rollout_length=3000, inner_batch_size=256

Non-MP:
timings: {'env_push': '27.776', 'agent_step': '5.610', 'trajectory_build': '7.411'}
env_fps: 97938.49, player_fps: 195876.99, sps: 23.91

MP 16 x 256:
timings: {'env_push': '4.066', 'agent_step': '5.470', 'trajectory_build': '12.154'}
env_fps: 179583.63, player_fps: 359167.25, sps: 43.84

Also verified a 100-step sim RL test run using the multiprocess env still trains - the trained checkpoint beat the base checkpoint 92-8 over 100 sim games.

@kyhavlov
kyhavlov force-pushed the sim-multiprocess-env branch from 5dbda60 to fa27984 Compare May 25, 2026 18:22
@kyhavlov
kyhavlov force-pushed the sim-multiprocess-env branch from fa27984 to 2aa4c2c Compare May 26, 2026 18:34
@kyhavlov

kyhavlov commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

Also worth noting: this now writes sim observations straight into the rollout trajectory buffer. GameBatchBuffers.time_major(...) owns the full T+1 observation tree, and each timestep has a GameBatchBuffers view that the env can fill directly. That removes the old extra copy from “current env observation” into the trajectory buffer.

same benchmark timings after removing the extra copy:

16 workers x 256 envs:

timings: {'env_pop': '0.000', 'env_push': '3.886', 'agent_step': {1: '1.808',
2: '1.808'}, 'trajectory_build': '8.091'}
env_fps: 247455.60, player_fps: 494911.19, sps: 60.41

@kyhavlov
kyhavlov force-pushed the sim-multiprocess-env branch from c7e0c23 to 63bfbce Compare May 26, 2026 21:19
Comment thread slippi_ai/sim_env/env.py
def current_game_batch(self, needs_reset: np.ndarray | None = None) -> GameBatch:
"""Return a [port1 views, port2 views] game batch for policy calls."""
@property
def game_batch_buffers(self) -> GameBatchBuffers:

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to be unused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants