Add multiprocess sim environment#47
Open
kyhavlov wants to merge 7 commits into
Open
Conversation
kyhavlov
force-pushed
the
sim-multiprocess-env
branch
from
May 25, 2026 18:22
5dbda60 to
fa27984
Compare
kyhavlov
force-pushed
the
sim-multiprocess-env
branch
from
May 26, 2026 18:34
fa27984 to
2aa4c2c
Compare
Contributor
Author
|
Also worth noting: this now writes sim observations straight into the rollout trajectory buffer. GameBatchBuffers.time_major(...) owns the full T+1 observation tree, and each timestep has a GameBatchBuffers view that the env can fill directly. That removes the old extra copy from “current env observation” into the trajectory buffer. same benchmark timings after removing the extra copy: |
kyhavlov
force-pushed
the
sim-multiprocess-env
branch
from
May 26, 2026 21:19
c7e0c23 to
63bfbce
Compare
vladfi1
reviewed
May 28, 2026
| def current_game_batch(self, needs_reset: np.ndarray | None = None) -> GameBatch: | ||
| """Return a [port1 views, port2 views] game batch for policy calls.""" | ||
| @property | ||
| def game_batch_buffers(self) -> GameBatchBuffers: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a multiprocessing env wrapper for the JAX sim rollout worker to use. The parent process keeps policy inference and trajectory assembly, and worker processes each own a shard of SimBatchedEnvironment and write observations/resets into shared memory. Uses the same
num_envs / inner_batch_size = worker countlogic as the dolphin multiprocessing env code, and is enabled byasync_envs=True. Doesn't supportnum_env_stepsfor now.benchmarking in run_evaluator.py:
Also verified a 100-step sim RL test run using the multiprocess env still trains - the trained checkpoint beat the base checkpoint 92-8 over 100 sim games.