Speed up Qwen3-VL-8B computer-use rollouts

[zhouhelena1]

Summary

Tunes the Qwen3-VL-8B GRPO recipe (rl/008_computer_use) to speed up rollouts from 17 mins to 13 mins on the same 8×H100 node while preserving the reward curve.

Recipe changes in slime_recipe/qwen3_vl_8b.py:

• tensor_model_parallel_size: 2 → 4 shards the model across more GPUs, which makes below safe to raise without OOMing during training.
• sglang_mem_fraction_static: 0.55 → 0.75 reserves a larger KV-cache fraction in GPU memory for more concurrent sequences, which speeds up rollout generation.

Tutorial _train() config:

-rollout_max_response_len=256,
+rollout_max_response_len=64,      # coordinate readouts are short; less to generate
-save_interval=10,
+save_interval=15,                 # only checkpoint at the end of the 15-rollout demo
+no_save_optim=True,               # skip optimizer state to cut checkpoint time

Checklist

• Example is documented with comments throughout, in a Literate Programming style.
• Example does not require third-party dependencies to be installed locally
• Example pins its dependencies
  • Example pins container images to a stable tag, not a dynamic tag like latest
  • Example specifies a python_version for the base image, if it is used
  • Example pins all dependencies to at least minor version, ~=x.y.z or ==x.y
  • Example dependencies with version < 1 are pinned to patch version, ==0.y.z

---

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR that start with 'DevinAI' or '@devin'.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

Devin Review found 2 potential issues.

[devin-ai-integration]

🔍 save_interval=15 with num_rollout=15 relies on slime saving at the final rollout

The tutorial changes save_interval from 10 to 15 while num_rollout stays 15, and _eval_trained() immediately does list_checkpoints(train_result.training_run_id)[-1] (tutorials/tutorial_generator/rl/008_computer_use.py:444). If slime's checkpoint gate (should_run_periodic_action(rollout_id, args.save_interval, num_rollout_per_epoch, args.num_rollout), referenced in modal_training_gym/frameworks/slime/modal_helpers/patches/patch_rollout_status_reporting.py:369) uses 0-based rollout_id and only fires when (rollout_id+1) % save_interval == 0 (or on the final rollout), then rollout_id 0..14 yields a save at the last step and this is fine. The extra args.num_rollout argument to that helper strongly suggests it also forces a final-step save. This should be safe, but if the periodic-action semantics differ (e.g. no final-step save), no checkpoint would be written and the eval step would raise IndexError.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

Devin Review found 1 new potential issue.

[devin-ai-integration]

🔍 TP=4 change assumed compatible with 8-GPU colocated topology

The bump tensor_model_parallel_size: 2 → 4 at modal_training_gym/train_recipes/slime_recipe/qwen3_vl_8b.py:17 is validated by validate_megatron_actor_parallelism / resolve_gpu_allocation. With 8 GPUs per node and colocate=True, TP=4 divides evenly so the allocation check should pass. No correctness issue was found, but the PR does not include validation output confirming the VL AutoBridge checkpoint load (comment updated at lines 45-46 to "the configured TP") actually works at TP=4 rather than the previously-tested TP=2; reviewer may want to confirm this was validated on hardware.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[zhouhelena1]

Can confirm this works

[github-actions]

Model Validation Results

Model	Status	Duration	Steps	Run
Qwen3-VL-8B-Instruct	✅ completed	658.0s	1	unsorted-snipe-cd3a78dc04b9

[github-actions]

Documentation preview: https://modal-labs-training-gym--training-gym-previews-preview-r-78da35.modal.run/259/docs

[devin-ai-integration]

Devin Review found 0 new potential issues.

[joyliu-q]

I found a prior deployment that runs faster thaan the speedup you made: #254 (comment)

Model	Status	Duration	Steps	Run
Qwen3-VL-8B-Instruct	✅ completed	360.0s	1	every-union-9bdfbfa2ff07

compared to

Model	Status	Duration	Steps	Run
Qwen3-VL-8B-Instruct	✅ completed	658.0s	1	unsorted-snipe-cd3a78dc04b9