feat(replay): support MPS gpu-resident replay

[TATP-233]

Summary

• Allow the single-GPU GPUResidentReplayPipeline to run on Apple MPS while keeping the packed CPU replay authoritative.
• Keep CUDA on its existing daemon-thread/side-stream path; submit MPS mirror copies and device gathers from the learner thread because PyTorch exposes no public MPS stream API and background Metal submission is unsafe.
• Add MPS event-based completion, pre-allocation memory-budget diagnostics, ring/snapshot/seed coverage, config guidance, and bilingual SAC documentation.
• This is opt-in through training.replay_pipeline=gpu_resident; the default remains cpu_pinned_double_buffer. It adds one full replay allocation on the learner device.

Linked Work

• Issue: Closes Work: 为 macOS/MPS 增加 gpu_resident replay pipeline 支持 #894
• Milestone: none

Validation

• make check (covered by make test-all)
• uv run pytest -m "not slow" (covered by the complete make test-all gate)
• Additional task-specific validation listed below

Commands actually run:

uv run pytest tests/ipc/test_replay_pipeline_gpu_resident.py -q

make test-all

cd docs/sphinx
UNILAB_DOCS_SKIP_AUTODOC=1 uv run --no-project --with-requirements requirements.txt \
  sphinx-build -b html -n source build/html

uv run train --algo sac --task g1_walk_flat --sim mujoco --render-mode none \
  training.device=mps \
  training.replay_pipeline=gpu_resident \
  training.no_play=true \
  training.export_onnx=false \
  training.log_dir=/tmp/unilab-issue894-gpu-resident-wrap-smoke \
  algo.num_envs=32 \
  algo.batch_size=32 \
  algo.replay_buffer_n=16 \
  algo.learning_starts=1 \
  algo.updates_per_step=2 \
  algo.max_iterations=25 \
  algo.save_interval=0 \
  algo.use_symmetry=false \
  algo.algo_params.use_compile=false

uv run benchmark/rl/benchmark_replay_buffer_placement.py \
  --algos sac --tasks g1_walk_flat --sim mujoco \
  --device mps --warmup 10 --repeat 100 \
  --incremental-source pageable --sampled-batch-memory pageable \
  --out-json /tmp/unilab-issue894-replay-placement-mps.json \
  --plot-dir /tmp/unilab-issue894-replay-placement-mps-plots

uv run benchmark/rl/benchmark_replay_buffer_placement.py \
  --algos sac --tasks g1_walk_flat --sim mujoco \
  --device cpu --warmup 10 --repeat 100 \
  --incremental-source pageable --sampled-batch-memory pageable \
  --out-json /tmp/unilab-issue894-replay-placement-cpu.json \
  --plot-dir /tmp/unilab-issue894-replay-placement-cpu-plots

Results:

• Targeted replay test: 17 passed, 13 skipped.
• Complete gate: 1630 passed, 59 skipped, 268 deselected, 1 xfailed; Ruff, mypy, Pyright, and benchmark smoke passed.
• Strict Sphinx build succeeded. It reported one pre-existing unrelated warning for en/3-deployment/2-sim_to_sim/7-config_guard.md.
• Real MPS training completed 25/25 iterations. Replay capacity was 512 rows and total env steps reached 864, so the runner exercised ring wrap without collector, snapshot, or learner failures.

Apple Silicon replay benchmark

Hardware and parameters:

• Apple M5 Max, 40-core GPU, 128 GB unified memory
• macOS 26.5.2, PyTorch 2.7.0
• SAC g1_walk_flat / MuJoCo default replay shape
• Capacity: 1,048,576 rows; full replay: 1.68 GiB
• Sample count: 32,768 rows; sampled batch: 53.75 MiB
• Increment: 2,048 rows; incremental copy: 3.36 MiB
• 10 warmups, 100 measured repetitions; values below are medians

Operation	Median
MPS resident device gather	0.864 ms
Incremental CPU to MPS mirror copy	0.218 ms
Resident scheme total	1.082 ms/tick
CPU pre-sample	1.876 ms
CPU to MPS sampled-batch transfer	0.783 ms
Existing staged scheme total	2.659 ms/tick

The isolated replay work was about 2.46x faster for the resident scheme in this run. This is benchmark evidence for the replay path only, not an end-to-end training-speed or convergence claim.

Impact

• Backend impact: none; replay behavior remains backend-isolated and the smoke used MuJoCo
• Platform impact: macOS MPS support added; existing CUDA path retained
• Training effect expected: yes, only when training.replay_pipeline=gpu_resident is explicitly selected

Artifacts

• W&B: none
• benchmark result: raw JSON produced at /tmp/unilab-issue894-replay-placement-{mps,cpu}.json; parameters and medians are included above
• video / screenshot: none
• ONNX / checkpoint: none

Checklist

• Added or updated tests where needed
• Updated docs if behavior or workflow changed
• Linked the driving issue
• Follow-up scope is explicit: no MPS multi-GPU lane, public replay contract change, permanent benchmark gate, or end-to-end performance guarantee