Latent Motion Adjuster (LMA)

LMA: Latent Motion Adjuster for Physics-based Multi-agent Interaction

• Paper: [TMLR]
• Website: [LMA Website]

Installation

We tested our code on Ubuntu 22.04.

1. Install Isaac Gym

2. Follow the instruction provides by Isaac Gym and create the conda environment:

   cd <isaacgym repository>
   create_conda_rlgpu.sh

3. Activate new conda environment:

   conda activate rlgpu

4. Clone SMPLOlympics repository in the third_party/SMPLOlympics:

   cd <LMA repository>
   git clone https://github.com/SMPLOlympics/SMPLOlympics.git third_party/SMPLOlympics

   or

   git submodule update --init --recursive

5. Create symbolic links to utilize the relevant code from SMPLOlympics:

   cd <LMA repository>
   ln -s third_party/SMPLOlympics/phc ./phc
   ln -s third_party/SMPLOlympics/poselib ./poselib
   

6. Install the required libraries for each repository:

   cd <LMA repository>/third_party/SMPLOlympics && pip install -r requirement.txt
   cd <isaacgym repository>/python && pip install -e .
   cd <LMA repository> && pip install -r requirement.txt

7. Download SMPL parameters from SMPL and SMPLX. Put them in the data/smpl folder, unzip them into 'data/smpl' folder. For SMPL, please download the v1.1.0 version, which contains the neutral humanoid. Rename the files basicmodel_neutral_lbs_10_207_0_v1.1.0, basicmodel_m_lbs_10_207_0_v1.1.0.pkl, basicmodel_f_lbs_10_207_0_v1.1.0.pkl to SMPL_NEUTRAL.pkl, SMPL_MALE.pkl and SMPL_FEMALE.pkl. For SMPLX, please download the v1.1 version. Rename The file structure should look like this:

   |-- data
       |-- smpl
           |-- SMPL_FEMALE.pkl
           |-- SMPL_NEUTRAL.pkl
           |-- SMPL_MALE.pkl
           |-- SMPLX_FEMALE.pkl
           |-- SMPLX_NEUTRAL.pkl
           |-- SMPLX_MALE.pkl
   
   

8. Download data and pretrained models from SMPLOlympics

   cd <LMA repository>
   bash third_party/SMPLOlympics/download_data.sh

Training

Training High-level policy for task stage

We leveraged the provided code from SMPLOlympics to acquire single-agent skills for table tennis and tennis.

• table tennis:

  cd <LMA repository>
  bash ./lma/scripts/tabletennis/run_tabletennis.sh

• tennis:

  cd <LMA repository>
  bash ./lma/scripts/tennis/run_tennis.sh

Training LMA for multi-agent interaction stage

Update the high-level policy weight paths in the files under <LMA repositry>/lma/data/cfg/[learning/env].

• <LMA repositry>/lma/data/cfg/learning/*.yaml

  ...
      checkpoint: "<your weight path>"
  ...
  

• <LMA repositry>/lma/data/cfg/env/*.yaml

  ...
  task_models:
      ...
          model: "<your weight path>"
      ...
  

We provide python scripts to train our models and baselines:

• method:
  • (i) scratch: PULSE (from scratch)
  • (ii) finetune: PULSE (fine tuning)
  • (iii) expansion: PULSE (fine tuning w/ expansion)
  • (iv) residual: PULSE + residual RL (action)
  • (v) film: PULSE + residual RL (hidden-layer)
  • (vi) kl_penalty: PULSE+KL
  • (vii) lma_w_freeze: PULSE + LMA (ours w/ freeze)
  • (viii) lma: PULSE + LMA (ours w/o freeze)
  • (ix) sepmc: PULSE+SEPMC
  • (x) lma_w_freeze_sepmc: PULSE + LMA + SEPMC (ours w freeze)
  • (xi) lma_sepmc: PULSE + LMA + SEPMC (ours w/o freeze)

All scripts are in the scripts folder. Please check the contents of the script and pick one command for training.

• Cooperation:

  cd <LMA repository>
  python3 ./scripts/<tablettennis2/tennis2>/run_all_<method_name>.py
  # e.g) python3 ./scripts/tennis2/run_all_lma.py

• Competition:

  cd <LMA repository>
  python3 ./scripts/<tablettennis2/tennis2>/run_all_<method_name>_compete.py
  # e.g) python3 ./scripts/tennis2/run_all_lma_compete.py

To evaluate, append no_virtual_display=True epoch=-1 test=True env.num_envs=1 headless=False to the end of the command.

Run round-robin tournament

To evaluate the strength of the models trained by each method at each step, a round-robin tournament is conducted using the following python program:

```bash
cd <LMA repository>
python3 ./scripts/<tablettennis2/tennis2>/run_all_match.py
```

Acknowledgement

This code builds upon the following repositories. Please visit the URLs to see the respective LICENSES:

• SMPLOlympics
• SMPL model
• PULSE