AnyMorph: Learning Transferable Polices By Inferring Agent Morphology

ICML 2022

OpenReview | Arxiv

Blog post and here as Mila

Brandon Trabucco, Mariano Phielipp, Glen Berseth

TL;DR

Our paper, Learning Transferable Policies By Inferring Agent Morphology, delivers state-of-the-art generalization and robustness for controlling large collections of reinforcement learning agents with diverse morphologies and designs.

@inproceedings{Trabucco2022AnyMorph,
title={AnyMorph: Learning Transferable Policies By Inferring Agent Morphology},
author={Trabucco Brandon and Phielipp Mariano and Glen Berseth},
journal={International Conference on Machine Learning},
year={2022}
}

Setup

All the experiments are done in a Docker container. To build it, run ./docker_build.sh <device>, where <device> can be cpu or cu101. It will use CUDA by default.

To build and run the experiments, you need a MuJoCo license. Put it to the root folder before running docker_build.sh.

Running

./docker_run <device_id> # either GPU id or cpu
cd amorpheus             # select the experiment to replicate
bash cwhh.sh             # run it on a task

We were using Sacred with a remote MongoDB for experiment management. For release, we changed Sacred to log to local files instead. You can change it back to MongDB if you provide credentials in modular-rl/src/main.py.

Acknowledgement

• The code is built on top of SMP repository.
• NerveNet Walkers environment are taken and adapted from the original repo.
• Initial implementation of the transformers was taken from the official Pytorch tutorial and modified thereafter.