PPO Quadruped Locomotion

This project implements quadruped robot locomotion training using Proximal Policy Optimization (PPO) with the Genesis physics simulation engine.

Overview

Train a quadruped robot to walk, run, and jump using reinforcement learning. The implementation uses:

• Genesis: High-performance physics simulation engine
• RSL-RL: Robotics Systems Lab's RL library for locomotion
• PPO: Proximal Policy Optimization algorithm

Project Structure

ppo-dog/
├── genesis/           # Genesis physics simulation engine
├── rsl_rl/           # RSL-RL reinforcement learning library
├── src/              # Main training and evaluation code
│   ├── train.py              # Training script
│   ├── quadruped_env.py      # RL environment implementation
│   ├── evaluate.py           # Evaluation script
│   ├── teleop.py            # Teleoperation interface
│   └── checkpoints/         # Saved model checkpoints
└── README.md         # This file

Requirements

• Python 3.8+
• CUDA-capable GPU (recommended)
• PyTorch
• Genesis physics engine
• RSL-RL library

Installation

1. Clone the repository:

git clone <repository-url>
cd ppo-dog

2. Install Genesis:

cd genesis
pip install -e .
cd ..

3. Install RSL-RL:

cd rsl_rl
pip install -e .
cd ..

4. Install additional dependencies:

pip install torch numpy

Usage

Training

Train a new quadruped locomotion policy:

cd src
python train.py --exp_name my_run --device cuda:0

Arguments:

• --exp_name: Name for the experiment (default: locomotion_run)
• --device: Device to use (default: cuda:0, options: cuda:0, cpu)
• --num_envs: Number of parallel environments (default: 4096)
• --max_iterations: Maximum training iterations (default: 500)

Training checkpoints will be saved to src/checkpoints/<exp_name>/.

Evaluation

Evaluate a trained policy:

cd src
python evaluate.py --exp_name my_run --ckpt 200

Arguments:

• --exp_name: Name of the experiment to evaluate
• --ckpt: Checkpoint iteration to load (default: 200)
• --device: Device to use (default: cuda:0)

Teleoperation

Interactively control the robot:

cd src
python teleop.py --exp_name my_run --ckpt 200

Use keyboard commands to control velocity and test the trained policy in real-time.

License

This project uses components from:

• Genesis: Check genesis/LICENSE
• RSL-RL: Check rsl_rl/LICENSE

Acknowledgments

• Genesis physics engine team for the high-performance simulation framework
• Robotics Systems Lab (RSL) at ETH Zurich for the RSL-RL library