Reinforcement Learning Training

The RPO training framework is built on IsaacLab 2.3.1, IsaacSim 5.1.0, and rslrl 3.2.0, providing a direct reinforcement learning environment similar to legged_gym and a manager-based reinforcement learning environment native to IsaacLab. This framework supports sim2sim transfer in MuJoCo and the terrain export function of IsaacLab.

This framework implements tasks such as blind walking, anthropomorphic gait (AMP), perception-based multi-terrain walking (AttnEnc, see Attention-Based Map Encoding for Learning Generalized Legged Locomotion), and separate control of upper and lower limbs (Interrupt, see HugWBC). All of the above tasks have completed sim2sim and sim2real validation on the RPO platform.

Open Source Address: RPO training repository

RPO Train is built on the latest version of IsaacSim/IsaacLab, and it is recommended to follow the latest updates of RPO Train.

• Install Isaac Lab according to the Installation Guide. We recommend using conda for installation because it simplifies the process of invoking Python scripts from the terminal.

• Clone this repository independently of the Isaac Lab installation (i.e., outside the IsaacLab directory):

• Install libraries using the Python interpreter with Isaac Lab installed:

• Verify that the extension is correctly installed by running the following command, which prints the list of available environments:

For the workflow of obtaining datasets for AMP and BeyondMimic, please refer to GMR .

The joint order in the dataset obtained through GMR is consistent with the order in the robot's URDF and XML, but it differs from the order used by Isaac Lab. Therefore, we need to prepare a .yaml file (as shown in scripts/tools/retarget/config/atom01.yaml) that contains joint mapping information, and use scripts/tools/retarget/dataset_retarget.py to reorder the joint sequence before training.