Evaluation and Simulation

This guide follows Reinforcement Learning Training, covering policy evaluation and export, MuJoCo Sim2Sim validation, and the deployment handoff checklist. If you already have a checkpoint, start here and verify the policy inputs and outputs.

play.py loads a checkpoint matching the task, creates an inference policy, and runs the environment while exporting TorchScript and ONNX. Complete the I/O and deployment configuration checks before using it on the physical robot.

Walking motion · Simulation

Multiple simulated models walk in a flat scene.

Download video

Set RP1_CHECKPOINT to the path of an existing checkpoint file that matches the task:

Use RP1.3-Rough for Rough policies. The task, training configuration, action dimensions, and checkpoint must match.

If no explicit checkpoint is provided, the script searches logs/rsl_rl/<experiment_name> using the agent's run/checkpoint criteria. For reproducibility, record the file actually loaded and its hash.

By default, exports are written to exported/ under the directory containing the checkpoint:

The base play branch exports automatically, without --exportonnx. RP1.3 uses a direct environment, so its I/O contract must be recorded separately; --export_io_descriptors supports only manager-based environments.

Play first looks for actor_obs_normalizer, then student_obs_normalizer, and passes None if neither attribute is present. It passes the normalizer to the Isaac Lab export helper.

Before release or deployment, inspect the exported file's inputs/outputs, normalizer handling, statistics, shapes, dtypes, and external data files, and associate them with the checkpoint and configuration that produced the export. See Policy inputs and outputs for the checklist.

Record task playback, Sim2Sim, and physical robot results separately, including scenes, command conditions, seeds, model/policy versions, and output artifacts. Use the formal acceptance specification for the corresponding version to determine pass criteria on the physical robot.

The RP1.3 entry point uses a 24-action TorchScript policy and converts the Isaac policy order to the MuJoCo model order. This entry point checks simulation transfer in MuJoCo.

Prepare the training environment and MuJoCo dependencies, then set RP1_TORCHSCRIPT to the matching exported/policy.pt path:

Add --terrain when using the rough scene. Adding --headless selects the path without a GUI and saves a video; suitable EGL/rendering support is still required, so it does not mean rendering is disabled entirely.

Motion tracking uses sim2sim_rp1.3_bm.py. Set RP1_TORCHSCRIPT to a TorchScript policy matching RP1.3-BeyondMimic, then use the same reference motion:

These paths are relative to the root of the training repository. This entry point uses 24 actions, a 126-dimensional single-frame input, and the RP1.3 joint mapping; --loop loops the reference motion. A 243-dimensional Flat/Rough policy cannot be passed directly to this entry point.

The Flat/Rough scenes come from rp1.3/mjcf/ in roboparty_train and both reference robot_24dof.xml. The configuration uses 24 actions, 81 observations per frame, and 3 history frames, totaling 243 observation elements. torch.jit.load takes a TorchScript file as its input.

The shared run_mujoco checks index permutations, gain lengths, observation dimensions, and model nq/nv/nu. The RP1.3 script calls this shared function; the RPO main entry point with 23 actions is for a different configuration.

The MuJoCo order is 6 left-leg joints, 6 right-leg joints, 2 waist joints, 5 left-arm joints, and 5 right-arm joints. The source defines the following policy→MuJoCo permutation:

The default pose, PD gain, and torque clipping arrays in the same file are all arranged in MuJoCo order. These arrays are for simulation; confirm physical ratings and safety limits against the device specifications.

The shared function saves simulation.mp4, joint_positions.png, and base_velocities.png. Record the scene, policy, configuration, actual output location, and check results.

Before connecting to the inference implementation, verify the history layout and I/O contract. The 28-action workflow requires its own task, policy, and Sim2Sim entry point.

Save the policy actually used and its hash, model and mappings, reference motion or velocity commands, initial conditions, playback, and output records. Sim2Sim checks an independent software path; calibration, feedback freshness, protection, and constrained tasks on the physical robot require separate validation.

Continue with the deployment workflow, and check the deployment resources and configuration.

BeyondMimic uses a 24-axis body and a 126-dimensional single-frame motion tracking input; the default 28-axis velocity policy uses a 930-dimensional history input. Before deployment, select the corresponding policy path, then check the deployment resources and configuration. Passing Sim2Sim does not replace calibration and validation of the delivered configuration on the physical robot.