MimicLite User Guide

MimicLite is an open-source general motion tracking system for humanoid robots from RoboParty Lab. Given a reference motion (from a motion-capture dataset, or human motion captured live by PICO), the trained policy controls the robot to follow it under physical constraints. The same policy can both replay motions offline and be used for real-time teleoperation.

MimicLite's focus is fast training: with 8 RTX 4090s, a deployable policy can be trained in about 3 hours.

Resources:

This section introduces MimicLite's components, the released models, and the environment setup. After completing this section, you can run the released policies in simulation and prepare the training environment.

MimicLite consists of four repositories; Roboparty/MimicLite is the project entry point:

All three models only need 0.08 seconds of future reference motion, making them suitable for low-latency teleoperation. The ONNX and YAML files for deployment are in the shared sim2real artifacts. Evaluation results for each model are in the tracking leaderboard.

active-adaptation and mimic-lite must be cloned separately; mimic-lite goes under active-adaptation's projects/:

Create a separate environment for each simulation backend:

Enable MimicLite and confirm the tasks are registered:

Set up accounts:

Note

• If mjlab training errors out with missing mujoco.mjtEnableBit.mjENBL_MULTICCD, the environment has mujoco>=3.8 installed. Run uv --project venv/mjlab add 'mujoco<3.8' and then re-sync.

• IsaacLab may mistakenly use the Warp bundled with Isaac Sim instead of the warp-lang installed in the environment. When this happens, clear the cache by following the Troubleshooting section of the mimic-lite README.

On the workstation:

Download checkpoints/ and third_party/ from the shared sim2real artifacts and place them in the repository root:

Run MimicLite v1.1 in MuJoCo. Terminal A:

Terminal B:

After both processes have started, press ] in the policy terminal to begin tracking, then open the mjviser URL printed by base_sim.py to view it.

Check inference speed:

This section prepares the motion data used for training. MimicLite uses the unified format defined by any4hdmi; data conversion and inspection are handled by the any4hdmi repository.

An any4hdmi dataset consists of:

The positions and orientations of all body parts can be computed from qpos and the robot model, so they don't need to be stored redundantly. On the first load during training, forward kinematics is computed once and cached.

Already-converted datasets are in the any4hdmi Hugging Face collection, including LAFAN, 100STYLE, the AMASS hard subset, ground motions, high-dynamic motions, PICO-captured data, and more. Training configs can reference them directly with hf:// paths, without downloading them locally.

Note The BONES-SEED data used by SONIC is under license restrictions and cannot be redistributed. It must be obtained from the original source and then converted with any4hdmi.

Run the following in the any4hdmi repository:

To use a different robot model, specify the MJCF on Hugging Face with --mjcf-repo, --mjcf-path, and --mjcf-revision.

Converted datasets can be uploaded to Hugging Face with uv run any4hdmi-upload <local directory> <HF repo name>.

Motions captured live with PICO and retargeted to the robot can be recorded directly as an any4hdmi dataset (you need to set up PICO per Section 5.4 first):

The output directory looks like <robot>_motion_YYYYMMDD_HHMMSS/, containing manifest.json and motions/motion.npz.

The data combinations used for training are written under mimic-lite's cfg/task/motion/, for example <robot>/lafan.yaml, <robot>/mixture.yaml. Each data source can set:

Note In the default mixture config, some data sources (e.g. SONIC, LAFAN) point to local directories under ../any4hdmi/output/. Before use, convert them per Section 2.3, or change the paths to the corresponding hf:// datasets.

This section trains the tracking policy. All commands are run from the root of the active-adaptation repository. In the commands, <robot> denotes the robot directory under cfg/task/motion/.

Single-stage PPO, 8-GPU training:

The arguments of launch_ddp.sh are, in order: GPU IDs, the training script, the simulation backend environment, followed by Hydra config overrides.

PPO-ROA runs in three sequential stages: first train a teacher with privileged information and a longer future reference, then have a student that only uses deployable inputs imitate the teacher, and finally fine-tune the student directly with PPO.

• +task/patches=teacher_future_t16 lets the teacher see 16 frames (0.32 seconds) of future reference; the student still only sees 0.08 seconds.

• If you run out of GPU memory, append task.command.diff_future_steps=[0,1].

• The division of the three stages is written in cfg/stages/, defaulting to normal; v1.1 uses cfg/stages/v1_1.yaml (4000 / 1000 / 2000 updates).

• The run directory is ./outputs/<date>/<time>-<task name>-<algorithm name>/.

• By default, the model is saved every 500 updates and uploaded to W&B.

• Afterwards you can reference the model on W&B directly with checkpoint_path=run:<entity>/mimic_lite/<run_id>.

This section views the trained policy in simulation and exports the ONNX and YAML that sim2real can use directly.

View a PPO model:

View only the fine-tuned PPO-ROA student:

When viewing, disable root drift termination (root_pos_error.enabled=false) so you can observe long motions continuously. Replace <entity> and <run_id> in checkpoint_path with your own W&B run to view your own trained model.

Append export_policy=true to the viewing command; to export only without opening the simulation, also add export_only=true.

The exported files are under mimic-lite's scripts/exports/<task name>/, named policy-<run_id>-<checkpoint number>:

sim2real automatically loads the ONNX with the same name as the YAML, so this exported pair can be used directly as --policy-config (see Section 5).

Note The ONNX and YAML must come from the same export and be used together.

scripts/eval.py is used for quantitative evaluation: by default it runs 512 parallel environments for 1000 steps headlessly and outputs a TensorDict for later statistics. For unified cross-codebase evaluation, see Section 6.

This section uses the sim2real repository to validate the policy in MuJoCo, deploy it to the real robot, and teleoperate with PICO.

sim2real separates the policy from the execution side: the policy process reads the robot state, runs ONNX, and issues joint commands; the execution side is MuJoCo in simulation and the robot body on the real robot.

Two hardware layouts are supported: a workstation controls the robot over an Ethernet cable, or all programs run on the robot's onboard compute unit.

Control methods:

The control method is selected by --controller keyboard|joystick|pico, defaulting to keyboard. joystick is the robot's remote controller; see the sim2real docs for the key mapping.

Terminal A starts MuJoCo, which prints the mjviser URL:

Terminal B starts the policy:

<motion path> is a motion from an any4hdmi dataset; it can be a local .npz or an hf:// path.

Press ] in the policy terminal to start. In the mjviser interface you can toggle the virtual gantry (Elastic Band) and adjust its length.

Single-process run: the policy and MuJoCo can also run in the same process. It places the robot at the first frame of the motion, waits 5 seconds, then starts tracking, and holds the last frame when finished:

Add --headless to run without a GUI; add --trajectory-output <path>.npz to save the trajectory, then use the scripts under scripts/tracking_experiment/ to compute tracking metrics.

Install robot dependencies:

See the Root Project section of the sim2real docs for the name of <robot communication extra>.

If installation reports Could not locate cyclonedds, you need to build and install CycloneDDS (0.10.x) and set CYCLONEDDS_HOME; see the Root Project section of the sim2real docs for the steps.

Choose the robot communication method:

See the Robot I/O section of the sim2real docs for both bridge scripts.

The recommended inline method:

If the robot's network interface is not eth0, add --robot-interface <interface name>.

Tip On the real robot, if the joints shake violently or high-dynamic motions degrade noticeably, first suspect unstable ZMQ communication latency and switch to --robot-io inline to retry, rather than rushing to tune the policy or gains.

Network layouts:

• Cable connection: all programs run on the laptop; the laptop connects to the robot via an Ethernet cable and to PICO via the lab Wi-Fi.

• Onboard operation: all programs run on the robot's onboard compute unit; the robot, laptop, and PICO all connect to the lab Wi-Fi, and the laptop is only used for SSH.

When running on the robot, it's recommended to set HF_HUB_OFFLINE=1 and HF_HUB_DISABLE_TELEMETRY=1 to prevent programs from trying to access the network.

Prepare the teleoperation environment (workstation):

1. Install venv/pico: uv sync --project venv/pico

2. Install the .deb of XRoboToolkit PC Service and build and install xrobotoolkit_sdk (see the Teleop Project section of the sim2real docs for the steps).

3. Put on the leg trackers, complete full-body tracking calibration, connect the workstation in XRoboToolkit, and enable the full-body data stream.

For the configuration to run teleoperation on the robot's onboard compute unit, see the Onboard section of the Teleop Project in the sim2real docs.

Run:

First confirm in Terminal A's mjviser that the retargeted robot motion is correct. On the real robot, drop Terminal B and add --robot-io inline per Section 5.3.

Operations on PICO: press A to enter the initial pose, press A + B to enable the policy, and press X to start following the motion stream. If retargeting and the policy run on different computers, add --motion-zmq-connect tcp://<retargeting computer IP>:28701 to the policy.

Tip sim2real also supports using the PICO trigger to control the DH116S dexterous hand; see the sim2real docs for details.

This section evaluates policies under a unified protocol and explains how to run or integrate policies trained with other codebases.

The tracking leaderboard uses four frozen subsets from ChingMu's MotionDecode data; all policies run in the same MuJoCo environment using CPU ONNX Runtime:

Evaluate your own policy:

• By default, Locomotion-80 and Manipulation-48 are evaluated; add --splits locomotion manipulation ground dance to evaluate all four subsets.

• Repeat --policy name=path to compare multiple policies at once.

• Data is cached in datasets/motiondecode; the script verifies each file; --dry-run only downloads and verifies the data without running the simulation.

• Each subset outputs trajectories, runs.csv, tracking_metrics.csv/json, and summary.json.

Key points of the evaluation protocol:

• If the pelvis height error exceeds 0.3 m, or the gravity direction error exceeds 0.8, it is judged a failure; failure is not judged by root position.

• Progress and Tracking Return are both truncated at the failure frame; errors are only counted for frames before the failure, so they should be read together with Progress.

• Global root error is only reported on Locomotion.

• These data are only for evaluation; do not resample them or use them for other robots.

sim2real has converted a variety of external policies into the same deployment format; you only need to replace --policy-config:

For the full list and each policy's special requirements (e.g. SONIC's SMPL mode needs a separate motion source), see the Run External Policies section of the sim2real docs.

Having just an ONNX file is not enough: the construction of observations, joint order, PD gains, and action scaling must all match training. Integrating a new policy requires:

1. Export the ONNX;

2. Implement the corresponding observation class and write the deployment YAML;

3. Validate in Sim2Sim.

The sim2real repository includes a Codex skill (.agents/skills/adapt-policy-to-sim2real) that walks through these steps. Models trained with MimicLite itself just use the export in Section 4.2 and don't need this step.