Human-Humanoid-Tools

Human-humanoid-tools (hhtools) is RoboParty Lab's open-source motion data toolchain for humanoid robots. In about 30 seconds, it retargets human motion (motion capture data, public datasets, monocular video) or another robot's motion onto any URDF humanoid robot. It also provides batch processing and dataset analysis.

The output is a joint trajectory for the target robot (CSV or PKL), which can be used as motion data for training control policies such as MimicLite and UFO (format conversion may be needed when plugging into each training framework). Retargeting a single complex motion clip takes about 30 seconds.

Resources:

This section covers what hhtools does, the ways to use it, and environment setup. By the end of this section, you will be able to open hhtools in a browser or the desktop app, and use the command line to check that each component is ready.

hhtools can be used in four ways. They share the same motion, robot and retargeting core, so you can pick whichever fits your scenario:

The WebUI and the desktop GUI share the same interface: the desktop GUI is an Electron app that starts the hhtools Python service locally and then loads the same page. Your data always stays on your machine.

We recommend managing the environment with uv. Install uv first, then clone the repository:

Install all features (formats, robots, retargeting, WebUI, Agent / MCP):

If you only need some of the features, choose extras as needed:

If a dependency is missing, the launcher lists the missing packages and the commands to fix it.

Note all installs the Python integration libraries only. It does not include GVHMR or the SMPL-family model weights; you need to prepare these yourself.

Install the built-in robots. hhtools provides 7 curated robots, downloaded from pinned versions of their upstream repositories:

Each robot keeps its upstream license and a checksummed provenance record. Other robots can be imported in the WebUI (see Section 2.2).

Prepare the body model weights. Register and download from the SMPL, SMPL+H and SMPL-X websites, then place the files in the following directories:

hhtools looks in this directory first, then in ~/.cache/hhtools/body_models/; you can also point it to another location with the HHTOOLS_BODY_MODELS environment variable. Without the weights, SMPL-family motions can still be loaded and played back as a simplified skeleton, but without a real body mesh or accurate joint positions. (Note: this cannot be installed on Mac.)

Check the environment. Run the following command to check whether the WebUI, robots, retargeting, MCP, body models and GVHMR are ready. It only runs checks and does not start any task:

Open http://127.0.0.1:8009 in your browser.

Common launch options:

Interface layout. The left navigation has three groups:

The stage in the middle displays motions and robots with three.js, and the right side holds the control panel for the current workflow. All background jobs (loading, retargeting, batch processing, analysis) appear in the task drawer, where you can export results once they finish.

Workspace Settings. Here you can switch the interface language (Simplified Chinese / English; follows the system by default), show or hide the left and right panels, change the motion library directory, check whether GVHMR is ready, force re-analysis, and adjust background-job scheduling.

Note When running on a remote server, access the WebUI through SSH local port forwarding. Until authentication for remote administration is implemented, the settings page is read-only in an ordinary remote browser.

The desktop GUI is currently a 0.1.0 preview release:

• Windows: The installer bundles the Python runtime and the app source, so you don't need to prepare Python or the source repository separately.

• Linux: Install, then launch:

  sudo apt install ./hhtools-0.1.0-amd64.deb
  hhtools-desktop
  

• macOS: Drag Human-Humanoid Tools into Applications, launch it, and follow the first-launch page to finish setup (an internet connection is required; you don't need to preinstall Python or Homebrew). The preview has not been notarized by Apple yet. If macOS blocks it, choose "Open Anyway" in System Settings → Privacy & Security.

The installers for all three platforms include 30 sample motions and 6 robots, but none of them include GVHMR or the SMPL-family weights.

Note The macOS version retargets on the CPU, and supports neither Intel Macs nor Video to Motion.

The command line uses the same core as the WebUI. When processing tens of thousands of clips, use the command line or the offline scripts instead of dragging files into the browser.

For the usage of each command, see uv run hhtools \<command> --help; examples are given in Sections 2–5.

hhtools provides two interfaces for scripts and AI agents, both backed by the same versioned task contract:

Covered: human-to-robot retargeting (motion only, human–object interaction, terrain scenes), robot-to-robot retargeting without scenes, batch processing for both, and calibration assistance. Calibration assistance can generate calibration proposals, run deterministic validation, and output front and side preview images for vision-capable models to inspect.

Not covered yet: robot-to-robot retargeting with scenes, Video to Motion, Analysis, and real-robot deployment.

Install and check:

hhtools-mcp is a stdio server: it must be launched by an MCP client rather than run directly in a terminal. The repository ships a Codex project config (.codex/config.toml). After opening Codex in the trusted repository directory, use /mcp to confirm that hhtools is connected.

Recommended flow: query capabilities → register and inspect motions and robots → check calibration status (if missing, go through calibration proposal, validation, preview and save) → preflight in smoke mode to get an executable plan → start the job and wait → review the evaluation results and manifest → once everything looks right, run a full preflight again, and run the full job only with the user's explicit consent.

Note

• A job being "complete" does not mean the motion quality is acceptable. Final quality and full runs must be confirmed by a human.

• A save directory can be used by only one running instance at a time. Do not let the MCP server and the WebUI use the same directory at the same time; to switch, disconnect one before opening the other.

See Agent interfaces in the repository for full details.

This section retargets a human motion clip to a target robot. In the WebUI, open Workflows → Human → Robot and complete four steps: Motion → Target robot → Calibration → Result.

hhtools automatically recognizes motions from the following sources and converts them into its internal Motion format:

Manage the motion library under Assets → Motion:

• Built-in motions: The repository ships 30 sample motions covering all three types above.

• Import: Choose an import type (motion / object interaction / terrain scene), then select or drag in files or folders.

• Scan a directory: The WebUI recursively scans the directory given by --source; you can also change the library directory in Workspace Settings.

Note hhtools only provides format adapters and does not redistribute datasets. Download the full data from each dataset's official website and follow its license.

Manage the robot library under Assets → Robot. Besides the 6 built-in robots from Section 1.5, you can import any URDF humanoid robot:

1. Select the .urdf file;

2. Select the corresponding mesh folder;

3. hhtools automatically detects the joint structure and generates the config file robot.yaml (including ik_map, the mapping from human joints to robot links), with no manual tuning needed.

Imported robots are saved in the user config directory (~/.config/hhtools/robots/\<name>/ on Linux) and can be removed from the library when no longer needed.

Humans and robots differ in height, proportions and zero pose. Calibration determines the scaling and correspondence between the two. Each robot needs to be calibrated only once per reference pose, and the calibration is reused for all motions of the same kind afterward.

Reference poses are distinguished by motion source, for example SMPL, SMPL-X, GVHMR, LAFAN BVH, SOMA BVH, mocap BVH, Xsens (100STYLE) and GLB. hhtools recommends one automatically based on the selected motion.

Calibration steps:

1. Select a reference pose and click Edit calibration;

2. Adjust the joint values until the robot pose aligns with the reference skeleton; or click Auto-propose pose to have hhtools generate a calibration proposal, which comes with a score and highlights the joints that still need adjustment;

3. Check that everything looks right, then save.

The calibration is saved next to the robot's URDF as retarget_calibration_\<reference pose>.yaml.

hhtools has two solvers. The WebUI recommends one automatically depending on whether the motion includes objects or terrain, and you can also switch manually:

Set Retarget FPS, then click Start Retarget. The first run for a given robot takes longer to load.

When retargeting finishes, switch between the Source, Target, Result and Overlay views on the stage to compare the human motion with the robot motion.

Result diagnostics reports the following metrics and rates them as Stable / Review / Large deviation:

Note Result diagnostics are only for spotting tracking and contact anomalies. Validate the final motion in simulation before using it on a real robot.

The CSV columns are, in order: time, root position (x, y, z), root quaternion (qx, qy, qz, qw), and the joint angles dof_\<joint name>.

Results with objects or terrain are packaged as a ZIP containing the robot trajectory, each object's trajectory, and the object and terrain meshes, all converted to the robot's scale and coordinate frame.

If the retargeting result is unsatisfactory, edit the robot's robot.yaml. Built-in robots are in configs/robots/\<name>/, and imported robots are in ~/.config/hhtools/robots/\<name>/. Changes take effect on the next retarget, without restarting the WebUI.

Tip If the human's arms are already hanging down but the robot still holds its arms out, first lower the value of \*_shoulder_roll_link in smooth_joint_filter_masks to 0.1–0.3, rather than only raising weights.

Note Re-uploading the URDF regenerates robot.yaml, which may overwrite a manually edited ik_map and weights; calibration files are kept.

See configs/robots/_template/robot.yaml in the repository for the full template, and framework.md for the underlying principles.

Use uv run hhtools robot list to see the available values of \<robot name>. Before retargeting from the command line, complete calibration in the WebUI, or prepare the matching calibration file next to the URDF.

This section transfers existing motion from one robot to another. It is useful when you only have motion data for one robot (for example, a dataset already retargeted to RP1) and want to use it on a different robot. In the WebUI, open Workflows → Robot → Robot and complete five steps.

hhtools treats the source robot as if it were a human:

1. It runs forward kinematics on every frame with the source robot's URDF to get the spatial positions and orientations of the key joints;

2. It arranges them into a standard skeleton according to the ik_map in the source robot's robot.yaml, which is equivalent to a human motion clip;

3. It calibrates the target robot, using the source robot's zero pose as the reference;

4. It solves the target robot's joint trajectory with Newton IK.

Note If the source file has no time column or sample rate (for example, MotionDecode), you must set Source trajectory FPS manually. Set it to 120 for MotionDecode; processing it at the default 50 Hz makes playback and retargeting run at the wrong speed.

The steps here are similar to Human -> Robot; follow Section 2.

1. Source robot: Select and load the robot the trajectory originally belongs to.

2. Source trajectory: Load from the robot trajectory library, or upload files or folders; set Source trajectory FPS as needed.

3. Target robot: Select and load the robot to transfer the motion to.

4. Calibration: Calibrate this pair of robots once; after you save, the calibration is reused automatically. Calibrations are saved per "source robot + target robot" pair, so a different pair needs its own calibration.

5. Result: Set Retarget FPS and start retargeting. Review and export the result the same way as in Sections 2.5 and 2.6.

Before transferring from the command line, complete the calibration for this robot pair in the WebUI, or specify a calibration file with --calibration.

This section estimates human motion from a monocular video. The generated motion is saved to the motion library automatically, and can then be retargeted to a robot as described in Section 2. hhtools runs inference with the officially released weights of GVHMR.

hhtools does not bundle GVHMR's source code, weights or Python environment. Install it separately by following the official GVHMR instructions.

Linux: hhtools calls your installed GVHMR environment in a separate subprocess. Set the paths before launching the WebUI:

Requirements:

• The GVHMR directory keeps the official inputs/checkpoints structure, including the licensed inputs/checkpoints/body_models/smplx/SMPLX_NEUTRAL.npz;

• The selected Python environment has GVHMR's dependencies and CUDA;

• ffmpeg is on PATH.

If no paths are set, hhtools tries ~/GVHMR, a virtual environment in the repository, and a Conda environment named gvhmr, but setting them explicitly is the most reliable. Inference times out after two hours by default; adjust this with HHTOOLS_GVHMR_TIMEOUT_SECONDS.

Windows: GVHMR runs as an optional Docker component. Point HHTOOLS_GVHMR_ROOT to the official source directory and HHTOOLS_GVHMR_IMAGE to a prepared image; if the body models are stored separately, mount them with HHTOOLS_GVHMR_BODY_MODELS.

Desktop GUI: You can configure or repair GVHMR under "Optional components" in Workspace Settings.

Note macOS does not support Video to Motion.

Open Workflows → Video → Motion:

1. Select video: MP4, MOV, MKV, AVI, WebM and M4V are supported.

2. Environment: Confirm that GVHMR and the official weights show as "Ready". If they are not ready, the page lists what is missing.

3. Generate: Choose whether to enable Static camera depending on how the video was shot, set the focal length to auto or enter it manually, then click Start GVHMR.

4. Motion result: Shows the frame count, duration and frame rate. The generated motion is already registered in the motion library.

Results you have already produced with GVHMR (hmr4d_results.pt) can be imported directly under Assets → Motion, or converted from the command line:

The conversion requires licensed SMPL-family body models on your machine (see Section 1.5).

Note Video to Motion only uses GVHMR's officially released weights, and does not support custom weights or training.

This section processes many motion clips at once. Tens to hundreds of clips can be handled in the WebUI under Workflows → Batch; for whole datasets (tens of thousands of clips), use the offline scripts.

Note Before batch processing, complete the relevant calibration in Motion to Robot or Robot to Robot. Batch processing does not open the calibration interface.

The Batch page supports three kinds of jobs:

Human → Robot batch

1. Input: Drag in motion files or folders, or add them from the motion library; duplicates are skipped automatically.

2. Target robot & calibration: Load the target robot. The page groups the input motions by reference pose and checks whether each group is calibrated.

3. Run settings:

   Setting	Description
   Solver	Newton IK runs in parallel chunks on the GPU; Interaction-Mesh processes clips one at a time
   GPU batch size	Number of motions per Newton IK chunk (1–256); tuned automatically if left blank
   Source FPS / Retarget FPS / Export FPS	Export FPS follows Retarget FPS by default
   Start / End time	Processes only a segment of each clip
   Output format	CSV or PKL, with an optional CSV header
   Result name	Name of the downloaded ZIP

4. Click Start H2R batch. The page shows overall and current progress; when the job finishes, it lists failed items with their reasons, and you can download the ZIP.

Robot → Robot batch

Drag in source trajectories (CSV, PKL, NPZ or dataset folders), load the source and target robots, set the frame rates, then click Start R2R batch. This robot pair must first be calibrated in Robot → Robot.

Video → Motion batch

Drag in multiple videos or folders, confirm the GVHMR runtime, set the static camera option and focal length, then click Start V2M batch. Videos are processed one by one in a queue, and the generated motions are saved to the motion library. For items that fail midway, click Retry unfinished.

The offline scripts are designed for large-scale data. They support resuming after an interruption, process each clip in its own subprocess, and export the same content as the WebUI.

Common options:

Output layout: motion-only clips are written to \<out>/…/\<name>.csv; clips with scenes are written to \<out>/\<name>/\<name>.csv, alongside the object or terrain files in the robot's coordinate frame.

Tip If Newton IK batch solving fails, it automatically falls back to processing clips one at a time. For tumbling motions, turn off foot grounding correction (see Section 2.7).

Background jobs in the WebUI have no concurrency limit by default. On a shared server, or when GPU memory is tight, you can enable first-in, first-out queuing:

You can also change these under Workspace Settings → Background-job scheduling. Changes take effect as soon as you save, without interrupting running jobs. Command-line options and environment variables override the values saved in the UI on the next launch.

Note The concurrency limit applies only to queued web jobs. It does not cover the solver warm-up that may be triggered when you select a robot, so it is not a GPU concurrency limit for the whole process.

This section analyzes a motion dataset: it computes kinematic and physical quality metrics for each clip, assigns interpretable tags, clusters the clips in an embedding space, and recommends a training subset that is both diverse and challenging. In the WebUI, open Analysis → Data Analysis.

Analysis never deletes or moves any source files. Poor-quality motions are only tagged; whether to keep them is up to you.

1. Select data: Choose dataset folders (you can add several to the upload basket), or use the built-in library. Human motions and retargeted robot trajectories can sit in the same directory and be analyzed together.

2. Configure: Choose an embedding method. Hand-crafted features are currently used; PAE is a reserved option.

3. Analyze: Click Start analysis to compute dynamics, quality, tags, embeddings and clusters. Analyzed data is cached, so next time you can click Load existing result directly; the cache is invalidated automatically when the inputs, scene files or configuration change.

4. Results: Explore the data with histograms, scatter plots and filters, then recommend and export a subset.

Physical quality S_phy. Points are deducted from 100 for six kinds of problems:

A score of 90 or above is tagged quality_ok, 70–90 is quality_warn (manual review recommended), and below 70 is quality_bad. For motions with terrain, floating, penetration and sliding are computed against the local terrain height.

Dynamic level. Motion complexity is computed from joint velocities and accelerations, then split into five levels by library-wide quantiles: static, low_dynamic, mid_dynamic, high_dynamic and burst.

Behavior tags:

There are also scalar metrics such as root speed, turning rate, airborne ratio, step frequency, leg and arm energy, maximum torso tilt and terrain roughness, which you can inspect one by one in the interface.

• Metric histogram: See how any metric is distributed across the library, and brush-select a range to filter the matching clips, for example all motions whose horizontal speed falls within a certain range.

• Filters: Filter by tag, source (human / robot) and folder.

• Embedding scatter plot: Each point is a motion clip; the closer two points are, the more similar their behavior, and colors indicate clusters. The embedding standardizes 15 hand-crafted features and reduces them to 2D (with UMAP if it is installed, otherwise PCA). Clustering uses HDBSCAN when available, otherwise KMeans.

• Preview: Click any clip to play it on the stage; robot trajectories are matched to their robot automatically.

Recommended subset picks, from the currently filtered clips, a set of motions that is both diverse and weighted toward difficulty:

At each step, the clip with the highest score is selected, until the target ratio is reached. The default ratio is 10% and α defaults to 0.99; the smaller α is, the more the selection favors complex motions. The recommendation only highlights clips without removing any others, and you can add or remove clips manually on top of it.

Selected clips can be exported as a JSON or CSV manifest; robot trajectories can also be exported with Export robot ZIP.

The analysis thresholds and weights are in configs/analysis/default.yaml, including the height thresholds for floating and contact, the foot-slide speed threshold, the quality deduction weights, the α and default ratio for subset recommendation, and the number of parallel processes.

Note Analysis is currently available only in the WebUI; the command line and Agent interfaces do not support it yet.