HandUMI is a hand-worn interface for collecting robot-free bimanual demonstrations. This repository contains its synchronized data collection, calibration, validation, replay, teleoperation, and robot-retargeting software.
Collect once, retarget to many robots. Record demonstrations with HandUMI once, then retarget and reuse the same data across different bimanual arms with parallel grippers, without recollecting demonstrations for each robot.
Requires uv and Python 3.12 or newer.
git clone https://github.com/murobotics-ai/handumi-sw.git
cd handumi-sw
bash install.sh
source .venv/bin/activate
cp configs/rig.example.yaml configs/rig.yaml
handumi doctor
handumi record --output-dir outputs/my-first-dataset --dry-runPICO support is installed by default. Use bash install.sh --skip-xrt for a Meta Quest-only workstation.
Edit configs/rig.yaml once with the local cameras, tracking device and robot
profile. handumi setup --check prints the same read-only readiness checklist
as handumi doctor; handumi setup runs the guided hardware setup. Existing
workflows are grouped under the single handumi command; use handumi --help
to see the final command tree. Bash, Zsh, and Fish completion is loaded when
the virtual environment is activated, so handumi re<Tab> completes record
and replay. The shorter hu executable is an equivalent alias, so commands
such as hu record use the same interface and completion.
HandUMI can also be installed directly from this repository:
pip install "handumi @ git+https://github.com/murobotics-ai/handumi-sw.git"Optional robot and simulation backends can be selected with extras:
pip install "handumi[sim,piper,openarm] @ git+https://github.com/murobotics-ai/handumi-sw.git"For reproducible environments, pin a branch, tag, or commit:
dependencies = [
"handumi @ git+https://github.com/murobotics-ai/handumi-sw.git@main",
]HandUMI can also be installed directly from this repository:
pip install "handumi @ git+https://github.com/murobotics-ai/handumi-sw.git"Optional robot and simulation backends can be selected with extras:
pip install "handumi[sim,piper,openarm] @ git+https://github.com/murobotics-ai/handumi-sw.git"For reproducible environments, pin a branch, tag, or commit:
dependencies = [
"handumi @ git+https://github.com/murobotics-ai/handumi-sw.git@main",
]flowchart LR
A[HandUMI data collection] --> B[Synchronized robot-agnostic dataset]
B --> C[Review: recording + retargeting]
C --> K[Curate]
K --> D[Retarget]
D --> E[AgileX PiPER]
D --> F[OpenArm]
D --> G[TRLC-DK1]
D --> H[I2RT YAM]
D --> J[MakerMods Metal]
D --> I[Other bimanual arms with parallel grippers]
Raw captures remain robot-agnostic. After collecting data with HandUMI, the same
demonstrations can be retargeted to different bimanual arms with parallel
grippers. handumi dataset qa reviews a recording and how a given robot
retargets it, curation removes what the review rejects, and conversion then
executes that decision rather than judging it again.
Before training, handumi replay-real streams a converted dataset to the
physical robot through the same backend teleop-real uses and reports how
closely the arms followed it, so the data that reaches a policy has already
run on the hardware.
Robot configuration and physical controller-to-TCP calibration are fingerprinted in dataset metadata, so a converted dataset records exactly which geometry and calibration produced it. Conversion reuses the trajectories the review solved, which is what makes its output reproducible: the solver warm-starts each frame from the previous one, so re-solving from scratch lands on a slightly different answer for a small fraction of frames.
HandUMI is optimized for fixed-base bimanual manipulators equipped with parallel-jaw grippers. Demonstrations remain robot-agnostic, so support for new embodiments can be added without changing the capture format.
| Bimanual | Repository | Preview |
|---|---|---|
| AgileX PiPER | Repository |  |
| OpenArm | Repository |  |
| TRLC-DK1 | Repository |  |
| I2RT YAM | Repository |  |
More embodiments coming soon. See Add a new robot embodiment to contribute an integration. If you want support for a specific bimanual arm, open an embodiment request.
- Tracking: PICO through XRoboToolkit and Meta Quest through HandUMI Quest App.
- Robot models and simulation: AgileX PiPER, OpenArm, TRLC-DK1, Axol, I2RT YAM, and MakerMods Metal.
- Real-robot teleoperation: AgileX PiPER and OpenArm through optional backends.
- Dataset format: LeRobot-compatible synchronized captures.
- Episode control: hands-free by voice ("start recording", "stop recording", "restart"), recognized offline; gripper double-squeezes and PICO buttons remain available.
This is research software. Preview and validate trajectories before commanding physical robots, keep an emergency stop accessible, and enforce the robot's joint, velocity, acceleration, workspace, and collision limits.
HandUMI builds on UMI, HandUMI Quest App, XRoboToolkit, LeRobot, PyRoki, Viser, Rerun, and MuJoCo. See the documentation and LICENSE for attribution and third-party licensing details.
Project lead and original hardware design: BrikHMP18. Core software contributors include Leonardo Pérez, Raul Bastidas, Mitshell Ramos, and Alvaro Mendoza-Li.
Original HandUMI software and documentation are licensed under the Apache License 2.0. Dataset, hardware, headset application, robot firmware, and trademark licenses remain separate.