LeIsaac 🚀

leisaac.mp4

This repository provides teleoperation functionality in IsaacLab using the SO101Leader (LeRobot), including data collection, data conversion, and subsequent policy training.

• 🤖 We use SO101Follower as the robot (and other related robot) in IsaacLab and provide relevant teleoperation method.
• 🔄 We offer scripts to convert data from HDF5 format to the LeRobot Dataset.
• 🧠 We utilize simulation-collected data to fine-tune GR00T N1.5 and deploy it on real hardware. And more policies will be supported.

Remote Policy Evaluation

LeIsaac can also run rendering-side evaluation while the policy is hosted on a separate remote inference server. In this setup:

• The rendering server runs Isaac Sim, the environment, cameras, and scripts/evaluation/policy_inference.py.
• The policy inference server hosts the model and answers policy requests over ZMQ.
• The rendering server sends observations to the policy server, receives actions, and executes the rollout in simulation.

Typical usage:

python scripts/evaluation/policy_inference.py \
  --headless \
  --enable_cameras \
  --device=cuda:0 \
  --task=LeIsaac-SO101-PickOrange-v0 \
  --eval_rounds=10 \
  --policy_type=molmoact2_so101 \
  --policy_host=<policy-server-host> \
  --policy_port=5556 \
  --policy_timeout_ms=30000 \
  --policy_action_horizon=30 \
  --policy_language_instruction="Pick three oranges and put them into the plate, then reset the arm to rest state."

Use --policy_port=5555 for GR00T N1.5 and --policy_port=5556 for MolmoAct2 SO101 compatibility. For multi-host deployments, make sure the rendering server can reach the policy server over the configured TCP port.

Tip

Welcome to the Lightwheel open-source community!

Join us, contribute, and help shape the future of AI and robotics. For questions or collaboration, contact Zeyu or Yinghao.

News 🗞️

• [26/04/14] Remote teleoperation is now available in LeIsaac! Try it out here.
• [26/03/10] With the new datagen module, LeIsaac can generate motion trajectories programmatically. See State Machine Data Generation.
• [26/01/16] Added inference support for GR00T N1.6; details are in Available Policy Inference.
• [26/01/13] Try our tutorial LeIsaac x Cosmos to get a video2action data generation pipeline.
• [26/01/12] Extra feature of lerobot recorder integration released! You can now record data directly in LeRobot Dataset format during teleoperation.
• [25/12/19] Try our tutorial LeIsaac x Marble to build and evaluate diverse embodied tasks across large-scale generalized environments.
• [25/12/19] We now support lekiwi-based teleoperation and provide a Loft scene for the community. See the example task here.
• [25/11/27] We now support more teleoperation devices, including the enhanced keyboard and gamepad. Refer to the device guide for usage details.
• [25/11/26] LeIsaac is now the official imitation-learning (IL) simulation playground integrated into LeRobot’s EnvHub. It provides fast, scalable Isaac-based environments designed for imitation learning, control, and policy evaluation. It is featured in LeRobot: LeIsaac × LeRobot EnvHub

Getting Started 📚

Please refer to our documentation to learn how to use this repository. Follow these links to learn more about:

• Installation and Setup
• Extra Features
• Policy Inference
• Available Robots, Environments and Policy

Contributing 🤝

Please see CONTRIBUTING.md for how to report issues and submit pull requests.

Citation 📝

If you use leisaac, please cite it as follows.

@software{Lightwheel_and_LeIsaac_Project_Developers_LeIsaac_2025,
author = {{Lightwheel} and {LeIsaac Project Developers}},
license = {Apache-2.0},
month = dec,
title = {{LeIsaac}},
url = {https://github.com/LightwheelAI/leisaac},
version = {0.4.0},
year = {2026}
}

Acknowledgements 🙏

We gratefully acknowledge IsaacLab and LeRobot for their excellent work, from which we have borrowed some code.

Join Our Team! 💼

We're always looking for talented individuals passionate about AI and robotics! If you're interested in:

• 🤖 Robotics Engineering: Working with cutting-edge robotic systems and teleoperation
• 🧠 AI/ML Research: Developing next-generation AI models for robotics
• 💻 Software Engineering: Building robust, scalable robotics software
• 🔬 Research & Development: Pushing the boundaries of what's possible in robotics

Join us at Lightwheel AI! We offer:

• Competitive compensation and benefits
• Work with state-of-the-art robotics technology
• Collaborative, innovative environment
• Opportunity to shape the future of AI-powered robotics

Apply Now → | Contact Now → | Learn More About Us →

---

Let's build the future of robotics together! 🤝

---