Feat/turtle2 deploy v4

[MuggleZzzH]

Description

Motivation and Context

How has this been tested?

Additional information (optional, e.g., figures and logs):

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Documentation update (Document-only update)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests passed.

Summary by Sourcery

引入以部署为核心的 Turtle2 双臂环境与评估配置，并优化 Turtle2 平滑控制器，以更好地处理夹爪收敛问题。

New Features:

• 新增 Turtle2DeployEnv-v1 gym 环境，支持可配置的相对与绝对位姿动作模式，并提供显式的双臂夹爪观测。
• 引入 Turtle2DeployConfig 以及用于真实 Turtle2 部署和评估运行的 Hydra 配置。

Bug Fixes:

• 通过增加单独的夹爪容差与控制路径，确保在位姿收敛后，Turtle2 平滑控制器仍能持续发布夹爪目标。

Enhancements:

• 为不同动作模式注册 Turtle2 部署环境入口点，并配备合适的双臂位姿封装器（DualRelativeFrame 和 DualQuat2EulerWrapper）。

Original summary in English

Summary by Sourcery

Introduce a deploy-focused Turtle2 dual-arm environment and evaluation config, and refine the Turtle2 smooth controller to better handle gripper convergence.

New Features:

• Add Turtle2DeployEnv-v1 gym environment with configurable relative and absolute pose action modes and explicit dual-arm gripper observations.
• Introduce Turtle2DeployConfig and Hydra configs for real-world Turtle2 deployment and evaluation runs.

Bug Fixes:

• Ensure Turtle2 smooth controller continues publishing gripper targets after pose convergence by adding a separate gripper tolerance and control path.

Enhancements:

• Register the Turtle2 deploy env entry point with appropriate dual-arm pose wrappers (DualRelativeFrame and DualQuat2EulerWrapper) for different action modes.

[sourcery-ai]

审阅者指南（Reviewer's Guide）

添加了面向部署的 Turtle2 双臂环境（Gym 入口 + 配置），支持相对/绝对位姿控制模式，将其接入真实环境注册表，并修复 Turtle2 平滑控制器，使得在达到位姿容差后，夹爪指令仍能收敛。

Turtle2SmoothController 夹爪收敛逻辑的时序图

sequenceDiagram
    participant Turtle2SmoothController
    participant Turtle2State
    participant Turtle2Controller

    Turtle2SmoothController->>Turtle2State: read follow1_pos
    Turtle2SmoothController->>Turtle2State: read follow2_pos
    Turtle2SmoothController->>Turtle2SmoothController: smooth_action_callback(event)
    Turtle2SmoothController->>Turtle2SmoothController: compute pose_reached
    Turtle2SmoothController->>Turtle2SmoothController: compute err_gripper1, err_gripper2
    Turtle2SmoothController->>Turtle2SmoothController: compute gripper_reached

    alt pose_reached and not gripper_reached
        Turtle2SmoothController->>Turtle2Controller: arms_control(left_arm_target, right_arm_target)
        Turtle2SmoothController-->>Turtle2SmoothController: return (keep publishing gripper targets)
    else pose_reached and gripper_reached
        Turtle2SmoothController-->>Turtle2SmoothController: return (no further commands)
    else not pose_reached
        Turtle2SmoothController->>Turtle2SmoothController: interpolate xyz and rpy
        Turtle2SmoothController->>Turtle2Controller: arms_control(left_arm_target, right_arm_target)
    end

Loading

文件级变更（File-Level Changes）

变更	详情	文件
引入 Turtle2DeployEnv/Turtle2DeployConfig，用于双臂部署，支持相对/绝对位姿动作模式和夹爪观测通道，并提供仅用于部署的绝对位姿步进路径（零奖励）。	添加 Turtle2DeployConfig 数据类，继承 Turtle2RobotConfig，增加 action_mode 选择器、task_description、双臂默认值，以及与位姿相关字段的 ndarray 转换 实现 Turtle2DeployEnv，继承 Turtle2Env，扩展 observation_space 以包含 state.gripper，在 absolute_pose 模式下有条件地重写 action_space，并将 step() 分派到父类（relative_pose）或新的 _step_absolute_pose 路径 实现 _step_absolute_pose，对绝对基座坐标系目标进行裁剪、重塑并通过控制器 move_arm API（或虚拟状态）转发，加入安全边界裁剪和固定频率步进，总是返回 reward 0，并在达到 max_num_steps 时 truncated 重写 _calc_step_reward 使其始终返回 0，并暴露用于部署元数据的 task_description 属性	rlinf/envs/realworld/xsquare/tasks/turtle2_deploy_env.py
注册新的 Turtle2 部署 Gym 环境，并定义一个 wrapper 链，用于强制 action_mode/use_arm_ids，并适当地组合 DualRelativeFrame 和 DualQuat2EulerWrapper。	在 xsquare tasks 模块中导入 DualQuat2EulerWrapper、DualRelativeFrame、Turtle2DeployEnv 和 Turtle2DeployConfig 添加 create_turtle2_deploy_env 工厂函数，校验 override_cfg.action_mode 和 override_cfg.use_arm_ids，构造 Turtle2DeployEnv，在 relative_pose 时有条件地用 DualRelativeFrame 包装，并始终用 DualQuat2EulerWrapper 包装 使用 gymnasium 注册 Turtle2DeployEnv-v1，将 create_turtle2_deploy_env 作为入口点	rlinf/envs/realworld/xsquare/tasks/__init__.py
通过增加专门的夹爪容差，并在夹爪达到目标前持续发布保持姿态指令，使 Turtle2SmoothController 在位姿收敛后继续驱动夹爪目标。	引入类级别的 _GRIPPER_TARGET_TOLERANCE，将夹爪收敛与位姿容差解耦 在 smooth_action_callback 中，按臂计算夹爪与目标的误差，并推导出 pose_reached 和 gripper_reached 布尔量 修改提前返回条件，使其在 pose_reached 时触发；当位姿已到达但夹爪未到达时，发送 arms_control 指令以保持当前 xyz/rpy，同时继续命令目标夹爪宽度	rlinf/envs/realworld/xsquare/turtle2_smooth_controller.py
添加 Hydra 配置，以使用 HuggingFace 策略模型运行 Turtle2 部署评估，接入新环境、双臂观测/动作维度以及 Turtle2 硬件的集群布局。	创建 realworld_turtle2_deploy.yaml，定义 Turtle2DeployEnv-v1 的 env init_params，部署用 override_cfg（双臂、相机、action_mode、位姿限制、task_description），并添加关于为 state_dim 进行观测扁平化的说明 创建 realworld_turtle2_deploy_eval.yaml 驱动配置，组合 env/realworld_turtle2_deploy 用于训练/评估，配置 Turtle2 硬件的集群 node_groups，设置 runner/algorithm/rollout/actor 参数，并将模型的 state_dim/action_dim=14 与部署时的观测/动作约定对齐	examples/embodiment/config/env/realworld_turtle2_deploy.yaml examples/embodiment/config/realworld_turtle2_deploy_eval.yaml

---

技巧与命令（Tips and commands）

与 Sourcery 交互（Interacting with Sourcery）

• 触发新的审阅： 在拉取请求（pull request）上评论 @sourcery-ai review。
• 继续讨论： 直接回复 Sourcery 的审阅评论。
• 从审阅评论生成 GitHub issue： 回复某条审阅评论，请求 Sourcery 从该评论创建 issue。你也可以在某条审阅评论下回复 @sourcery-ai issue 来从中创建 issue。
• 生成拉取请求标题： 在拉取请求标题的任意位置写入 @sourcery-ai，可随时生成标题。你也可以在拉取请求中评论 @sourcery-ai title 以（重新）生成标题。
• 生成拉取请求摘要： 在拉取请求正文任意位置写入 @sourcery-ai summary，即可在相应位置生成 PR 摘要。你也可以在拉取请求中评论 @sourcery-ai summary 以（重新）生成摘要。
• 生成审阅者指南： 在拉取请求中评论 @sourcery-ai guide，可随时（重新）生成审阅者指南。
• 解决所有 Sourcery 评论： 在拉取请求中评论 @sourcery-ai resolve，以解决所有 Sourcery 评论。如果你已经处理了所有评论且不希望再看到它们，这会非常有用。
• 关闭所有 Sourcery 审阅： 在拉取请求中评论 @sourcery-ai dismiss，以关闭所有现有的 Sourcery 审阅。特别适合你想从一次全新的审阅开始时——别忘了再评论 @sourcery-ai review 触发新的审阅！

自定义你的体验（Customizing Your Experience）

打开你的 dashboard 来：

• 启用或禁用审阅特性，例如 Sourcery 自动生成的拉取请求摘要、审阅者指南等。
• 修改审阅语言。
• 添加、删除或编辑自定义审阅说明。
• 调整其他审阅设置。

获取帮助（Getting Help）

• 通过 联系支持团队 反馈问题或建议。
• 访问我们的 文档 获取详细指南和信息。
• 关注 Sourcery 团队在 X/Twitter、LinkedIn 或 GitHub 的账号以保持联系。

Original review guide in English

Reviewer's Guide

Adds a deploy-focused Turtle2 dual-arm environment (Gym entry + config) with relative/absolute pose control modes, wires it into the real-world env registry, and fixes the Turtle2 smooth controller so gripper commands converge even after pose tolerance is reached.

Sequence diagram for Turtle2SmoothController gripper convergence logic

sequenceDiagram
    participant Turtle2SmoothController
    participant Turtle2State
    participant Turtle2Controller

    Turtle2SmoothController->>Turtle2State: read follow1_pos
    Turtle2SmoothController->>Turtle2State: read follow2_pos
    Turtle2SmoothController->>Turtle2SmoothController: smooth_action_callback(event)
    Turtle2SmoothController->>Turtle2SmoothController: compute pose_reached
    Turtle2SmoothController->>Turtle2SmoothController: compute err_gripper1, err_gripper2
    Turtle2SmoothController->>Turtle2SmoothController: compute gripper_reached

    alt pose_reached and not gripper_reached
        Turtle2SmoothController->>Turtle2Controller: arms_control(left_arm_target, right_arm_target)
        Turtle2SmoothController-->>Turtle2SmoothController: return (keep publishing gripper targets)
    else pose_reached and gripper_reached
        Turtle2SmoothController-->>Turtle2SmoothController: return (no further commands)
    else not pose_reached
        Turtle2SmoothController->>Turtle2SmoothController: interpolate xyz and rpy
        Turtle2SmoothController->>Turtle2Controller: arms_control(left_arm_target, right_arm_target)
    end

Loading

File-Level Changes

Change	Details	Files
Introduce Turtle2DeployEnv/Turtle2DeployConfig for dual-arm deploy with relative/absolute pose action modes and a gripper observation channel, plus a deploy-only absolute pose stepping path with zero reward.	Add Turtle2DeployConfig dataclass extending Turtle2RobotConfig with action_mode selector, task_description, dual-arm defaults, and ndarray conversion for pose-related fields Implement Turtle2DeployEnv subclass of Turtle2Env that augments observation_space with state.gripper, conditionally overrides action_space for absolute_pose, and dispatches step() to either parent (relative_pose) or a new _step_absolute_pose path Implement _step_absolute_pose to clip, reshape and route absolute base-frame targets through the controller move_arm API (or dummy state) with safety-box clipping and fixed-frequency stepping, always returning reward 0 and truncated on max_num_steps Override _calc_step_reward to always return 0 and expose a task_description property used for deploy metadata	rlinf/envs/realworld/xsquare/tasks/turtle2_deploy_env.py
Register the new Turtle2 deploy Gym environment and define a wrapper chain that enforces action_mode/use_arm_ids and composes DualRelativeFrame and DualQuat2EulerWrapper appropriately.	Import DualQuat2EulerWrapper, DualRelativeFrame, Turtle2DeployEnv, and Turtle2DeployConfig into the xsquare tasks module Add create_turtle2_deploy_env factory that validates override_cfg.action_mode and override_cfg.use_arm_ids, constructs Turtle2DeployEnv, conditionally wraps it with DualRelativeFrame (relative_pose) and always with DualQuat2EulerWrapper Register Turtle2DeployEnv-v1 with gymnasium using create_turtle2_deploy_env as the entry point	rlinf/envs/realworld/xsquare/tasks/__init__.py
Make Turtle2SmoothController continue to drive gripper targets after pose convergence by adding a dedicated gripper tolerance and publishing hold-pose commands until grippers reach their goals.	Introduce a class-level _GRIPPER_TARGET_TOLERANCE to decouple gripper convergence from pose tolerances Within smooth_action_callback, compute per-arm gripper errors vs targets and derive pose_reached and gripper_reached booleans Modify early-return condition to trigger on pose_reached and, when pose is reached but grippers are not, send arms_control commands that hold current xyz/rpy while continuing to command the target gripper widths	rlinf/envs/realworld/xsquare/turtle2_smooth_controller.py
Add Hydra configs to run Turtle2 deploy evaluation with a HuggingFace policy model, wiring in the new env, dual-arm obs/action dimensions, and cluster layout for Turtle2 hardware.	Create realworld_turtle2_deploy.yaml defining env init_params for Turtle2DeployEnv-v1, override_cfg for deploy (dual arms, cameras, action_mode, pose limits, task_description), and notes on obs flattening for state_dim Create realworld_turtle2_deploy_eval.yaml driver config that composes env/realworld_turtle2_deploy for train/eval, configures cluster node_groups for Turtle2 hardware, sets runner/algorithm/rollout/actor parameters, and aligns model state_dim/action_dim=14 with the deploy obs/action contract	examples/embodiment/config/env/realworld_turtle2_deploy.yaml examples/embodiment/config/realworld_turtle2_deploy_eval.yaml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.