20260302: voxblox_msgs 拿到一个公共的仓库中维护（https://gitee.com/zeng_xiangwei/vln_msgs）

ros2

changelog:

1. config_utilities 改造为从 YAML::Node 中读取参数，不再依赖 xmlrpcpp
2. 将 voxblox 相关内容合并到 panoptic_mapping 中
3. 移除对 catkin_simple、minkindr_ros 等模块的依赖
4. 使用本机已安装的 opencv、protobuf

env：

• ubuntu 22.04 + ros2 humble

dependencies:

1. OpenCV4、PCL (安装 ros2 时会默认安装)
2. Eigen3
3. Protobuf (libprotoc 3.4.0， 如果有 anaconda，需要注意 cmake 时使用的是哪个 protobuf)
4. glog、gflags
5. yaml-cpp

ros1 迁移到 ros2 的注意项：

1. 时间系统，rclcpp::Time 内部有三种时钟，ros、系统、steady，不同种类的时间之间不能比较，目前对于 ros::Time 都用节点中的时钟来获取 node_ptr->get_clock()->now()，voxblox_rviz_plugin 中用 context_->getClock()->now()。
2. ament_cmake 默认是编译为静态库，由于编译为静态库时 panoptic_mapping_ros 中的工厂模式中所需的类都没有注册，因此都编译为动态库，且需要显示添加编译选项为 Release。
3. voxblox_rviz_plugin 使用 ogre 时，在头文件的位置不要写如 <OGRE/OgreSceneManager.h>，写成 <OgreSceneManager.h>，否则可能会找到系统中的 OGRE 库，而不是 ros2 中的 OGRE 库。
4. voxblox_rviz_plugin 中使用自定义显示材料时，在使用时要加上组名，否则找不到，在以下位置增加材料注册时的组名

ogre_object->begin(material_name, Ogre::RenderOperation::OT_TRIANGLE_LIST,
                       "VoxbloxMaterials");

实采测试数据

通过网盘分享的文件：self_test_data.zip
链接: https://pan.baidu.com/s/1VA8H-urXWkGnPfprMhM4Fw?pwd=w4im 提取码: w4im 复制这段内容后打开百度网盘手机App，操作更方便哦

运行 需要修改 yaml 配置参数中的 labels.file_name 为实际的类别文件路径

ros2 launch panoptic_mapping_ros run_self_data_online.launch.py

Panoptic Mapping

This package contains panoptic_mapping, a general framework for semantic volumetric mapping. We provide, among other, a submap-based approach that leverages panoptic scene understanding towards adaptive spatio-temporally consistent volumetric mapping, as well as regular, monolithic semantic mapping.

Multi-resolution 3D Reconstruction, active and inactive panoptic submaps for temporal consistency, online change detection, and more.

Table of Contents

Credits

• Paper
• Video

Setup

• Installation
• Datasets

Examples

• ros2
• Panoptic Mapping
• Table of Contents
• Paper
• Video
• Installation
• Datasets
• Examples
  • Running the Panoptic Mapper
  • Monolithic Semantic Mapping
  • Running the RIO Dataset
• Contributing

Other

• Contributing

Paper

If you find this package useful for your research, please consider citing our paper:

• Lukas Schmid, Jeffrey Delmerico, Johannes Schönberger, Juan Nieto, Marc Pollefeys, Roland Siegwart, and Cesar Cadena. "Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency" in IEEE International Conference on Robotics and Automation (ICRA), pp. 8018-8024, 2022. [ IEEE | ArXiv | Video ]

  @inproceedings{schmid2022panoptic,
    title={Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency},
    author={Schmid, Lukas and Delmerico, Jeffrey and Sch{\"o}nberger, Johannes and Nieto, Juan and Pollefeys, Marc and Siegwart, Roland and Cadena, Cesar},
    booktitle={2022 IEEE International Conference on Robotics and Automation (ICRA)},
    year={2022},
    volume={},
    number={},
    pages={8018-8024},
    doi={10.1109/ICRA46639.2022.9811877}}
  }

Video

For a short overview explaining the approach check out our video on youtube:

Installation

Instructions for different installation options. The repository was developed and tested on Ubuntu 18.04 with ROS melodic and Ubuntu 20.04 with ROS noetic.

Docker To run the panoptic mapper without installation, just use the docker image:

docker pull ghcr.io/ethz-asl/panoptic_mapping:main
docker run -it ghcr.io/ethz-asl/panoptic_mapping:main bash

VSCode For development with vscode, you can just clone this repository and follow the prompts to open it in a devcontainer.

System Installation To install on linux, follow the instructions below matching your OS and ROS version:

Ubuntu 18.04 + ROS Melodic.

Prerequisites

1. If not already done so, install ROS (Desktop-Full is recommended).

2. If not already done so, create a catkin workspace with catkin tools:

sudo apt-get install python-catkin-tools
mkdir -p ~/catkin_ws/src
cd ~/catkin_ws
catkin init
catkin config --extend /opt/ros/melodic
catkin config --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo
catkin config --merge-devel

Installation

1. Install system dependencies:

sudo apt-get install python-wstool python-catkin-tools autoconf libtool git

2. Move to your catkin workspace:

cd ~/catkin_ws/src

3. Download repo using SSH or HTTPS:

git clone git@github.com:ethz-asl/panoptic_mapping.git  # SSH
git clone https://github.com/ethz-asl/panoptic_mapping.git  # HTTPS

4. Download and install package dependencies using ros install:

• If you created a new workspace.

wstool init . ./panoptic_mapping/panoptic_mapping_ssh.rosinstall    # SSH
wstool init . ./panoptic_mapping/panoptic_mapping_https.rosinstall  # HTTPS
wstool update

• If you use an existing workspace. Notice that some dependencies require specific branches that will be checked out.

wstool merge -t . ./panoptic_mapping/panoptic_mapping.rosinstall
wstool update

5. Compile and source:

catkin build panoptic_mapping_utils
source ../devel/setup.bash

Ubuntu 20.04 + ROS Noetic.

Prerequisites

1. If not already done so, install ROS (Desktop-Full is recommended).

2. If not already done so, create a catkin workspace with catkin tools:

sudo apt-get install python3-catkin-tools
mkdir -p ~/catkin_ws/src
cd ~/catkin_ws
catkin init
catkin config --extend /opt/ros/noetic
catkin config --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo
catkin config --merge-devel

Installation

1. Install system dependencies:

sudo apt-get install python3-pip python3-wstool python3-catkin-tools autoconf libtool git
pip3 install osrf-pycommon

2. Move to your catkin workspace:

cd ~/catkin_ws/src

3. Download repo using SSH or HTTPS:

git clone git@github.com:ethz-asl/panoptic_mapping.git  # SSH
git clone https://github.com/ethz-asl/panoptic_mapping.git  # HTTPS

4. Download and install package dependencies using ros install:

• If you created a new workspace.

wstool init . ./panoptic_mapping/panoptic_mapping_ssh.rosinstall    # SSH
wstool init . ./panoptic_mapping/panoptic_mapping_https.rosinstall  # HTTPS
wstool update

• If you use an existing workspace. Notice that some dependencies require specific branches that will be checked out.

wstool merge -t . ./panoptic_mapping/panoptic_mapping.rosinstall
wstool update

5. Compile and source:

catkin build panoptic_mapping_utils
source ../devel/setup.bash

Datasets

The datasets described in the paper and used for the demo can be downloaded from the ASL Datasets.

A utility script is provided to directly download the flat data:

roscd panoptic_mapping_utils
export FLAT_DATA_DIR="/home/$USER/Documents"  # Or whichever path you prefer.
chmod +x panoptic_mapping_utils/scripts/download_flat_dataset.sh
./panoptic_mapping_utils/scripts/download_flat_dataset.sh

To run the RIO demos, the original dataset needs to be downloaded and augmented with our supplementary data. Instructions on which scenes to download and how to combine them are found on our dataset page.

Examples

Running the Panoptic Mapper

This example explains how to run the Panoptic Multi-TSDF mapper on the flat dataset.

1. First, download the flat dataset:

   export FLAT_DATA_DIR="/home/$USER/Documents"  # Or whichever path you prefer.
   chmod +x panoptic_mapping_utils/scripts/download_flat_dataset.sh
   ./panoptic_mapping_utils/scripts/download_flat_dataset.sh
   

2. Replace the data base_path in launch/run.launch (L10) and file_name in config/mapper/flat_groundtruth.yaml (L15) to the downloaded path.

3. Run the mapper:

   roslaunch panoptic_mapping_ros run.launch
   

4. You should now see the map being incrementally built:

   

5. After the map finished building, you can save the map:

   rosservice call /panoptic_mapper/save_map "file_path: '/path/to/run1.panmap'"
   

6. Terminate the mapper pressing Ctrl+C. You can continue the experiment on run2 of the flat dataset by changing the base_path-ending in launch/run.launch (L10) to run2, and load_map and load_path in launch/run.launch (L26-27) to true and /path/to/run1.panmap, respectively. Optionally, you can also change the color_mode in config/mapper/flat_groundtruth.yaml (L118) to change to better highlight the change detection at work.

   roslaunch panoptic_mapping_ros run.launch
   

7. You should now see the map being updated based on the first run:

   

Monolithic Semantic Mapping

Panoptic Mapping supports also the monolithic use case. This example explains how to run the Panoptic Single-TSDF mapper on the flat dataset.

1. If not already done so, download the flat dataset:

   export FLAT_DATA_DIR="/home/$USER/Documents"  # Or whichever path you prefer.
   chmod +x panoptic_mapping_utils/scripts/download_flat_dataset.sh
   ./panoptic_mapping_utils/scripts/download_flat_dataset.sh
   

2. Replace the data base_path in launch/run.launch (L10) and file_name in config/mapper/single_tsdf.yaml (L15) to the downloaded path.

3. To use the single-TSDF mapper and real segmentation predictions, set use_detectron in launch/run.launch (L6) to true and config in launch/run.launch (L22) to 'single_tsdf'.

4. Run the mapper:

   roslaunch panoptic_mapping_ros run.launch
   

5. You should now see the map being incrementally built.

6. Varying visualization modes are supported, the classes (default) will color the mesh according to the predicted semantic class. Other modes, such as classification will show the confidence of the aggregated predictions.

Predicted classes (left) and corresponding fusion confidence (right, low to high in red to green).

Running the RIO Dataset

This example explains how to run the Panoptic Multi-TSDF mapper on the RIO dataset.

1. First, download the original dataset and our supplementary data.

2. Set use_rio in launch/run.launch (L5) to true, replace the data_path in (L16) and file_name in config/mapper/rio_groundtruth.yaml (L15) to the downloaded path.

3. Run the mapper:

   roslaunch panoptic_mapping_ros run.launch
   

4. You should now see the map being incrementally built:

   

   (Left in reconstructed color, right colored by submap)

Contributing

panoptic_mapping is an open-source project, any contributions are welcome!

For issues, bugs, or suggestions, please open a GitHub Issue.

To add to this repository:

• Please employ the feature-branch workflow.
• Setup our auto-formatter for coherent style (we follow the google style guide):

  # Download the linter
  cd <linter_dest>
  git clone git@github.com:ethz-asl/linter.git -b feature/noetic
  cd linter
  echo ". $(realpath setup_linter.sh)" >> ~/.bashrc
  bash
  roscd panoptic_mapping/..
  init_linter_git_hooks
  # You're all set to go!
  

• Please open a Pull Request for your changes.
• Thank you for contributing!