Skip to content

PREPROCESSING.md

Latest commit

 

History

History
122 lines (94 loc) · 6.74 KB

PREPROCESSING.md

File metadata and controls

122 lines (94 loc) · 6.74 KB
Raw

Data Preprocess

This document will help you processing the data recorded during the experiments of the GRADE dataset.

There are many options.

The first choice you need to make is if you want to process the rosbags or the full size data (the stored npy files). In the first case go to ROSBAG Processing, in the latter go to File Processing.

In practice, given an input folder, the script can:

  • select the correct section of the rosbags (after the starting_experiment signal)
  • correct the topic times of the bags to account for possible wrong clock times due to isaac_sim known issues
  • add noise to the IMU and depths
  • limit the recorded depth to a maximum distance
  • add motion blur noise and rolling shutter effect to the RGB Images
  • extract RGB and depth PNG images
  • extract odom and imu in npy files
  • ...

The process_data.sh will take care of:

  • reindexing the bag (if there is a .bag.active)
  • start the correct processing script (located in src/[file,bag]_process/play_[file,bags].py)

NOTE: WE DO NOT ADD NOISE TO THE ODOM. However, it would be trivial to do so given the current setup. Adding white noise to the twist vector is in general considered sufficient, with an integration step to get the pose/orientation.

Procedure Overview

  • In Bag Processing, time correction procedure will create /reindex_bags folder with reindex bags in your input folder

  • In Bag Processing, add noise procedure will create the /noisy_bags folder with noisy bags in your input folder. The folder structure will be as follows:

      DATA_FOLDER/
    ├── *.bag                           # raw rosbags
    ├── reindex_bags/                   # generated by time correction
    └── noisy_bags/                     # generated by adding noise

  • Data Extraction from rosbags is necessary for file processing, since IMU data and odom data are required when generating motion blur noise on full size RGB images

  • Data Extraction procedure also generates depth and rgb images (640X480) from rosbags, which can be fed into SLAM frameworks efficiently

  • Data Extraction procedure takes a folder as input and extract data from rosbags to the output directory as follows:

    bag_folder/
  ├── *.bag                           # processed bags from bag_processing
  └── data/                           # generated by data extraction
       ├─── depth[id]/                # depth img in *.png
       ├─── imu_body/                 # imu-body data in *.npy
       ├─── imu_camera/               # imu-camera data in *.npy
       ├─── odom/                     # odom data in *.npy
       └─── rgb[id]/                  # rgb img in *.png

  • In File Processing procedure, noise will be added to full size data including rgb, depth and imu data

  • File Processing procedure will take depthLinear/rgb/camera data from Viewport folder [you can change this in preprocessing/src/file_process/add_noise.py] and raw imu/odom data from extracted data as inputs. The output folder structure is as follows:

    Viewport0/
  ├── (rgb, depthlinear, ...)         # original data
  └── data/                           # generated by file processing
       ├─── depth/                    # [optional] depth npy/img
       ├─── imu/                      # [optional] imu data in *.npy
       ├─── odom/                     # [optional] odom data in *.npy
       └─── rgb/                      # [optional] rgb img in *.png
  └── data_noisy/                     # same structure but noisy. Clearly data is being saved here for sure.

1. ROSBAG Processing

./process_data.sh -t bag -p [PATH_TO_YOUR_DATA]
  • Customized parameters are defined in config/bag_process.yaml. Please refer to that for the complete set of parameters.
    • If message timestamps are overlapped, set time_correction/enable to True to generate reindex bags.
    • Set noise to True to generate Noisy Bags
    • Set camera/pointcloud to True to generate Pointcloud Message in Noisy Bags
    • Set blur/enable to True to generate Blurry Image Message in Noisy Bags
    • Set blur/save/enable to True to save Blur Parameter Files for each image, which can be used to generate corresponding transformed masks for blurry images
  • Reindex Bags will be saved in /reindex_bags folder
  • Noisy Bags will be saved in /noisy_bags folder

2. Data Extraction

./process_data.sh -t extract -p [PATH_TO_YOUR_BAG_DATA]
  • Data extraction should be performed after processing rosbags and before processing files
  • Customized parameters are defined in config/extract_data.yaml
    • Set out_dir to your desired output directory, or it will be saved in /{INPUT_PATH}/data
    • Each topic row is composed of three elements. [topic_name, out_subfolder, True/False]. If False it won't be saved.
    • The imu_reference_topic is used to link the imu messages to the images. The output names will have a structure like img_idx_imu_idx2.npy to distinguish imu intercoming between two images.
  • NOTE that most SLAM methods require rgb folder, so you'll need to manually rename it or change the config of the SLAM.

3. File Processing

./process_data.sh -t file -p [PATH_TO_YOUR_DATA]/Viewport
  • /rgb, /depthLinear, and /camera data are required in /Viewport folder
  • Customized parameters are defined in config/file_process.yaml
    • Set raw_data_dir to the extracted data folder from Data Extraction procedure. Since we are going to add noise to the full images using this data, this will be the reindexed_bag data folder.
    • Set out_dir to your desired output directory, or it will be saved in /{INPUT_PATH}/data
    • Set camera/enable to True to generate Noisy Depth files in *.npy files (1920X1080)
    • Set camera/output_img to True to generate Original and Noisy Depth Images in *.png files (1920X1080)
    • Set blur/enable to True to generate Blurry RGB Images in *.png files (1920X1080)
    • Set imu/enable to True to generate Noisy IMU Data in *.npy files
    • Set replicate to True to copy old data in the new output directory (including rgb images, depth npy files, imu data, and odom). This applies as output in the data folder, since IMU, depth, rgb will have the noisy version in the data_noisy.

Example:

./process_data.sh -t file -p ~/exp/Viewport0/