Calibration information incomplete

[anja-sheppard]

Hi there,

I was excited to use this dataset but I am confused by the calibration information provided. It appears you are just providing the factory ZED calibration instead of actually running a stereo calibration routine. This is troubling as these calibrations are very important for doing accurate localization.

Additionally, the ZED calibration files provided give a principal point of around (968, 591), but the actual images provided in the dataset are only 960 x 540 px. Did you downsample the images? If so, by how much?

Finally, are there any stereo to IMU extrinsic calibration details provided? All I see is a static_transform file but it just contains the following, which can't be right:

0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.000000000000000000e+00

This is really deeply crucial information for using this dataset.

Thanks!

[Parv-Maheshwari]

Thank you for showing interest in our dataset.

• We did not perform stereo calibration on zedx due to time constraints. Moreover, since the dataset is more targeted towards increasing paired RGBT images rather than accurate localization (since there is no odometry GT), we feel the factory calibrations are good enough fo rthis purposes

• Thank you for pointing this out. The original resolution is 1920 x 1080, and we downsample by 2.

• The imu extrinsics are now updated - https://huggingface.co/datasets/theairlabcmu/TartanRGBT/blob/main/calibration/zed_imu_extrinsics.json

Please let me know if I can help with anything else

[anja-sheppard]

Hi @Parv-Maheshwari --

Thanks so much! One follow up--where is the IMU data? I don't see an IMU folder under the sequence directories. I could just be missing it, so I'd appreciate a pointer to where to find the 30 Hz IMU data that the website says was captured for each sequence.

[Parv-Maheshwari]

Thank you for pointing this out. We have not released the IMU data yet. I will try to release it over the weekend. Will let you know whenever it is released.

[anja-sheppard]

Hi @Parv-Maheshwari -- were you able to release the IMU data? Thanks!

[Parv-Maheshwari]

Hi @anja-sheppard — thanks for the detailed report, and sorry for the delay. You were right on all three counts. We've just pushed a corrected, expanded release of theairlabcmu/TartanRGBT that fixes the calibration/TF packaging and adds the missing data. Point by point:

1. Downsampling factor / principal point mismatch

Yes — the released images are downsampled 2× from the ZED's factory rectified output: 1920×1080 → 960×540. That's exactly why the factory principal point (~968, 592) didn't line up with the images. The intrinsics must be halved to match the shipped frames:

	fx	fy	cx	cy
Factory (1920×1080)	738.862	738.811	968.505	591.855
Released (960×540)	369.431	369.406	484.253	295.993

We now ship these explicitly as calibration/released_zed_rectified_960x540.yaml so there's no ambiguity — use that file with the zed_*_rect, stereo_depth, and rgb_in_thermal data. (Metric depth = fx · baseline / disparity, with fx = 369.431, baseline = 0.120022 m.)

2. Factory vs. actual calibration

To clarify the intent: the ZED stereo images are rectified by the ZED SDK using the factory calibration, so for the rectified frames we distribute, the (resolution-scaled) factory rectified intrinsics above are the correct model to use — not a placeholder. What was genuinely missing/incorrect before was the inter-sensor geometry (stereo↔IMU, ZED↔thermal), which we've now shipped properly (below).

3. Stereo→IMU extrinsics (the all-zeros transform)

That identity/zero static_transform was a packaging bug in the previous release — the real transforms weren't being written out. Fixed now. The full static TF tree is shipped as a single calibration/tf_static.json (constant across sequences), including the real values:

• ZED left → ZED right (stereo): translation ≈ [0.120022, 0, 0] m (the ~120 mm baseline)
• ZED left camera frame → ZED IMU: translation [0, -0.035649, -0.000147] m, quaternion (xyzw) [0.003440, -0.000689, 0.000857, 0.999993]
• ZED → thermal-left and thermal-left → thermal-right extrinsics

The per-camera calibration files (zed.yaml, thermal_left.yaml, thermal_right.yaml, cross_camera_extrinsics.yaml, zed_imu_extrinsics.json) are all in calibration/.

Also new in this release

• Raw IMU data is now released — zed_imu_raw.zip per sequence (~100 Hz native rate: angular velocity, linear acceleration, and orientation).
• 10 Hz data (up from 1 Hz) across all modalities.
• Added 16-bit left thermal (thermal_left_rect_16), and split odometry into its own odometry.zip.
• Every image modality now carries its own timestamps.txt / errors.txt sync-quality sidecars, and each sequence has a target_timestamps.txt master clock.

Please pull the updated dataset and let us know if anything still looks off — happy to help.