Loss of precision in external calibration import (MayaCam & CSVs)

[pfalkingham]

Description
When a known-good calibration is exported (e.g., as a MayaCam 2.0 file) and then reloaded using the "External Calibration" feature, the calibration loses accuracy. Visually, the epipolar lines no longer align as they did in the original project.

Inspecting the imported values reveals that they begin to diverge from the original values around the 5th–7th decimal place and gain extra trailing garbage decimals. Manually copying values from the internal project CSVs into the external calibration UI also fails to reproduce the exact original calibration, suggesting precision loss during both import and export.

Steps to Reproduce

1. Open a working project with a high-quality calibration.
2. Export the camera calibration using the MayaCam 2.0 export.
3. Create a new dataset using "External Calibration" and load the exported MayaCam file.
4. Compare the resulting epipolar lines and raw calibration values to the original project.

Root Cause Analysis
There are two distinct precision issues causing this bug:

1. MayaCam Import Downcasts to 32-bit float:
   In src/ui/ExternalCalibrationFrame.cpp, the on_pushButton_MayaCam_clicked() function parses the text file using fscanf into 32-bit float variables (which only hold ~7 decimal digits of precision).

// src/ui/ExternalCalibrationFrame.cpp ~Line 198
float C11,C12,C13,C21,C22,C23,C31,C32,C33;
fscanf(pfile, "%f,%f,%f\n", &C11, &C12, &C13);

When these 32-bit floats are passed into the UI's QDoubleSpinBoxes (which expect 64-bit double), the missing precision is populated with trailing garbage decimals. This is the primary reason the loaded calibration values diverge heavily at the 5th-7th decimal place.

2. Calibration Exports Truncate to 12 Decimals:
   A 64-bit double in memory provides 15–17 significant decimal digits. However, the export functions currently cap the output stream precision to 12 when writing to text/CSV files.

// e.g. src/core/Camera.cpp (saveMayaCamVersion2, saveCameraMatrix)
std::ofstream outfile(filename.toStdString());
outfile.precision(12);

While 12 decimals are usually enough, in highly sensitive non-linear multi-camera optimizations, stripping those last few digits from the raw double can be enough to slightly misalign the epipolar geometry when the values are re-imported via the workaround method.

Proposed Fixes

1. Fix the Import (Critical): In src/ui/ExternalCalibrationFrame.cpp, update on_pushButton_MayaCam_clicked() to read into double variables instead of float.
   • Change float declarations to double.
   • Change all fscanf format specifiers from %f to %lf.
2. Fix the Export (Recommended): In src/core/Camera.cpp and src/core/CalibrationImage.cpp, locate all instances where outfile.precision(12); is called and increase the precision to 17 (e.g., outfile.precision(17); or outfile.precision(std::numeric_limits<double>::max_digits10);) to ensure lossless round-tripping of 64-bit doubles.

[pfalkingham]

This turned out to not actually be due to precision (though that's been improved now).

The problem is the Maya cams don't include anything about distortion. So your epipolar lines are miles off on your distorted images. If you want to create an xmalab file with an external calibration, it's only going to work if you load the undistorted trial images.

This is by design - in Maya/Blender you load the undistorted images, so those cameras don't need any distortion information.

A potential fix for me to implement is to:

Export a separate companion .dist file alongside the MayaCam — a small text file with just the 8 coefficients, and add a "Load Distortion" button in the External Calibration dialog. We could even have 'export distortions' as a check box on camera export, because I don't think anyone has ever needed this and spamming .dist files by default would inconvenience most users.

I'm reluctant to add the distortions to the maya cam file format and hope maya/blender ignore them, because that could break something fairly pivotal.

I am open to comments from the community about this.