More details about Inference time

[AditTuf]

Hi ,I see you believe that your algorithm can work at real time speeds ,based on your observations can you predict how much time this will take to stitch 4 ,1280x720 images into one Panorama , I plan to use RTX3070 GPU and hope to use this method for video stitching

[MungoMeng]

In our experiments, it usually takes less than 30s per iteration to merge the pre-processed 9000x4000 images when using a much older GPU. If you use RTX3070 to merge 1280x720 images, I think it should take less than 1s per iteration because your image is much smaller.
Then, the execution time depends on how many iterations are needed. If no more than 2 images overlap in the same area, you can merge all 4 images in one iteration (see ./Test_data_4Input for example). If three or more images overlap in the same area, you need to merge them iteratively (see ./Test_data for example); in this case, you might need up to 3 iterations to merge all 4 images.
Besides, you need to consider the time of pre-processing. This time depends on what parkage/algorthim you choose.

[snenyl]

If all cameras are fixed, is it possible to save a "stitched" configuration file and then only perform "inference" in real-time. If the cameras is moved relative to each other will the image be stitched wrong, and a new "stitched" configuration-file is required to be generated for the new extrinsics (camera poses).

[MungoMeng]

If all cameras are fixed, I think it's possible to save a configuration file for all position-specific operations, such as the first pre-processing stage (coarse registration). In the second image-blending stage, the global blending coefficient is calculated merely based on position, which could be saved as configuration. However, the optical flow is calculated based on photo pixels, which have to be computed every time.