DEEP-IMAGE-MATCHING

SIFT	DISK	DISK

Multivew matcher for COLMAP. Support both deep-learning based and hand-crafted local features and matchers and export keypoints and matches directly in a COLMAP database or to Agisoft Metashape by importing the reconstruction in Bundler format. It supports both CLI and GUI. Feel free to collaborate!

Key features:

• Multiview
• Large format images
• SOTA deep-learning and hand-crafted features
• Full compatibility with COLMAP
• Support for image rotations
• Compatibility with Agisoft Metashape (only on Linux and MacOS by using pycolmap)
• Support image retrieval with deep-learning local features

Supported extractors:

• SuperPoint
• DISK
• ALIKE
• ALIKED
• Superpoint free
• KeyNet + OriNet + HardNet8
• ORB (opencv)
• SIFT (opencv)

Matchers:

• Lightglue (with Superpoint, Disk and ALIKED)
• SuperGlue (with Superpoint)
• LoFTR
• Nearest neighbor (with KORNIA Descriptor Matcher)
• GlueStick
• RoMa

Install and run

Install in a conda environment:

conda create -n deep-image-matching python=3.10
conda activate deep-image-matching

Install pytorch. See https://pytorch.org/get-started/locally/#linux-pip

python -m pip install --upgrade pip
pip install -e .

Install hloc (https://github.com/cvg/Hierarchical-Localization/tree/master):

git clone --recursive https://github.com/cvg/Hierarchical-Localization/
cd Hierarchical-Localization/
python -m pip install -e .
git submodule update --init --recursive

Example usage

Sequential matching with LighGlue

Before running check options with python ./main.py --help, then:

python ./main.py  --config superpoint+lightglue --images assets/example_images --outs assets/output --strategy sequential --overlap 1

See other examples in run.bat. If you want to customize detector and descpritor options, change default options in config.py.

To run with the GUI:

python ./main.py --gui

Merging databases with different local features

See scripts in the ./scripts dir

TODO:

• Tile processing for high resolution images
• Manage image rotations
• Add image retrieval with global descriptors
• add GUI
• Add pycolmap compatibility
• Add exporting to Bundler format ready for importing into Metashape (only on Linux and MacOS by using pycolmap)
• Add visualization for extracted features and matches
• Improve speed
• Autoselect tiling grid in order to fit images in GPU memory
• Add tests, documentation and examples
• Apply mask during feature extraction
• Check scripts

References

If you find the repository useful for your work consider citing the papers:

@article{morelli2022photogrammetry,
  title={PHOTOGRAMMETRY NOW AND THEN--FROM HAND-CRAFTED TO DEEP-LEARNING TIE POINTS--},
  author={Morelli, Luca and Bellavia, Fabio and Menna, Fabio and Remondino, Fabio},
  journal={The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
  volume={48},
  pages={163--170},
  year={2022},
  publisher={Copernicus GmbH}
}

@article{ioli2023replicable,
  title={A Replicable Open-Source Multi-Camera System for Low-Cost 4d Glacier Monitoring},
  author={Ioli, F and Bruno, E and Calzolari, D and Galbiati, M and Mannocchi, A and Manzoni, P and Martini, M and Bianchi, A and Cina, A and De Michele, C and others},
  journal={The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
  volume={48},
  pages={137--144},
  year={2023},
  publisher={Copernicus GmbH}
}

Depending on the options used, consider citing the corresponding work of KORNIA, HLOC, and local features.