Mining Cross-Modality Implicit Semantic Association for Unsupervised Visible-Infrared Person Re-Identification

This is a code implementation and supplement to Mining Cross-Modality Implicit Semantic Association for Unsupervised Visible-Infrared Person Re-Identification, which was accepted by TIFS.

Contributions

1. We propose a novel Mining Cross-Modality Implicit Semantic Association (MCSA) framework, which enhances robust feature representation by enabling the construction of a modality-invariant semantic space. This framework effectively integrates semantic and visual features to address and mitigate cross-modality discrepancies.
2. We design a simple yet effective Modality-invariant Prompt Learning module to generate modality-invariant implicit semantic information. The semantic features that emerge from this process demonstrate enhanced modality generalization capabilities, thus providing valuable insights for semantic supervision.
3. We propose a GCN-driven Collaborative Alignment module that employs Graph Convolutional Networks (GCNs) to propagate similarity relationships across image and semantic graphs. This enhances the integration of multimodal information, enabling the construction of more robust cross-modality correspondences while effectively mitigating intra-modality and inter-modality variations.
4. Extensive experiments conducted on the SYSU-MM01 and RegDB datasets demonstrate the effectiveness of our proposed MCSA framework, improving unsupervised cross-modality retrieval performance.

Prepare Datasets

Put SYSU-MM01 and RegDB dataset (run prepare_sysu.py and prepare_regdb.py to convert to market1501 format) into data/sysu and data/regdb. (Following previous work ADCA)

Prepare Pre-trained model

Following SDCL, we adopt the self-supervised pre-trained models (ViT-B/16+ICS) from Self-Supervised Pre-Training for Transformer-Based Person Re-Identification. Download the model to the examples folder

Installation

git clone https://github.com/liulekai123/MCSA.git

cd yourproject

conda env create -f environment.yml

conda activate yourproject-env

Training

sh run_train_sysu.sh
sh run_train_regdb.sh

Models

Datestes	Rank-1	mAP	Download
SYSU-MM01 (All Search)	66.38%	64.15%	model
RegDB (Visible to Infrared)	91.87%	85.37%	model

Acknowledgment

Our implementation is mainly based on the following codebases. We gratefully thank the authors for their wonderful works.

TokenMatcher. SDCL. ADCA. TransReID. DC-former. CALR