复现并且对比CLIP前后的性能差异 [CVPR2025] CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Dataset

Overview

The overall framework of our model is as follows：

Getting Started

Installation

1. Install requirements

Install requirements using pip:

pip install -r requirements.txt

2. Prepare dataset

We follow R2Gen dataset process to download IU-Xray and MIMIC-CXR, you can download orin MIMIC-CXR dataset from official website

For CheXpert Plus dataset: Download the original CheXpert Plus dataset from here. You can download our preprocess annotation file from here.

The data set we used in the first stage of training was an undisclosed private data set, so I can't provide it here. However, you can use other medical X-ray data for training. In the second stage we use a collection of three public data sets IU-Xray, Mimic-CXR, and CheXpert-plus. You can download the three raw data and then use the annotation file we provide. The third stage of downstream task fine-tuning uses separate, normal three public data sets.

Using cheXpert_plus as an example, the file format of the data set is as follows:

cheXpert_plus 
├── chexbert_labels
├── df_chexpert_plus_240401.csv
├── PNGs
├── annotation.json    <- place annotation.json here

Training

1. Pretrain Stage

You need to change '--data_path' in pretrain/pretrain.sh to your own data set configuration file. Then run the command:

bash pretrain/pretrain.sh

2. MambaXrayVLCLIP Stage

At this stage, you first need to download the annotation file. Note that the information in this file is used in absolute path, so you need to modify the path corresponding to the three datasets you downloaded before using it. And you need to change '--annotation' in launch/launch_mambaclip.sh. Then run the command:

bash launch/launch_mambaclip.sh

3. DownStream Stage

For CheXpert_plus as example, update '--annotation' and '--base_dir' in launch/launch_mambaclip_chexpert.sh to your data path.

For IU-xray:

bash launch/launch_mambaclip_iu.sh

For MIMIC-CXR:

bash launch/launch_mambaclip_mimic.sh

For CheXpert Plus:

bash launch/launch_mambaclip_chexpert.sh

Testing (For CheXpert_plus as example)

Once you finished the training, update '--annotation' , '--base_dir' and 'load_model' in launch/launch_mambaclip_test_chexpert.sh to your data path, then you can test the model by running the following method:

bash launch/launch_mambaclip_test_chexpert.sh

Results and Checkpoints

name	stage	#params	model
Pretrain-B	1	Base	model
Pretrain-L	1	Large	model
MambaXrayCLIP-B	2	Base	model
MambaXrayCLIP-L	2	Large	model
IU-Finetune-B	3	Base	model
IU-Finetune-L	3	Large	model
Mimic-Finetune-B	3	Base	model
Mimic-Finetune-L	3	Large	model
CheXpert-Finetune-B	3	Base	model
CheXpert-Finetune-L	3	Large	model

Acknowledgement

• CXPMRG-Bench: Pre-training and Benchmarking for X-ray Medical Report Generation on CheXpert Plus Dataset Our work is based on the MambaXray_VL framework.