SAMIX: Reinforcing SAM2 with Semantic Adapter and Reference Selecting Policy for Mix-Supervised Segmentation (CVPR 2026)

📖 Paper

Qiang Hu¹ · Jiajie Wei¹ · Zhenyu Yi² · Zhifen Yan¹ · Yingjie Guo¹ · Hongkuan Shi³ · Ge-Peng Ji⁴ · Qiang Li¹ · Zhiwei Wang¹

¹HUST   ²SJTU   ³Wuhan United Imaging Surgical Healthcare Co., Ltd.  ⁴ANU  

🚀Overview

Problem Formulation: Mix-supervised Segmentation aims to train a single segmentation model using heterogeneous-annotated data, including mask, box, scribble, point, class-labled, and unlabled data.

Paradigm Comparison:

• Exsing SAM-based methods (1) heavily rely on sparse spatial prompts (e.g., box, point); (2) can not address scenarios with ambiguous boundaries; (3) can not use class-labeled and unlabeled data for training; (4) overlooks the potential of collaborative learning across heterogeneous data.

• Ours (1) repurpose the SAM2's instance tracking mechanism to promote semantic tracking across data, i.e., in-context segmentation; (2) introduce a RL-empowered Network to actively select in-context examples for each query; (3) can use class-labeled and unlabeled data for training; (4) achieve collaborative learning across heterogeneous data.

⚙️ Environment Setup

1. Create environment

Recommended baseline:

• Python 3.10
• PyTorch 2.5.1
• CUDA 12.4

Install Python dependencies:

pip install -r requirements.txt

2. Initialize SAM2 submodule

git submodule update --init --recursive

3. Install official SAM2

cd external/sam2
pip install -e .

4. Download SAM2 checkpoints

Use the official checkpoint script inside the SAM2 submodule, or provide your own checkpoint path when launching training.

📊 Data Preparation

The repository supports a unified mixed-supervision layout. See:

• docs/data_format.md
• examples/dataset_manifest.example.json

For the polyp experiments in this codebase, the training script expects a manifest JSON that enumerates training samples and their supervision type.

📈️ Training

The recommended entrypoint is:

bash scripts/train.sh

The default script configuration uses:

• SAM2.1 Hiera Tiny
• warmup for 10 epochs
• joint training for 50 epochs

Citation

If you use this repository, please cite the SAMIX paper. Final BibTeX metadata will be added in the public release.

AI Contributor

This repository is also developed with assistance from @codex, OpenAI's coding agent.

Acknowledgements

This codebase builds on:

• SAM2
• SANSA