DIRS: Principled Reflection Separation via Nonlinear Superposition and Feature Interaction

📄 Paper | 🌐 Project Page | 🖼️ Demo | 🤗 Models & Data

Qiming Hu, Mingjia Li, Yuntong Li, Xiaojie Guo
College of Intelligence and Computing, Tianjin University

DIRS is a unified reflection separation framework that models real sRGB layer formation with learnable nonlinear superposition and separates transmission and reflection through explicit dual-stream feature interaction. The release contains three public variants, from efficient CNNs to a high-quality Transformer, all sharing the same xreflection interface and returning T, R, Phi, and Psi.

Method Overview

DIRS is built around two tightly coupled ideas:

• Learnable Offset-Residual Superposition (LORS) models sRGB reflection formation with nonlinear residuals Phi and zero-order offsets Psi, instead of forcing all layer coupling into a linear mixture.
• Dual-Stream Interactive Reflection Separation (DIRS) keeps transmission and reflection in dedicated streams while exchanging discriminative features through a Selection-Interaction-Fusion design.

The released variants instantiate this design at different operating points:

• DIRS-YTMT recycles features suppressed by one branch into the other branch through paired activations, giving a compact and efficient CNN baseline.
• DIRS-MuGI replaces hard activation exchange with mutual gating, improving robustness under spatially varying nonlinear coupling.
• DIRS-PAIR uses Dual-Stream Joint Attention and parallel self-attention, so cross-stream information can be selected rather than blindly aggregated.

Visual Results

_{Transmission predictions in challenging real-world scenarios.}

_{Reflection separation and reflection scene reconstruction.}

_{DIRS adapted to polarized multi-image reflection separation.}

Model Zoo

The model table follows the paper protocol. FLOPs and latency are measured on a single NVIDIA RTX 3090 at 384 x 384 input resolution. PSNR/SSIM are averaged over Real20 and SIR2.

Model	Type	Params	FLOPs	Time	PSNR	SSIM	Checkpoint
DIRS-YTMT	CNN, activation interaction	32.42M	102.91G	31.35 ms	24.94	0.902	ckpt
DIRS-MuGI	CNN, mutual gating	84.47M	153.98G	49.95 ms	25.63	0.913	ckpt
DIRS-PAIR	Transformer, joint attention	48.80M	200.22G	75.36 ms	26.37	0.918	ckpt
DIRS-PAIR + Nature	Transformer, joint attention	48.80M	200.22G	75.36 ms	26.95	0.926	ckpt

Supplementary checkpoints:

Task	Checkpoint	Notes
Reflection Scene Reconstruction	rsr_supplement.ckpt	Uses local SDXL base model files.
Polarized Reflection Separation	polarized_dirs_pair_lors.ckpt	DIRS-PAIR adapted to PolaRGB inputs.
Swin prior	swin_large_o365.pth	Shared by DIRS-PAIR and polarized DIRS.

Installation

conda create -n dirs python=3.8
conda activate dirs
pip install -r requirements.txt

The supplementary SDXL/ControlNet experiments require extra packages:

pip install -r requirements-supplement.txt

Checkpoints and Data

Download the released assets from DIRS-Models and place the directories at the repository root:

DIRS/
  checkpoints/
    dirs_ytmt_lors.ckpt
    dirs_mugi_lors.ckpt
    dirs_pair_lors.ckpt
    dirs_pair_lors_nature.ckpt
    polarized_dirs_pair_lors.ckpt
    rsr_supplement.ckpt
    pretrained/swin_large_o365.pth
  datasets/
    TJReflection/

Evaluation

Run the Lightning/YAML evaluation path:

python -m xreflection.train \
  --config options/test_dirs_pair_lors.yml \
  --test_only \
  --resume checkpoints/dirs_pair_lors.ckpt

Use the matching config and checkpoint for other variants:

Variant	Config	Checkpoint
DIRS-YTMT	options/test_dirs_ytmt_lors.yml	checkpoints/dirs_ytmt_lors.ckpt
DIRS-MuGI	options/test_dirs_mugi_lors.yml	checkpoints/dirs_mugi_lors.ckpt
DIRS-PAIR	options/test_dirs_pair_lors.yml	checkpoints/dirs_pair_lors.ckpt
DIRS-PAIR + Nature	options/test_dirs_pair_lors.yml	checkpoints/dirs_pair_lors_nature.ckpt

For a lightweight direct evaluation entry:

python scripts/eval_dirs_variants.py --variant all --limit 10

Training

DIRS provides two training settings for each main variant. wo_nature trains with synthetic VOC and Real20 pairs; w_nature additionally uses the Nature split.

python -m xreflection.train --config options/train_dirs_pair_lors_wo_nature.yml

Available training configs:

Variant	Setting I	Setting II
DIRS-YTMT	options/train_dirs_ytmt_lors_wo_nature.yml	options/train_dirs_ytmt_lors_w_nature.yml
DIRS-MuGI	options/train_dirs_mugi_lors_wo_nature.yml	options/train_dirs_mugi_lors_w_nature.yml
DIRS-PAIR	options/train_dirs_pair_lors_wo_nature.yml	options/train_dirs_pair_lors_w_nature.yml

Supplementary experiments:

python -m xreflection.train --config options/train_supp_rsr.yml
python -m xreflection.train --config options/train_supp_polarized_dirs_pair_lors.yml

Supplementary evaluation scripts:

python scripts/test_supp_rsr.py \
  --data_root datasets/supp_rsr/tjreflection_mid_eval/separation_768 \
  --limit 1

python scripts/test_supp_pola.py \
  --data_root datasets/PolaRGB \
  --limit 1

Repository Map

Path	Description
xreflection/archs/dirs_lors_arch.py	DIRS-LORS base class and three released variants.
xreflection/archs/dirs_ytmt_arch.py	YTMT interaction blocks.
xreflection/archs/dirs_mugi_arch.py	MuGI interaction blocks.
xreflection/archs/dirs_pair_arch.py	PAIR attention interaction blocks.
xreflection/models/dirs_model.py	Lightning model wrapper with grouped DIRS losses.
xreflection/models/rsr_model.py	Reflection scene reconstruction training wrapper.
xreflection/models/dirs_polar_model.py	Polarized multi-image DIRS wrapper.
xreflection/data/dirs_dataset.py	DIRS training and evaluation datasets.
options/	Training and evaluation YAML configs.
scripts/	Direct evaluation and supplementary testing scripts.

Citation

If DIRS is useful for your research, please cite:

@article{hu2026dirs,
  title={Principled Reflection Separation via Nonlinear Superposition and Feature Interaction},
  author={Hu, Qiming and Li, Mingjia and Li, Yuntong and Guo, Xiaojie},
  journal={arXiv preprint},
  year={2026}
}

Acknowledgement

This codebase builds on the xreflection training interface used in our previous reflection separation projects. We also thank the maintainers of the public reflection removal benchmarks used in the paper.