[ECCV'20] CA-NoiseGAN

Project | Paper | Abstract | Long Video

Overview

CA-NoiseGAN is a PyTorch implementation of
"Learning Camera-Aware Noise Models",
Ke-Chi Chang, Ren Wang, Hung-Jin Lin, Yu-Lun Liu, Chia-Ping Chen, Yu-Lin Chang, Hwann-Tzong Chen
in European Conference on Computer Vision (ECCV) 2020 conference.

Modeling imaging sensor noise is a fundamental problem for image processing and computer vision applications. While most previous works adopt statistical noise models, real-world noise is far more complicated and beyond what these models can describe. To tackle this issue, we propose a data-driven approach, where a generative noise model is learned from real-world noise. The proposed noise model is camera-aware, that is, different noise characteristics of different camera sensors can be learned simultaneously, and a single learned noise model can generate different noise for different camera sensors. Experimental results show that our method quantitatively and qualitatively outperforms existing statistical noise models and learning-based methods.

Requirements

This test code is implemented under Python3.
Following libraries are required:

• PyTorch == 1.1.0
• see requirements.txt for more detail

If you want to visualize the results of Noise Flow, the libraries are also required:

• TensorFlow == 1.13.0
• tensorflow-probability >= 0.5.0con

Usage

1. Prepare Data
   We prepare our test data as Data.zip in [Google Drive] and they are totally derived from SIDD dataset. Please unzip it and you can change the data_dir in config.yml into your data path.

2. Download Pretrained Models
   We provide pretrained baseline models of noise models and denoisers as checkpoints.zip in [Google Dirve].
   Please unzip it under the root directory.

3. Prepare Runtime Environment

   pip install -r requirements.txt

4. Test the Noise Models and Denoisers
   You need to check the correctness of each path in config.yml.
   Moreover, you can modify the amount of samples and patch size. See config.yml for more detail. Then, you can run the following scripts.

   • Noise Models

     python test_noise_models.py --config config.yml

   • Denoisers

     python test_denoisers.py --config config.yml

5. Visual Results
   The results will be saved in ./samples/, including raw images and sRGB images. We provide the metrices we used in our paper, you can evaluate the quantitative results of each model, too.

Citation

@InProceedings{chang2020learning,
	author={Chang, Ke-Chi and Wang, Ren and Lin, Hung-Jin and Liu, Yu-Lun and Chen, Chia-Ping and Chang, Yu-Lin and Chen, Hwann-Tzong},
	title={Learning Camera-Aware Noise Models},
	booktitle={Proceedings of European Conference on Computer Vision (ECCV)},
	year={2020}
}

Resources

• SIDD Dataset
• Noise Flow (Tensorflow)
• Simple Camera Pipeline (Python, MATLAB)

Acknowledgement

• Mediatek Inc.