Fog Simulation in Arbitrary Video Dataset

We propose a framework for dataset augmentation with homogeneous and heterogeneous fog of different intensity levels, illustrated in the MOT17 benchmark. For more information, please refer to our preprint:

Fog Simulation
Nadezda Kirillova, Muhammad Jehanzeb Mirza, Horst Possegger, Horst Bischof

Fog in MOT17 dataset

Clear images are taken from MOTChallenge benchmark.

From left to right: Clear Conditions, Homogeneous Fog, Heterogeneous Fog.

• Daytime front-view large square captured by a static camera:

MOT17-02-fog2-collage.mp4

• Heavily illuminated night crowd square captured by a surveillance camera:

MOT17-04-fog2-collage.mp4

• Close up indoor sceen captured by a low angle static camera:

MOT17-09-smoke2-collage.mp4

Setup

1. Clone the repository recursively:

git clone --recurse-submodules https://github.com/nadezola/IntoTheFog_MOT17.git

• If you already cloned and forgot to use --recurse-submodules you can run git submodule update --init

2. We recommend to use:

• Ubuntu 20.04
• CUDA 10.2
• Python 3.7 virtual environment with requirements.txt

python3.7 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

3. Working directory is the root of the repository.

Model and Data preparation

1. We leverage the code for monocular depth estimation from MiDaS:

   • Download the pretrained model dpt_beit_large_512.pt and put it in ./DepthEstimation/weights

2. Download MOT17 dataset images (we place them in ./data/mot17/train/clear/images).

Run

1. Configure the file opt.py.
2. Run:

   python run_augmentation.py --input       # Specify the path to clear images
                              --out         # Specify the path to outputs
                              --loaddepth   # [optional] Load depth images instead of run depth estimation (default loading from '[outroot]/[seqname]/depth_pred')

Citation

If you find our code useful, please reference:

@article{kirillova2024fogmot17
  title={{Into the Fog: Evaluating Multiple Object Tracking Robustness}},
  author={Kirillova, Nadezda and Mirza M. Jehanzeb and Possegger, Horst and Bischof, Horst},
  journal={arXiv preprint: 2404.10534},
  year={2024},
  note= {\url{https://arxiv.org/abs/2404.10534}},
}