Applying self-supervised and contrastive pretraining methods to improve certified robustness via randomized smoothing.
This repository is a customized fork of OpenMMLab's mmpretrain toolbox, adapted for research on randomized smoothing — a technique for building classifiers with provable robustness guarantees against adversarial perturbations.
The key idea is that self-supervised pretraining (SimCLR, BYOL, MoCo, BarlowTwins, etc.) can produce representations that lead to tighter certified robustness bounds when combined with randomized smoothing at inference time.
Randomized smoothing constructs a "smoothed classifier" by averaging the predictions of a base classifier over Gaussian noise perturbations of the input. This provides a certified radius within which the classifier's prediction is guaranteed not to change, regardless of the adversarial perturbation applied.
The repo inherits the full mmpretrain ecosystem:
mmpretrain/— core library (models, datasets, training engine)configs/— training configurations for 100+ pretrained model architecturestools/— training, evaluation, and analysis scriptsprojects/— standalone research projects built on top of the frameworkdocker/— containerized environment setup
BarlowTwins, BYOL, SimCLR, MoCo v2/v3, MAE, BEiT, DINO, SwAV, and many more — see the full model zoo.
pip install -r requirements.txt
pip install -e .Or using Docker:
cd docker && docker build -t selfsup-ransmooth .# Pretrain with a self-supervised method (e.g., SimCLR on CIFAR-10)
python tools/train.py configs/simclr/simclr_resnet50_8xb256-coslr-200e_in1k.py
# Evaluate with randomized smoothing
python tools/test.py <config> <checkpoint> --cfg-options ...- Randomized Smoothing with Contrastive Pretraining — the companion research repository
- mmpretrain — the upstream toolbox
- Python, PyTorch
- OpenMMLab (mmengine, mmpretrain)
- CUDA for GPU acceleration
- WandB for experiment tracking