PyTorch CIFAR-10 Vision Experiments

This repository benchmarks convolutional networks, transfer learning, and data augmentation on CIFAR-10, with additional pretrained model-zoo demonstrations. A default random seed of 2 controls subset selection, weight initialization, DataLoader shuffling, NumPy, PyTorch CPU, and CUDA.

Reproduce the experiments

Use a CUDA-enabled Google Colab/Kaggle runtime when possible. From the project root:

pip install -r requirements.txt
CIFAR10_SEED=2 python run_all.py

To regenerate the core experiments and model-zoo outputs in one run:

CIFAR10_SEED=2 python run_all.py --with-bonus

PowerShell equivalent:

$env:CIFAR10_SEED='2'
python run_all.py

The full default run uses 15 epochs for SmallCNN, 8 epochs for each transfer mode, and 8 epochs for each augmentation regime. Optional CLI overrides are available for diagnostics, for example python run_all.py --help. Changing an epoch count changes the experiment and therefore the reported numbers.

Generated outputs

run_all.py regenerates:

• figures/cnn_curves.png - SmallCNN train/validation loss and accuracy;
• figures/transfer_compare.png - SmallCNN, ResNet feature-extraction, and ResNet fine-tuning validation-accuracy curves;
• figures/augment_compare.png - no augmentation, crop/flip, and Mixup;
• figures/results_seed_2.json - exact metrics, parameter counts, hyperparameters, histories, package/device metadata, and runtime.

The three individual experiment scripts can also be run in order:

CIFAR10_SEED=2 python experiments/train_cnn.py
CIFAR10_SEED=2 python experiments/transfer_compare.py
CIFAR10_SEED=2 python experiments/augment_compare.py

The JSON files preserve unrounded histories, parameter counts, hyperparameters, validation-based architecture selection, final untouched-test metrics, runtime, and software/device provenance.

Dataset and experimental protocol

• CIFAR-10 classes: airplane, automobile, bird, cat, and deer (original class IDs 0,1,2,3,4, remapped to contiguous targets 0..4).
• Per-class sample counts: 800 selected training-subset images and 200 official held-out test images. A seeded, stratified 100 images per class are reserved from the training subset for validation, leaving 3,500 fit, 500 validation, and 1,000 untouched test images.
• The subset is stratified and fixed by seed 2.
• SmallCNN uses 32x32 inputs and CIFAR-10 channel normalization.
• Both ResNet-18 modes use 224x224 inputs and the ImageNet mean/std associated with the pretrained weights.
• Feature extraction freezes every pretrained backbone parameter and trains only the new 512-to-5 fc layer. Frozen BatchNorm running statistics remain in evaluation mode during training.
• Fine-tuning updates the full ResNet with a learning rate ten times smaller than feature extraction (1e-4 versus 1e-3). All other settings match.
• The augmentation study automatically selects the architecture with the highest validation accuracy, then reinitializes that same architecture from the same seed for all three regimes. Mixup uses a hand-written Beta(1,1) sampler and the convex-combination loss used throughout the project. Its explicit shared configuration is src.augment.DEFAULT_MIX_ALPHA = 1.0.
• The transfer comparison intentionally overlays SmallCNN's 15-epoch, 32x32 curve with the two 8-epoch, 224x224 ResNet curves. It compares validation trajectories, not equal compute budgets.
• The augmentation study reruns the no-augmentation fine-tuning baseline with the same seed and settings. Its exact agreement with the B2 fine-tuning result is an expected reproducibility check, while retaining the required three-regime experiment.

The official CIFAR-10 test subset is never passed to fit and is never used for architecture selection. It is evaluated once at the end of each completed run. The checked-in results were regenerated on a Tesla T4 with the full default schedule (15 SmallCNN epochs, 8 transfer-learning epochs per mode, and 8 augmentation epochs per regime). Metric files use protocol version stratified-validation-v2.

Experiment	Best validation accuracy	Final test accuracy
SmallCNN	66.8%	63.7%
ResNet-18 feature extraction	91.8%	90.3%
ResNet-18 fine-tuning	93.6%	93.1%
Fine-tuned ResNet-18, no augmentation	93.6%	93.1%
Fine-tuned ResNet-18, standard crop/flip	94.2%	95.0%
Fine-tuned ResNet-18, Mixup	93.0%	92.1%

Exact unrounded values and histories are in figures/results_seed_2.json.

Pretrained model-zoo demonstrations

The included inference-only script runs three pretrained torchvision models on the images in data/samples/:

CIFAR10_SEED=2 python experiments/model_zoo_bonus.py

It produces:

• figures/zoo_detection.png using fasterrcnn_resnet50_fpn, including labels, scores, a score threshold, and post-NMS overlap metadata;
• figures/zoo_segmentation.png using maskrcnn_resnet50_fpn, with instance-mask overlays and boxes;
• figures/zoo_pose.png using keypointrcnn_resnet50_fpn, with human keypoints and COCO skeleton connections;
• figures/zoo_metrics.json with predictions and the required FPN, NMS, RoIAlign, and heatmap-regression explanations.

The current sample inputs were obtained from the internet and are used only for the inference demonstration. Replace them with images you are authorized to redistribute before publishing the repository.

Project structure and public API

src/data.py       set_seed, get_cifar10_subset, make_loaders
src/models.py     SmallCNN, build_resnet18
src/augment.py    mixup_batch, cutmix_batch, mix_criterion
src/engine.py     train_one_epoch, evaluate, fit
src/utils.py      accuracy, count_trainable_params, plot_history
experiments/      training, transfer-learning, and augmentation entry points
                    plus model_zoo_bonus.py
notebooks/        Colab entry point using the same source code
run_all.py        complete reproducible pipeline

No training framework or pre-built Mixup/CutMix utility is used. The optimizer step, mode switches, device transfer, backward pass, metric accumulation, and mixed-label loss are explicit in src/engine.py.

Verification

Fast unit tests do not download CIFAR-10 or pretrained weights:

python -m unittest discover -v

The tests cover SmallCNN shapes and parameter count, ResNet freezing semantics, the Mixup/CutMix contracts, the convex loss, stratified train/validation splitting, validation-based model selection, the training/evaluation API, and bonus input/post-processing helpers.

Notes

The first full run downloads CIFAR-10 and the official torchvision weights. If the legacy CIFAR-10 host fails TLS verification, the downloader uses a checksum-verified mirror while retaining torchvision's official archive MD5. Bonus inference downloads three additional model checkpoints. Results are deterministic for a fixed software/device setup; tiny floating-point differences can still occur between CPU and different CUDA hardware. AI assistance was used to help structure the validation protocol, review and test the code, and create the clearly labelled model-zoo examples.