A small CNN that tells apart real photographs from AI-generated ones, plus a hyperparameter study over convolutional width and batch size.
ML.py trains a LeNet-style CNN on the CIFAKE dataset to classify 32x32 images as REAL (photographs from CIFAR-10) or FAKE (generated with Stable Diffusion 1.4). On top of that I ran a 5x4 grid over conv width and batch size, logged every run to research.csv, and used data_processing.py to plot the results.
This is exploratory work, not a confirmatory study. Everything below is a single seed with one run per grid cell, so treat the numbers as directional rather than settled.
Python 3.13, torch and torchvision.
python3 -m venv .venv
source .venv/bin/activate
pip install torch torchvision matplotlibThe dataset is not in this repo. It's CIFAKE by Jordan J. Bird, available on Kaggle at birdy654/cifake-real-and-ai-generated-synthetic-images. Download it and unzip so the layout looks like this:
archive/
├── train/
│ ├── REAL/ 50,000 images
│ └── FAKE/ 50,000 images
└── test/
├── REAL/ 10,000 images
└── FAKE/ 10,000 images
ImageFolder assigns class indices alphabetically, so FAKE = 0 and REAL = 1. Worth remembering, since it decides which column of the output means what.
ML.py splits the 100,000 training images into 90,000 for training and 10,000 for validation using a fixed seed of 42, so the split stays identical across runs. The 20,000-image test set is untouched — every accuracy reported here is validation accuracy.
Four flags, all optional:
| Flag | Default | What it does |
|---|---|---|
--batch |
64 | Images per gradient step |
--c1 |
6 | Filters in the first conv layer |
--c2 |
16 | Filters in the second conv layer |
--seed |
0 | Seeds the initial weights |
python ML.py --batch 128 --c1 32 --c2 64 --seed 0Training is fixed at 10 epochs. Each run saves weights to model_b{batch}_c{c1}_{c2}_s{seed}.pth and appends a row to research.csv (it writes the header only if the file doesn't exist yet), so the sweep is just a loop:
for b in 16 32 64 128; do
for w in "3 8" "6 16" "12 32" "18 32" "32 64"; do
set -- $w
python ML.py --batch $b --c1 $1 --c2 $2
done
done
python data_processing.py # writes the tables and the eight PNGsdata_processing.py is the analysis half. It reads research.csv, prints three tables to the terminal, and writes the eight figures used below into figures/ at 150 dpi, creating the folder if it isn't there. It needs matplotlib and nothing else — no seaborn.
The three tables are the accuracy grid with row and column means, the within-width range (max minus min across the four batch sizes), and the mean real_acc - fake_acc per batch size. Those are the numbers quoted throughout this README, so re-running the script regenerates everything here from the CSV.
One thing worth knowing: before plotting anything it recomputes each configuration's parameter count from the layer shapes and asserts it matches the params column in the CSV. If ML.py and the logged results ever drift apart, the script fails loudly instead of quietly drawing figures from stale data.
input 3 x 32 x 32
conv1 (5x5) + ReLU c1 x 28 x 28
maxpool (2x2) c1 x 14 x 14
conv2 (5x5) + ReLU c2 x 10 x 10
maxpool (2x2) c2 x 5 x 5
flatten c2 * 25
fc1 + ReLU 120
fc2 + ReLU 84
fc3 2 (logits)
Cross-entropy loss, SGD at lr 0.01 with momentum 0.9. The c2*5*5 input to fc1 isn't a number I picked — it falls out of the arithmetic. A 5x5 conv with no padding takes 32 down to 28, pooling halves it to 14, the second conv takes that to 10, and pooling halves it again to 5. So whatever c2 is, the tensor reaching the classifier is c2 x 5 x 5. Change the kernel size or add padding and this number has to be recomputed.
Validation accuracy (%), 10 epochs, seed 0:
| width (c1,c2) | params | batch 16 | batch 32 | batch 64 | batch 128 | row mean |
|---|---|---|---|---|---|---|
| (3,8) | 35,290 | 88.23 | 88.89 | 88.91 | 84.21 | 87.56 |
| (6,16) | 61,326 | 89.59 | 89.90 | 90.56 | 84.27 | 88.58 |
| (12,32) | 116,998 | 89.08 | 92.14 | 91.84 | 92.23 | 91.32 |
| (18,32) | 122,254 | 90.48 | 92.48 | 91.67 | 93.07 | 91.92 |
| (32,64) | 256,150 | 92.48 | 92.88 | 91.78 | 93.47 | 92.65 |
| col mean | 89.97 | 91.26 | 90.95 | 89.45 | 90.41 |
Best run is batch 128 with (32,64) at 93.47%. For reference, Bird & Lotfi report 92.98% on CIFAKE, so this lands slightly above it — though their number is on the test set and mine is on a validation split, so it isn't a clean comparison.
Going from 35k to 256k parameters buys about 5 points of accuracy, but almost all of it arrives early. The first stretch, 35k to 117k, is worth 3.76pp. The last stretch, 122k to 256k, is worth 0.73pp for more than double the parameters.
Batch 128 is the worst setting for the two smallest widths (84.21 and 84.27) and the best setting for the three largest (92.23, 93.07, 93.47). There is no such thing as a good batch size here independent of model size.
The column means completely hide this. They read 89.97, 91.26, 90.95, 89.45 — a gentle hump that makes batch 128 look mediocre, when it was the best setting in this run for each of the three largest widths.
Sensitivity to batch size also shrinks as the model gets wider — the spread across the four batch sizes falls from 4.70pp to 1.69pp, though (6,16) breaks the pattern with the largest spread in the grid at 6.29pp.
Between 68% and 82% of the weights in every configuration live in fc1, the first fully-connected layer. That reframes finding 1: "more parameters helped" is mostly "a wider input to the classifier helped," which is a weaker claim than added convolutional capacity helping. The conv layers are a rounding error by parameter count.
The batch 128 (6,16) run scores 84.27% overall, which looks like a mildly bad run. Split by class, it gets 70.4% on fakes and 98.0% on reals — it has mostly learned to answer "real." A single accuracy number would never show this.
The direction of the bias tracks batch size: mean real_acc - fake_acc is -3.26 and -2.56 at batches 16 and 32, then +7.84 and +8.38 at 64 and 128.
The data is unaudited. I never checked whether the REAL and FAKE images differ in file format, dimensions, or compression, and that's a known failure mode on this dataset — a model can score well by reading encoding artifacts rather than anything about the image content.
Single seed. Everything is seed 0, n=1 per cell, no error bars. The batch main effect spans 1.8pp across the column means, and I have no evidence that's larger than run-to-run noise.
Batch size and update count are confounded. At a fixed 10 epochs, batch 16 takes 56,250 weight updates and batch 128 takes 7,040 — an 8x difference. The batch axis is really a batch-and-training-length axis, and the small-model failures at batch 128 may be undertrained models rather than a batch size effect.
Only final-epoch validation is recorded. There's no per-epoch curve, so I can't tell an undertrained run from a converged one. Training accuracy isn't logged either, so overfitting is unmeasured.
c1 and c2 vary together everywhere except at c2=32, so convolutional width and classifier width can't be separated. The one near-controlled comparison, c1=12 vs c1=18 at c2=32, averages +0.60pp with one of four cells negative. That's probably nothing.
Three seeds on the baseline and on the two cells the interaction claim rests on — (6,16) and (32,64) at batch 128 — so the headline result has error bars. Log validation accuracy per epoch to separate undertrained from converged. Then test on images from a different generator, because right now I can't tell whether the model learned to detect synthesis in general or just Stable Diffusion 1.4's fingerprint.
Dataset: CIFAKE, from Bird, J.J. and Lotfi, A. (2024), "CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images," IEEE Access. The real images come from CIFAR-10; the fake images were generated with Stable Diffusion 1.4.
CIFAKE is listed on Kaggle under an "Other" license, with CIFAR-10's academic-use terms underneath it. No images from the dataset are redistributed in this repository — you need to download it yourself.