In this project, I used Quadratic Discriminant Analysis (QDA) and a Bayesian Linear Regression model to classify images of dogs and frogs. The datasets used can be found at CIFAR-10 and contain 6,000 images—5,750 for training and 250 for testing.
Given the dataset I fit the mean of a Gaussian to this data. Suppose we use the following log-likelihood:
where we have a prior over the mean:
.
Using the posterior 
I found the Minimum Mean Squared Error (MMSE) estimate and have plotted the mean decision boundry 
.
I implemented QDA to create a generative classifier for images. To do this, I fit a multivariate Gaussian to each class of images independently and then compared the likelihoods to classify them.
The appropriate conjugate prior for the covariance of a Gaussian is called the inverse Wishart distribution. This distribution models the covariance matrix of a Gaussian distribution since it is defined over positive definite (PD) matrices. In this project I assumed that: 
where 
. The MMSE estimate for the Gaussian distribution is then: 
.
For each class (dogs and frogs), I fit a Gaussian distribution and used the following equation to classify both the training and test sets:
Below is the plot of the accuracy of QDA as a function of v:
I used a Bayesian Linear Regression model to classify images using the "classification as regression" method. To achieve this, I labeled the regression targets as ±1:
-
+1 for the first class (dogs)
-
-1 for the second class (frogs)
In other words, I fit the following model:
The basis functions I used were Gaussian basis functions, with centers defined by the first M training points:
.
I then trained the model with M basis functions from each class and plotted the training and test accuracy as a function of M. The test accuracy when using all the training data as basis functions (M = all) is 0.88.
Finally, I plotted the 25 dog images that the model was most and least confident in classifying.
A notable observation is that the 25 most confidently classified images often contain a dog with white fur, and the dog is clearly distinguished from the background.
- Clone the Repository
- Download the dogs and frogs CIFAR-10 Dataset from https://www.cs.toronto.edu/%7Ekriz/cifar.html
- Execute the code found in classifiers.py