Machine Learning & Advanced ML Journey @ PUCIT

Implementations, notes, and projects from CS-567 Machine Learning and CS-667 Advanced Machine Learning taught by Dr. Nazar Khan during my MPhil Studies at PUCIT.

📚 Topics Covered

Core Machine Learning (CS-567)

• Probability & Decision Theory
• Curve Fitting, Regularization & Bayesian Perspective
• Linear Regression & Classification
• Maximum Likelihood & MAP Estimation
• Gaussian Mixture Models & EM Algorithm
• Non-parametric Density Estimation
• PCA & Dimensionality Reduction

Advanced Topics (CS-667)

• Neural Networks & Backpropagation
• Convolutional Neural Networks
• Autoencoders
• Spectral Clustering
• Support Vector Machines
• Boosting
• Mixture Density Networks
• Conditional Mixture Models

🚀 Key Projects & Implementations

• Polynomial Curve Fitting — Regularized & Bayesian approaches
• Logistic Regression with IRLS (Iterative Reweighted Least Squares method) Test Accuracy: 98.49%
• Multiclass Logistic Regression with SGD (Stochastic Gradient Descent on MNIST dataset) Test Accuracy: 92%
• Gaussian Mixture Models from scratch + EM
• PCA for dimensionality reduction & visualization
• Neural Nets & CNNs on MNIST (NN MLP with one hidden layer)
• Spectral Clustering demo
• Autoencoders for representation learning

🛠️ Tech Stack

• MATLAB & Python
• Python, NumPy, Matplotlib, scikit-learn
• PyTorch / TensorFlow (for deep learning parts)
• Jupyter Notebooks
• MATLAB Files

📈 Results Highlights

(Will add soon screenshots of accuracy tables, confusion matrices, visualizations here; once i upload the implementation)

• Polynomial Curve Fitting

• Logistic Regression with IRLS

  • Training samples: 12665 -Test samples: 2115 -Using 1500 samples (for speed & memory) - Training finished with 1500 samples. -Test Accuracy: 98.49% on data subset - 100.00% of Full dataset

    

• Logistic Regression with SGD

  • Evaluating on Test Set: Test Accuracy: 88% on data subset Test Accuracy Full dataset: 92.12%

    

• Back Propagation Nural Networks with MLP

  • Test accuracy 80-90%

    

• Convolution Nural Network

  • High speed and accuracy using PyTorch
  • Low speed and accuracy while implimenting withoutout any extenal dependency. High complexity.

• Principal Component Analysis (PCA)

  • Dinemtionality Reduction

    • Early PCs capture global stroke patterns and digit shapes.

    • umulative explained variance plot shows diminishing returns after ~50–100 components.

    • Later PCs capture finer details and variations (e.g., loops, slants).

      

How to Run

Use MATLAB

pip install -r requirements.txt
jupyter notebook