HPC-Project

🧠 Neural Network Acceleration on GPUs — MNIST Classification

📌 Overview

This project explores the acceleration of a simple neural network model for MNIST digit classification using GPU programming via CUDA. Starting from a baseline CPU implementation (V1), we develop multiple versions with increasing levels of GPU optimization:

• V1: Sequential (CPU-based) implementation
• V2: Naive CUDA implementation
• V3: Optimized CUDA implementation using memory, communication, and kernel-level optimizations
• V4: Final version using NVIDIA Tensor Cores for peak performance

---

🎯 Objectives

• Learn GPU programming with CUDA
• Apply memory hierarchy and thread-level parallelism
• Optimize performance using CUDA best practices
• Leverage Tensor Cores for dense matrix operations

---

🧠 Neural Network Details

• Input: 28x28 pixel grayscale images (784 features)
• Output: Classification into digits 0 through 9
• Architecture:
  • Input Layer (784 neurons)
  • One or more Hidden Layers
  • Output Layer (10 neurons)
• Training: Supervised learning using stochastic gradient descent
• Loss Function: Cross Entropy or MSE
• Activation Functions: ReLU / Sigmoid

---

🧪 Dataset — MNIST

The MNIST dataset contains 70,000 labeled images:

• 60,000 for training
• 10,000 for testing

Each image is a 28x28 grayscale pixel matrix. Labels range from 0 to 9.

Ensure the dataset files are placed in the data/ folder. Use the .csv format or convert from .idx format using available scripts if needed.

---

📁 Repository Structure

.
├── src/              # Source code for all implementations
│   ├── V1/           # Sequential version (CPU)
│   ├── V2/           # Naive GPU version
│   ├── V3/           # Optimized GPU version
│   └── V4/           # Tensor Core accelerated version
│
├── data/             # Contains the MNIST dataset
│
├── report/           # Project report (PDF or LaTeX)
│
├── slides/           # Presentation slides (PDF or PPT)
│
├── Makefile          # Compilation script
│
└── README.md         # Project documentation and usage guide

---

👥 Team Members

Name
Muhammad Hamza Nadeem
Faiz-ul-hassan
Faris Ali