Denoising Diffusion Probabilistic Models (DDPM) – From First Principles

Overview

This project implements a Denoising Diffusion Probabilistic Model (DDPM) from scratch to understand the forward noising process, reverse denoising process, and the mathematical intuition behind diffusion-based generative models.

The emphasis of this repository is conceptual correctness and training dynamics, not pretrained weights or plug-and-play inference.

---

Important Note (Read This First)

⚠️ Model weights (model.pt) are not included in this repository. The model was trained locally, but the checkpoint was not saved.

This repository is intended to:

• Demonstrate a correct DDPM implementation
• Show understanding of diffusion mechanics
• Serve as a learning and experimentation reference

Re-training the model is straightforward using the provided code.

---

Why Diffusion Models?

Diffusion models form the backbone of:

• Stable Diffusion
• Image and audio generation systems
• Modern generative AI pipelines

Understanding DDPMs requires:

• Probability theory
• Stochastic processes
• Careful loss and noise scheduling

This project focuses on those internals, not abstraction layers.

---

Core Concepts Implemented

• Forward diffusion (progressive Gaussian noise injection)
• Reverse diffusion (learned denoising process)
• Noise schedule (β, α, and cumulative products)
• Time-step conditioning
• Mean Squared Error (MSE) noise prediction objective

---

Model Architecture

• U-Net–style architecture
• Time-step embeddings
• Skip connections for stable gradient flow

Design choices prioritize:

• Training stability
• Interpretability
• Alignment with original DDPM formulation

---

Training Objective

The model is trained to predict the noise added at each timestep:

[ L = \mathbb{E}{x_0, \epsilon, t} [|| \epsilon - \epsilon\theta(x_t, t) ||^2 ] ]

This formulation ensures:

• Stable optimization
• Clear separation between noise and signal

---

Dataset

• Image dataset (e.g., MNISTimages)

The dataset is intentionally simple to focus on diffusion behavior, not dataset complexity.

---

Results & Observations

• Progressive denoising observed across timesteps
• Sensitivity to noise schedule parameters
• Training instability when β schedule is poorly chosen

Sample outputs can be regenerated by retraining the model.

---

Project Structure

├── result/
├── notebooks/
└── README.md

---

Key Learnings

• How diffusion models differ from GANs and VAEs
• Importance of noise schedules
• Why diffusion training is stable but slow
• Relationship between score matching and denoising

---

Limitations

• No pretrained weights included
• Training requires significant compute for high-quality samples
• Focused on correctness over optimization

---

Future Work

• Save and version model checkpoints
• Improved noise schedules (cosine, learned)
• Class-conditional diffusion
• Latent diffusion models

---

Author

Hasan Kabir AI / Generative Models Engineer

---

Final Note

This repository reflects a theory-first approach to diffusion models. The absence of pretrained weights does not reduce the technical depth of the implementation.