🧠 Catastrophic Forgetting Analysis
in Large Language Models

Evaluating & Mitigating Catastrophic Forgetting during Continual Instruction Tuning of BLOOMZ (560M → 3B)

🤗 31 Model Checkpoints • 🖥️ Live Demo • 📊 Key Findings • 🚀 Quick Start

---

📖 About

This research project investigates Catastrophic Forgetting (CF) — the phenomenon where neural networks lose previously learned knowledge when trained on new tasks. We conducted systematic experiments on the BLOOMZ model family across 5 sequential instruction-tuning tasks, combining mechanistic interpretability with practical mitigation strategies.

🔬 Research Pipeline

┌─────────────────────────────────────────────────────────────────────────────┐
│  BLOOMZ (560M / 1.1B / 3B)                                                │
│  ┌──────┐    ┌──────┐    ┌───────┐    ┌─────┐    ┌──────┐                  │
│  │ SIMP │ →  │ EMDG │ →  │ InqQG │ →  │ EXP │ →  │ HGen │   Sequential    │
│  │Task 1│    │Task 2│    │Task 3 │    │Task4│    │Task 5│   Fine-Tuning   │
│  └──┬───┘    └──┬───┘    └──┬────┘    └──┬──┘    └──┬───┘                  │
│     ↓           ↓           ↓            ↓          ↓                      │
│  [Eval]      [Eval]      [Eval]       [Eval]     [Eval]   ← MMLU/BoolQ/   │
│                                                               RACE         │
├─────────────────────────────────────────────────────────────────────────────┤
│  Mechanistic Analysis:  Gradient Conflict │ CKA Drift │ Loss Sharpness    │
├─────────────────────────────────────────────────────────────────────────────┤
│  Mitigation Strategies: 🔀 Mixing │ 🧊 Freezing │ 📏 Curvature Reg.      │
└─────────────────────────────────────────────────────────────────────────────┘

---

📊 Key Findings

Forgetting Metrics (FG%) — Lower is Better

Model	Method	Naive (No Fix)	🔀 Mixing	🧊 Freezing	📏 Curvature
BLOOMZ-560M	Full FT	22.03%	6.05% ✅	13.61%	13.81%
BLOOMZ-1.1B	Full FT	12.94%	5.79% ✅	11.81%	7.92%
BLOOMZ-3B	Full FT	5.84%	2.51% ✅	4.88%	5.01%
BLOOMZ-560M	QLoRA 4-bit	13.30%	8.54%	5.54% ✅	8.97%
BLOOMZ-3B	QLoRA 4-bit	5.91%	4.33%	2.22% ✅	3.92%

💡 Key Insight: Mixing works best for Full Fine-Tuning (large parameter space), while Freezing is optimal for QLoRA (constrained adapter space).

📉 Forgetting Curves — Accuracy Degradation Over Sequential Tasks

BLOOMZ-560M Full Fine-Tuning: Accuracy drops significantly on MMLU, BoolQ, and RACE as the model learns new tasks sequentially.

BLOOMZ-3B: Larger models show more resilience to forgetting, but CF is still present.

🔥 CKA Heatmap — Representational Drift Across Layers

Left: Full FT 560M — Right: QLoRA 560M. CKA scores near 1.0 = stable representations, near 0.0 = severe drift. Middle layers (L10-L14) drift most after domain-shifting tasks like EMDG.

⚡ Gradient Conflict & Loss Landscape Analysis

Left: Gradient interference between consecutive tasks — high conflict ratio indicates opposing optimization directions. Right: Fisher Trace measures loss landscape sharpness — sharper landscapes correlate with higher forgetting.

📊 Mitigation Strategy Comparison

Left: Forgetting Metric comparison across strategies — Mixing achieves the lowest FG for Full FT. Right: Pareto chart showing the Stability-Plasticity trade-off — ideal position is top-right corner.

---

🖥️ Interactive Demo

A full-stack web application for real-time visualization and model inference:

Features

• 📈 6 chart types: Forgetting curves, CKA heatmaps, gradient conflict, Pareto, strategy comparison
• 🤖 Live inference: Generate text with any of the 31 model checkpoints
• ⚔️ A/B comparison: Side-by-side output comparison (Original vs Catastrophically Forgotten)
• 🔗 Chain comparison: Track output degradation across all 5 task checkpoints
• 📊 Strategy comparison: Compare mitigation strategies in real-time

Demo includes:

• 📈 6 interactive chart types rendered server-side with Matplotlib
• 🤖 Real-time text generation with any of the 31 BLOOMZ checkpoints
• ⚔️ Side-by-side A/B comparison of Original vs Forgotten model outputs
• 🔗 Chain comparison to visualize output degradation across all 5 task checkpoints

---

🚀 Quick Start

1. Clone & Install

git clone https://github.com/dovanminh100104-coder/Catastrophic-Forgetting-Analysis-BLOOMZ.git
cd Catastrophic-Forgetting-Analysis-BLOOMZ
pip install -r requirements.txt

2. Run Demo Server

python demo_server.py

Opens automatically at http://localhost:8000

💡 First run downloads models from HuggingFace (~1.1GB/model). Use --no-preload to skip.

3. Generate Paper Charts

python generate_charts.py
# Output: charts/ directory with all PNG figures

---

📁 Project Structure

.
├── 📄 demo_server.py            # FastAPI backend + inference engine + chart API
├── 🌐 demo_frontend.html        # Interactive web UI (vanilla JS)
├── 📊 demo_data.json            # Experiment results (accuracy, FG, CKA, gradients)
├── 📈 generate_charts.py        # Publication-quality chart generator
├── 🖼️ hinh1_framework.html      # Research framework diagram
├── 📋 requirements.txt          # Python dependencies
├── 📖 README_models.md          # Full list of 31 HuggingFace models
├── 📊 assets/                   # Charts and figures for README
├── 📝 Workflow_ThucNghiem.md    # 5-step experiment workflow
├── 📄 Baibaonhom04_detai28.*    # Research paper (DOCX + PDF)
├── 📄 BaoCaoMonHocNhom04_*      # Course report (DOCX + PDF)
└── 🧪 thucnghiem/               # Kaggle experiment notebooks (GPU T4)
    ├── thucnghiem-cf-llms-560m-final.ipynb    # Full FT 560M
    ├── thucnghiem-cf-llms-1b1-final.ipynb     # Full FT 1.1B (coming soon)
    ├── thucnghiem-cf-llms-3b.ipynb            # Full FT 3B
    ├── cf-bloomz-560m-lora-final.ipynb        # QLoRA 560M
    └── cf-bloomz-3b-lora-final.ipynb          # QLoRA 3B

---

🤗 Models on HuggingFace

31 model checkpoints hosted at huggingface.co/dovanminh100104

📦 Full Model List (click to expand)

BLOOMZ-560M Full Fine-Tuning (8 models)

Checkpoint	HuggingFace ID
After Task 1 (SIMP)	dovanminh100104/cf-experiment-v4-baseline-simp
After Task 2 (EMDG)	dovanminh100104/cf-experiment-v4-baseline-emdg
After Task 3 (InqQG)	dovanminh100104/cf-experiment-v4-baseline-inqqg
After Task 4 (EXP)	dovanminh100104/cf-experiment-v4-baseline-exp
After Task 5 (HGen)	dovanminh100104/cf-experiment-v4-baseline-hgen
Mixing Strategy	dovanminh100104/cf-experiment-v4-mixing-final
Curvature Strategy	dovanminh100104/cf-experiment-v4-curvature-final
Freezing Strategy	dovanminh100104/cf-experiment-v4-freezing-final

BLOOMZ-560M QLoRA 4-bit (4 models)

Checkpoint	HuggingFace ID
Baseline (5 tasks)	dovanminh100104/cf-lora-560m-final
Mixing Strategy	dovanminh100104/cf-lora-560m-mixing-final
Curvature Strategy	dovanminh100104/cf-lora-560m-curvature-final
Freezing Strategy	dovanminh100104/cf-lora-560m-freezing-final

See README_models.md for the complete list of all 31 models.

Quick Load Example

# Full Fine-Tuned model
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("dovanminh100104/cf-experiment-v4-baseline-hgen")
tokenizer = AutoTokenizer.from_pretrained("bigscience/bloomz-560m")

# QLoRA adapter
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("bigscience/bloomz-560m")
model = PeftModel.from_pretrained(base, "dovanminh100104/cf-lora-560m-final")

---

🛠️ Tech Stack

Category	Technologies
Training	PyTorch, HuggingFace Transformers, PEFT, BitsAndBytes
Quantization	QLoRA (NF4 4-bit)
Compute	Kaggle GPU T4 (15.6GB VRAM)
Demo Backend	FastAPI, Uvicorn
Demo Frontend	Vanilla JS, CSS3 (Glassmorphism UI)
Visualization	Matplotlib, Seaborn
Model Hosting	HuggingFace Hub (31 checkpoints)

---

📚 References

• Kirkpatrick et al., "Overcoming catastrophic forgetting in neural networks", PNAS 2017
• Kornblith et al., "Similarity of Neural Network Representations Revisited", ICML 2019
• Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models", ICLR 2022
• Dettmers et al., "QLoRA: Efficient Finetuning of Quantized LLMs", NeurIPS 2023

---

👥 Authors — Group 04

Name	Role
Đỗ Văn Minh	Researcher & Developer
Nguyễn Thị Tuyết Lan	Researcher & Developer

---

⭐ If you find this project helpful, please consider giving it a star!