Facial Emotion Recognition (Deep Learning – CNN)

A complete Facial Emotion Recognition (FER) system built with TensorFlow/Keras. It includes multiple trained models, evaluation metrics, confusion matrices, and a real-time emotion detector using OpenCV.

The system classifies facial expressions into 7 emotions:

• 😄 Happy
• 😠 Angry
• 😢 Sad
• 😮 Surprise
• 😐 Neutral
• 😨 Fear
• 🤢 Disgust

---

🌐 Try it online

A live demo is hosted on Hugging Face Spaces — upload a photo or use your webcam:

👉 https://huggingface.co/spaces/lokeshkumar79/facial-emotion-recognition

The trained model is also published standalone (with a usage example) on the Hub:

👉 https://huggingface.co/lokeshkumar79/facial-emotion-recognition

---

🚀 Key Features

• Multiple training experiments (Initial, Intermediate, Final)
• JSON + .keras model + weights for reproducibility
• Real-time emotion detection via webcam (local) and a hosted Gradio demo (Hugging Face Spaces)
• Confusion matrices (CSV + PNG), classification reports, accuracy & loss curves per stage
• Reproducible environment (requirements.txt + environment.yml)

---

📂 Project Structure

Facial_Emotion_Recognition/
├── FINAL_TRAINING.ipynb              # Final training experiment (produces the recommended model)
├── INITIAL_TRAINING.ipynb            # Initial training experiment
├── TRAINING.ipynb                    # Intermediate training
│
├── realtimedetection.py              # Real-time webcam emotion detection
├── oldrealtimedetection.py           # Older version of the detector
│
├── facialemotionmodel.json/.keras/.weights.h5           # Intermediate model
├── initialfacialemotionmodel.json/.keras/.weights.h5     # Initial model
├── finalfacialemotionmodel.json/.keras/.weights.h5       # Final model (recommended)
├── best_emotion_cnn.weights.h5       # Best-performing model weights
│
├── Confusion_matrix_*.png / confusion_matrix_*.csv       # Confusion matrices per stage
├── accuracy_curve_*.png / loss_curve_*.png               # Training curves per stage
├── classification_report_*.txt                           # Classification reports per stage
│
├── images/
│   ├── train/                        # FER-2013 training images, 7 emotion subfolders
│   └── test/                         # FER-2013 test images, 7 emotion subfolders
│
├── hf_deploy/                        # Hugging Face model card + Space source (see below)
├── requirements.txt                  # Python dependencies (pip)
├── environment.yml                   # Conda environment (see note under Known issues)
└── README.md

---

🧠 Model Architecture & Overview

The CNN is built using the following structure:

• 3 Convolutional layers
• 2 MaxPooling layers
• Dropout layers for regularization
• Fully connected Dense layers
• Output layer with softmax activation for 7 classes

Input -> Conv2D -> MaxPooling -> Conv2D -> MaxPooling -> Conv2D -> Flatten -> Dense -> Output

Input shape: (48, 48, 1) grayscale. Class order (index -> label), consistent across the working models:

0 angry, 1 disgust, 2 fear, 3 happy, 4 neutral, 5 sad, 6 surprise

Trained in three stages, each with its own notebook and saved model:

• Initial — baseline, fewer epochs (INITIAL_TRAINING.ipynb → initialfacialemotionmodel.*)
• Intermediate — improved tuning & accuracy (TRAINING.ipynb → facialemotionmodel.*)
• Final (recommended) — best accuracy (FINAL_TRAINING.ipynb → finalfacialemotionmodel.*)

FER-2013 (Facial Expression Recognition 2013)

• Source: Kaggle
• 35,887 grayscale images (48x48), 7 emotions labeled
• Split: 28,709 training / 3,589 validation / 3,589 test
• Bundled in this repo under images/train/ and images/test/

---

📊 Evaluation Results

Per-stage evaluation artifacts are included in the repo root:

• Confusion matrices: Confusion_matrix_*.png / confusion_matrix_*.csv
• Classification reports: classification_report_*.txt
• Accuracy curves: accuracy_curve_*.png
• Loss curves: loss_curve_*.png

---

🛠️ Setup & Installation

1️⃣ Clone the repository

git clone https://github.com/lokeshkumar80/Facial_Emotion_Recognition.git
cd Facial_Emotion_Recognition

2️⃣ Install dependencies

Option A — pip + venv (recommended, tested)

python -m venv FERvenv
# Windows:      FERvenv\Scripts\activate
# macOS/Linux:  source FERvenv/bin/activate

pip install -r requirements.txt

Tested with Python 3.10–3.12, tensorflow==2.20.0 (pulls Keras 3.15.1 and h5py 3.16.0 automatically) and numpy==1.26.4. Linux GUI/webcam extras if needed: sudo apt install libgl1 libglib2.0-0.

Option B — Conda

conda env create -f environment.yml
conda activate facial_emotion_env   # matches the pip-tested setup; see Known issues below

---

▶️ How to Run

Training — the training pipelines are notebooks: INITIAL_TRAINING.ipynb, TRAINING.ipynb, and FINAL_TRAINING.ipynb (the last one produces the recommended FINAL model). Open them in Jupyter/VS Code and run top to bottom. Each notebook handles preprocessing, model creation, training, saving weights/model, and generating the evaluation artifacts above.

Real-time webcam detection

python realtimedetection.py   # press q to quit

Needs a physical webcam and a display — run it locally, not headless/remote. Uses the FINAL model (finalfacialemotionmodel.json + finalfacialemotionmodel.weights.h5) by default; see the ACTIVE_MODEL setting near the top of the script to switch models.

---

🤗 Hugging Face deployment

• Model repo: lokeshkumar79/facial-emotion-recognition — the FINAL model (finalfacialemotionmodel.keras) plus a model card with a loading example.
• Space: lokeshkumar79/facial-emotion-recognition — a Gradio app (hf_deploy/space_repo/) that downloads the model from the repo above, runs OpenCV face detection, and classifies emotion from an uploaded photo or webcam snapshot. Runs on the free ZeroGPU (zero-a10g) hardware tier.

Deployment source files live in hf_deploy/ — model_repo/ holds the model card, space_repo/ holds the Space's app.py and requirements.txt. To push updates:

hf upload lokeshkumar79/facial-emotion-recognition finalfacialemotionmodel.keras finalfacialemotionmodel.keras --type model
hf upload lokeshkumar79/facial-emotion-recognition hf_deploy/space_repo . --type space

---

🛠️ Technologies Used

Python 3.10+ · TensorFlow / Keras · NumPy · OpenCV · Matplotlib · Jupyter Notebook · Gradio · Hugging Face Hub

---

⚠️ Known issues

• requirements.txt: fixed — the previous h5py==3.7.0 pin conflicted with tensorflow==2.20.0; TF now picks a compatible h5py automatically.
• Do not set ACTIVE_MODEL = "BASE" in realtimedetection.py — facialemotionmodel.json describes a 3-channel (48,48,3) architecture that doesn't match facialemotionmodel.weights.h5 and is incompatible with the script's grayscale preprocessing. If you need the BASE model, load facialemotionmodel.keras instead (self-consistent).
• environment.yml has some inconsistencies with the tested setup above (Python version, a stray Keras version comment, env name) — prefer the requirements.txt + FERvenv instructions if you hit issues with Conda.

Troubleshooting

NumPy 1.x vs 2.x / TensorFlow version errors — reinstall from the pinned versions: pip install -r requirements.txt.

Model load error — load via JSON + weights explicitly:

from tensorflow.keras.models import model_from_json

with open("finalfacialemotionmodel.json") as f:
    model_json = f.read()

model = model_from_json(model_json)
model.load_weights("finalfacialemotionmodel.weights.h5")

Or load the self-contained .keras file directly (works for all three models):

from tensorflow.keras.models import load_model
model = load_model("finalfacialemotionmodel.keras")

---

📄 License

MIT — see LICENSE.

---

📬 Contact

For improvements or issues, open an Issue or Pull Request on GitHub.