ML-Based Performance Regression Detection System

Automated software application performance regression detection system using Machine Learning (XGBoost Regressor + Classifier), Statistical Baselines, SHAP Explainability, FastAPI, SQLAlchemy, Streamlit, and Continuous Learning MLOps pipelines.

---

🚀 Key Features

1. Dual Machine Learning Engine:
   • XGBoost Regressor: Predicts expected performance metric (e.g. baseline response time) given system metrics context.
   • XGBoost Classifier: Classifies whether a version exhibits a regression.
2. Statistical Baseline Detector:
   • Z-score and percentage change fallback detection mechanism.
3. Multi-Metric Severity Classifier:
   • Classifies regressions into LOW, MEDIUM, HIGH, or CRITICAL across 7 key metrics: response_time, cpu_usage, memory_usage, error_rate, throughput, latency, and request_count.
4. SHAP Explainability:
   • Identifies specific contributing factors (e.g., CPU ↑ 55%, Error Rate ↑ 220%).
5. Continuous Learning Loop:
   • Automated model retraining on new release observations with challenger vs. production model promotion.
6. FastAPI Backend:
   • Interactive REST API with /predict, /detect-regression, /models, and /health endpoints.
7. Streamlit Web Dashboard:
   • Interactive visual graphs (Plotly) and version analyzer UI.
8. DevOps Ready:
   • Docker, Docker Compose, and GitHub Actions CI.

---

🛠️ Project Architecture

                 Application / Load Tests / CI
                              │
                              ▼
                      Data Generator / CSV
                              │
                              ▼
                     Data Preprocessing
                 (Cleaning, Outliers, Bounds)
                              │
                              ▼
                    Feature Engineering
                 (Rolling Stats, Z-scores)
                              │
                              ▼
            ┌──────────────────────────────────┐
            │       ML Detection Engine        │
            │  XGBoost Regressor + Classifier  │
            │               +                  │
            │   Statistical Baseline Engine    │
            └─────────────────┬────────────────┘
                              │
                              ▼
                     Regression Engine
                   + SHAP Explainability
                              │
               ┌──────────────┴──────────────┐
               ▼                             ▼
         FastAPI REST API            Streamlit Dashboard

---

⚙️ Quickstart Guide

1. Installation

Clone the repository and install dependencies:

git clone https://github.com/your-username/regression-detection-system.git
cd regression-detection-system
pip install -r requirements.txt

2. Generate Synthetic Data & Train Models

# 1. Generate realistic synthetic metrics across versions v1.0 -> v1.9
python src/data/generator.py

# 2. Train XGBoost Regressor and Classifier models
python src/models/train.py

3. Run Unit Tests

pytest tests/ -v

4. Launch FastAPI Server

uvicorn app.main:app --reload --port 8000

• Open Swagger API Documentation: http://localhost:8000/docs

5. Launch Streamlit Web Dashboard

streamlit run dashboard/app.py

• Access Dashboard UI: http://localhost:8501

---

🐳 Running with Docker Compose

docker-compose up --build

---

📋 API Endpoints

• GET /health — Application and model health status
• POST /predict — Model performance baseline prediction
• POST /detect-regression — Full multi-metric version regression analysis
• GET /regressions — List detected historical regression events
• GET /models — Model registry list
• POST /models/retrain — Trigger continuous retraining loop