🛡️ MLOps Guard

Autonomous ML Model Monitoring, Drift Detection & Self-Healing Retraining Platform

Upload any dataset. Detect data drift in real-time. Let the AI decide when and how to retrain — automatically.

Features • Tech Stack • Architecture • Quick Start • API Docs • Comparison • Contributing

---

📖 Project Overview

MLOps Guard is a full-stack, dataset-agnostic MLOps platform that automates the most critical — and most neglected — phase of the ML lifecycle: post-deployment model health monitoring.

Most ML models degrade silently. Market conditions shift, user behavior evolves, sensor readings drift — but the model keeps predicting with stale logic. By the time anyone notices, the damage is done.

MLOps Guard solves this by combining three statistical drift detection methods (KS Test, PSI, Isolation Forest) with an adaptive decision engine that autonomously decides whether to retrain, and a self-healing pipeline that only saves the new model if it actually outperforms the old one.

Why MLOps Guard?

• The Problem: 87% of ML models never make it to production, and those that do degrade within weeks without monitoring.
• The Solution: A turnkey platform that watches your model's health, detects when the world has changed, and autonomously recalibrates — with full explainability via SHAP.
• Who It's For: Data scientists, ML engineers, small teams, research labs, and enterprises that need production ML monitoring without the complexity of Kubeflow or the cost of SaaS platforms.

Real-World Use Cases

Domain	Use Case
Finance	Detect when stock market regime changes invalidate pricing models
Healthcare	Monitor patient data distribution shifts in diagnostic AI
E-Commerce	Track when customer behavior drift degrades recommendation engines
Manufacturing	Detect sensor drift in IoT predictive maintenance systems
Cybersecurity	Identify when threat landscape changes render anomaly detection stale

---

✨ Key Features

🔬 Multi-Method Drift Detection

• Kolmogorov-Smirnov (KS) Test — Statistical hypothesis testing for distribution shifts per feature
• Population Stability Index (PSI) — Industry-standard metric for population drift (banking/finance standard)
• Isolation Forest (AI-Based) — Unsupervised anomaly detection that identifies structural outliers in new data
• Single-CSV mode — Automatically splits one dataset into reference/current windows for drift analysis
• Dual-CSV mode — Compare explicit baseline vs. production datasets

🧠 Adaptive Decision Engine

• Rolling metric analysis — Tracks R², RMSE, MAE, Accuracy, F1 over a configurable window
• Adaptive thresholds — Uses historical mean ± σ instead of hardcoded cutoffs
• Multi-signal fusion — Combines drift detection + performance degradation signals
• Confidence scoring — Each retrain decision includes HIGH/MEDIUM/LOW confidence and structured reasoning

🔄 Self-Healing Retraining Pipeline

• Autonomous retraining — Triggers automatically when drift + degradation is detected
• Model comparison gate — New model is only saved if it outperforms the incumbent
• Versioned model registry — Every trained model is saved with timestamp and version number
• Full audit trail — Every retrain event is logged with metrics, reasoning, and drift signals

🔍 SHAP Explainability (XAI)

• Global feature importance — Mean |SHAP| values across the entire dataset
• Per-prediction explanations — SHAP vectors for individual samples
• Human-friendly interpretation — Auto-generated plain-English explanations
• Pipeline-aware — Correctly handles sklearn Pipeline + ColumnTransformer feature alignment

📊 Premium Dashboard & Visualization

• Glassmorphic dark-mode UI — Built with MUI v7, Framer Motion, Recharts
• Real-time system health cards — Model status, version, drift events, performance trends
• Interactive drift charts — P-value scatter plots, PSI bar charts, distribution overlay plots with brush zoom
• AI Insights drawer — Ask the AI to explain drift in natural language with typewriter animation
• Performance trend tracking — Line charts showing model accuracy/R² evolution over retrain events

🗃️ Dataset Intelligence

• Schema auto-detection — Numeric, categorical, datetime, and ID column classification
• Messy CSV handling — Automatic cleaning of metadata rows, type coercion, and null handling
• Target column suggestion — Intelligent recommendation of prediction target columns
• Data preview — Interactive DataGrid with sortable, resizable columns

🛡️ Resilience & Error Handling

• Global Error Boundary — React class component catches rendering crashes with stack trace display
• Backend error propagation — Structured error responses with actionable messages
• Legacy model detection — Gracefully handles incompatible pre-Pipeline model formats
• Feature mismatch detection — Clear error when uploaded data doesn't match model schema

---

🏗️ Tech Stack

Backend

Technology	Role	Why This Choice
Python 3.10+	Runtime	Industry standard for ML/data science with the richest library ecosystem
FastAPI	API Framework	Async-native, automatic OpenAPI docs, fastest Python web framework
XGBoost	Primary ML Model	State-of-the-art gradient boosting with superior performance on tabular data
LightGBM	Fallback ML Model	Fast, memory-efficient alternative when XGBoost is unavailable
scikit-learn	ML Pipeline	ColumnTransformer, Pipeline, and preprocessing infrastructure
SHAP	Explainability (XAI)	Gold standard for model-agnostic feature importance explanations
SciPy	Statistical Testing	KS test implementation for distribution comparison
Pandas / NumPy	Data Processing	De facto standard for tabular data manipulation and numerical computing
Joblib	Model Serialization	Efficient serialization of sklearn Pipeline objects

Frontend

Technology	Role	Why This Choice
React 19	UI Framework	Latest stable React with concurrent features and improved rendering
Vite 7	Build Tool	Sub-second HMR, native ES modules, 10-100x faster than Webpack
Material UI (MUI) v7	Component Library	Enterprise-grade component system with dark mode and customization
Framer Motion	Animations	Spring-physics animations for premium 3D card effects and transitions
Recharts	Data Visualization	Composable, responsive chart library built on D3
Axios	HTTP Client	Promise-based HTTP with interceptors and request/response transforms
React Router v7	Navigation	Declarative routing with animated page transitions
PapaParse	CSV Parsing	Client-side CSV parsing for instant column extraction before upload
React Query	State Management	Server state synchronization with caching and background refetching

Infrastructure

Technology	Role
CORS Middleware	Cross-origin frontend-backend communication
JSON File Storage	Lightweight model history persistence (no database required)
Joblib Model Registry	Versioned .pkl model files with latest pointer

---

🏛️ System Architecture

┌──────────────────────────────────────────────────────────────┐
│                    FRONTEND (React 19 + Vite)                │
│                                                              │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐│
│  │Dashboard │ │ Dataset  │ │  Drift   │ │  Explainability  ││
│  │  Page    │ │ Explorer │ │ Monitor  │ │   (SHAP)         ││
│  └────┬─────┘ └────┬─────┘ └────┬─────┘ └────────┬─────────┘│
│       │             │            │                 │          │
│  ┌────┴─────────────┴────────────┴─────────────────┴────┐    │
│  │              API Service Layer (Axios)                │    │
│  └──────────────────────┬────────────────────────────────┘    │
└─────────────────────────┼────────────────────────────────────┘
                          │ HTTP/REST
                          ▼
┌──────────────────────────────────────────────────────────────┐
│                  BACKEND (FastAPI + Python)                   │
│                                                              │
│  ┌─────────────┐  ┌───────────────┐  ┌────────────────────┐ │
│  │  main.py    │  │preprocessing.py│  │  model_trainer.py  │ │
│  │  (10 APIs)  │  │(Data Pipeline)│  │  (XGBoost/LGBM)    │ │
│  └──────┬──────┘  └───────┬───────┘  └────────┬───────────┘ │
│         │                 │                    │             │
│  ┌──────┴──────┐  ┌───────┴───────┐  ┌────────┴───────────┐│
│  │ drift_ai.py │  │decision_engine│  │  SHAP Explainer    ││
│  │ (PSI + IF)  │  │  .py          │  │  (TreeExplainer)   ││
│  └─────────────┘  └───────────────┘  └────────────────────┘ │
│                                                              │
│  ┌──────────────────────────────────────────────────────────┐│
│  │              Model Registry & History Store              ││
│  │  models/latest_model.pkl  |  history/model_history.json  ││
│  └──────────────────────────────────────────────────────────┘│
└──────────────────────────────────────────────────────────────┘

Request Lifecycle: Drift Detection Flow

sequenceDiagram
    participant U as User
    participant FE as React Frontend
    participant API as FastAPI Backend
    participant PP as Preprocessing
    participant DA as Drift AI Module
    participant DE as Decision Engine
    participant MT as Model Trainer
    participant MR as Model Registry

    U->>FE: Upload CSV + Click "Run Drift Analysis"
    FE->>API: POST /detect-drift-single (file, target_column)
    API->>PP: clean_dataframe() + split 70/30
    API->>API: KS Test (per-feature p-values)
    API->>DA: compute_psi_report(ref, curr)
    DA-->>API: PSI scores per feature
    API->>DA: detect_ai_drift(ref, curr)
    DA-->>API: Isolation Forest anomaly fraction
    API->>DE: should_retrain(metrics, drift_signals)
    DE-->>API: Decision + confidence + reasoning
    
    alt Retrain Decision = YES
        API->>PP: prepare_data(curr_df, target)
        API->>MT: train_model(X, y, preprocessor, task_type)
        MT-->>API: new_pipeline + metrics
        API->>DE: is_new_model_better(old, new)
        alt New Model Wins
            API->>MR: save_model(pipeline) → versioned .pkl
        end
        API->>MR: append to model_history.json
    end
    
    API-->>FE: Full drift report + decision + metrics
    FE->>U: Render charts, alerts, AI explanation

Loading

Module Dependency Graph

graph TD
    A[main.py<br/>FastAPI Router] --> B[preprocessing.py<br/>Data Pipeline]
    A --> C[model_trainer.py<br/>XGBoost/LightGBM]
    A --> D[drift_ai.py<br/>PSI + Isolation Forest]
    A --> E[decision_engine.py<br/>Adaptive Logic]
    
    B --> F[scikit-learn<br/>ColumnTransformer]
    C --> G[XGBoost / LightGBM]
    C --> F
    D --> H[scikit-learn<br/>IsolationForest]
    E --> I[NumPy<br/>Rolling Stats]
    A --> J[SHAP<br/>TreeExplainer]
    
    style A fill:#3b82f6,color:#fff
    style B fill:#8b5cf6,color:#fff
    style C fill:#ef4444,color:#fff
    style D fill:#f59e0b,color:#fff
    style E fill:#10b981,color:#fff

Loading

---

🚀 Quick Start

Prerequisites

• Python 3.10 or higher
• Node.js 18+ and npm 9+
• Git

1. Clone the Repository

git clone https://github.com/YOUR_USERNAME/mlops-guard.git
cd mlops-guard

2. Backend Setup

# Navigate to backend
cd backend

# Create virtual environment
python -m venv venv

# Activate virtual environment
# Windows:
venv\Scripts\activate
# macOS/Linux:
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Start the FastAPI server
uvicorn main:app --reload --host 0.0.0.0 --port 8000

The API will be available at http://127.0.0.1:8000 with interactive docs at http://127.0.0.1:8000/docs.

3. Frontend Setup

# Navigate to frontend (in a new terminal)
cd frontend/AI\ Drift\ Detection\ System

# Install dependencies
npm install

# Start development server
npm run dev

The UI will be available at http://localhost:5173.

4. Verify Installation

# Run smoke tests (from project root)
python smoke_test.py

# Run backend stability tests
python test_backend_stability.py

---

🔌 API Reference

Base URL: http://127.0.0.1:8000

Method	Endpoint	Description
POST	/upload-dataset	Upload CSV and get a preview (first 10 rows)
POST	/analyze-dataset	Get row/column counts, dtypes, missing values
POST	/detect-schema	Auto-detect column types, target suggestions, sample values
POST	/detect-drift-single	Single-CSV drift detection (auto-split reference/current)
POST	/detect-drift	Dual-CSV drift detection (explicit reference + current files)
POST	/retrain-model	Manually retrain model on new dataset
POST	/explain	SHAP-based model explainability
POST	/explain-drift-ai	AI-generated natural language drift explanation
GET	/model-history	Retrieve full model audit log
GET	/system-status	System health: model status, versions, drift event counts

Example: Single-CSV Drift Detection

Request:

curl -X POST http://127.0.0.1:8000/detect-drift-single \
  -F "file=@production_data.csv" \
  -F "target_column=close" \
  -F "split_ratio=0.7"

Response:

{
  "dataset_drift_report": {
    "open": { "p_value": 0.0023, "ks_statistic": 0.312, "dataset_drift_detected": true },
    "volume": { "p_value": 0.4521, "ks_statistic": 0.087, "dataset_drift_detected": false }
  },
  "dataset_drift_detected": true,
  "psi_report": {
    "open": { "psi_value": 0.2841, "psi_drift_detected": true }
  },
  "psi_drift_detected": true,
  "ai_drift_result": {
    "ai_drift_detected": true,
    "anomaly_fraction": 0.2100,
    "anomaly_count": 63,
    "total_samples": 300,
    "method": "IsolationForest"
  },
  "decision": {
    "should_retrain": true,
    "confidence": "HIGH",
    "reason": "R² dropped below adaptive threshold; Dataset drift detected (KS/PSI); AI-based drift detected",
    "evidence": { "R2_current": 0.72, "R2_rolling_mean": 0.89, "R2_threshold": 0.84 }
  },
  "auto_retraining": "Drift detected → New model saved (better performance)",
  "split_info": { "total_rows": 1000, "reference_rows": 700, "current_rows": 300 }
}

Example: SHAP Explainability

Request:

curl -X POST http://127.0.0.1:8000/explain \
  -F "file=@dataset.csv" \
  -F "target_column=close"

Response:

{
  "feature_importance": {
    "volume": 0.234521,
    "open": 0.198432,
    "high": 0.156789
  },
  "sample_explanations": [
    { "volume": 0.0523, "open": -0.0341, "high": 0.0189 }
  ],
  "task_type": "regression",
  "n_features": 5,
  "feature_names": ["open", "high", "low", "volume", "adj_close"]
}

---

🌍 Environment Variables

Create a .env file in the backend directory (optional — the system works with defaults):

Variable	Required	Default	Description
UVICORN_HOST	No	0.0.0.0	Backend server bind address
UVICORN_PORT	No	8000	Backend server port
VITE_API_URL	No	http://127.0.0.1:8000	Frontend → Backend API base URL

Note: MLOps Guard is designed for simplicity — it uses file-based storage and requires no database, no cloud credentials, and no external services to run.

---

📊 Competitive Analysis

Why MLOps Guard Over Alternatives

Feature	MLOps Guard	Evidently AI	NannyML	Whylabs	AWS SageMaker Monitor
Self-healing retraining	✅ Built-in	❌ Report only	❌ Report only	❌ Alert only	⚠️ Requires Lambda
Adaptive thresholds	✅ Rolling σ	❌ Fixed	⚠️ Partial	❌ Fixed	❌ Fixed
SHAP explainability	✅ Integrated	❌ Separate	❌ None	❌ None	❌ Separate
3-method drift ensemble	✅ KS+PSI+IF	⚠️ KS+PSI	⚠️ CBPE	⚠️ KS only	⚠️ KS only
Interactive dashboard	✅ React/MUI	⚠️ Basic HTML	✅ Good	✅ SaaS	⚠️ Console
Model comparison gate	✅ Auto	❌ Manual	❌ Manual	❌ Manual	❌ Manual
Zero dependencies infra	✅ File-based	✅ Local	✅ Local	❌ Cloud	❌ AWS
Setup time	< 5 min	~10 min	~15 min	~30 min	~2 hours
Cost	Free	Free/Paid	Free/Paid	Paid	Paid
Dataset-agnostic	✅ Any CSV	✅	✅	⚠️	⚠️
AI drift explanation	✅ NLP	❌	❌	❌	❌

Key Architectural Advantages

1. Closed-Loop Architecture — Most tools stop at "drift detected." MLOps Guard continues to retrain, compare, and only deploy if the new model is better.
2. Adaptive Intelligence — Thresholds evolve with your model's history instead of relying on arbitrary cutoffs.
3. Full Transparency — Every decision is logged with evidence, reasoning, and confidence levels. Auditable by design.
4. Zero Infrastructure Overhead — No Kubernetes, no databases, no cloud accounts required. Run it on a laptop.

---

📁 Repository Structure

mlops-guard/
├── backend/
│   ├── main.py                 # FastAPI app — 10 REST endpoints
│   ├── preprocessing.py        # Dataset-agnostic data pipeline
│   ├── model_trainer.py        # XGBoost/LightGBM training + versioning
│   ├── drift_ai.py             # PSI + Isolation Forest drift detection
│   ├── decision_engine.py      # Adaptive retrain decision logic
│   ├── requirements.txt        # Python dependencies
│   ├── models/                 # Versioned model registry (.pkl files)
│   └── history/                # Model audit log (JSON)
│
├── frontend/
│   └── AI Drift Detection System/
│       ├── src/
│       │   ├── App.jsx         # Root component + router + theme
│       │   ├── ErrorBoundary.jsx  # Global crash handler
│       │   ├── index.css       # Glassmorphism background + animations
│       │   ├── pages/
│       │   │   ├── Dashboard.jsx      # System health overview
│       │   │   ├── Dataset.jsx        # CSV upload + schema detection
│       │   │   ├── Drift.jsx          # Drift analysis + AI insights
│       │   │   ├── Retraining.jsx     # Manual model calibration
│       │   │   ├── Explainability.jsx # SHAP visualization
│       │   │   └── History.jsx        # Model audit log timeline
│       │   ├── services/
│       │   │   └── api.js      # Axios API service layer
│       │   └── components/
│       │       ├── Sidebar.jsx
│       │       └── Navbar.jsx
│       ├── package.json
│       └── vite.config.js
│
├── datasets/
│   ├── sp500.csv               # Sample S&P 500 dataset
│   ├── raw/                    # Raw market data (fundamentals, prices)
│   └── processed/              # Cleaned/transformed datasets
│
├── smoke_test.py               # Backend API smoke tests
├── test_backend_stability.py   # Preprocessing unit tests
├── .gitignore
├── README.md
├── CONTRIBUTING.md
├── SECURITY.md
├── CODE_OF_CONDUCT.md
└── LICENSE

---

🧪 Testing

Backend Smoke Tests

# Start the backend first, then:
python smoke_test.py

Tests: /upload-dataset preview response, /detect-schema column type detection.

Backend Stability Tests

python test_backend_stability.py

Tests:

• Small dataset ID detection (prevents false-positive ID column skipping)
• Large dataset real ID detection (correctly identifies auto-increment columns)
• Messy CSV cleaning (metadata rows, type coercion, empty row handling)

---

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for detailed guidelines.

Quick Start:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes and commit: git commit -m 'feat: add amazing feature'
4. Push to your branch: git push origin feature/amazing-feature
5. Open a Pull Request

---

🔒 Security

For security vulnerabilities, please see SECURITY.md. Do not open public issues for security bugs.

---

📜 License

This project is licensed under the MIT License — see the LICENSE file for details.

---

🙏 Acknowledgments

• SHAP — SHapley Additive exPlanations by Scott Lundberg
• XGBoost — Scalable gradient boosting by Tianqi Chen
• FastAPI — Modern Python web framework by Sebastián Ramírez
• Material UI — React component library
• Recharts — Composable charting library

---

Built with ❤️ for the ML community
If this project helps you, consider giving it a ⭐