🚀 MLOps Sentiment Analysis Pipeline

An end-to-end, production-ready MLOps pipeline for Sentiment Analysis and Text Classification. This repository demonstrates standard modern machine learning operations, including data version control (DVC), experiment tracking (MLflow), containerization (Docker), API deployment (FastAPI), and automated continuous integration/continuous deployment (CI/CD via GitHub Actions).

---

📊 Pipeline Overview

The complete machine learning lifecycle—from raw data ingestion to model serving—is managed and versioned systematically.

The core architecture spans across the following stages:

1. Data Preprocessing: Cleaning, tokenizing, and standardizing text features.
2. DVC Pipeline Execution: Defining modular pipeline stages and tracking data lineage.
3. Experiment Tracking: Utilizing MLflow to log parameters, performance metrics, and serialization of final model artifacts.
4. API Serving: Building a high-performance REST API with FastAPI to serve predictions in real-time.
5. Containerization & Deployment: Packaging the environment into a lightweight Docker container and hosting on Docker Hub.
6. CI/CD Automation: Triggering automated builds and pushes to Docker Hub upon updates to the codebase.

---

🛠️ Technology Stack

• Core Logic & ML: Python, Scikit-Learn (TfidfVectorizer, LogisticRegression), Pandas, Joblib
• Pipeline & Lineage: DVC (Data Version Control)
• Experiment Tracking: MLflow
• API Serving: FastAPI, Uvicorn, Pydantic
• Containerization: Docker (Docker Hub Repository)
• Automation: GitHub Actions (CI/CD Workflow)

---

📁 Repository Structure

├── .github/workflows/
│   └── ci-cd.yml                # GitHub Actions workflow for automatic Docker build & push
├── backend/
│   ├── app/
│   │   ├── main.py              # FastAPI server entry point and prediction routes
│   │   ├── model_loader.py      # Module to load the serialized sentiment model
│   │   └── schemas.py           # Pydantic request models
│   └── models/                  # Symlink or directory containing the model for the API
├── data/
│   ├── Reddit_Data.csv          # Raw sentiment dataset
│   └── processed_Reddit_Data.csv # Cleaned dataset output from preprocess stage
├── models/
│   └── sentiment_model.pkl      # Serialized trained model pipeline
├── src/
│   ├── preprocess.py            # Text cleaning and preprocessing script
│   └── train.py                 # Training script with MLflow tracking and joblib export
├── Dockerfile                   # Docker instruction set for the FastAPI application
├── dvc.yaml                     # DVC pipeline stages and dependency definition
├── params.yaml                  # Model training hyperparameters
├── requirements.txt             # Python dependencies
└── README.md                    # Project documentation (this file)

---

⚡ Getting Started

1. Prerequisites & Environment Setup

First, clone the repository and set up a virtual environment:

# Clone the repository
git clone <repository_url>
cd extension

# Create and activate a virtual environment
python -m venv venv
# On Windows (PowerShell):
.\venv\Scripts\Activate.ps1
# On Linux/macOS:
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

---

2. Running the MLOps Pipeline (DVC)

Our data preprocessing and model training stages are managed via DVC. This ensures reproducibility and tracks changes in both code and configuration.

Pipeline parameters are configured in params.yaml:

train:
  max_features: 5000
  test_size: 0.2
  random_state: 42

To run/reproduce the complete pipeline:

dvc repro

This executes:

1. Preprocess Stage: Cleans raw dataset inputs to produce standard structured features.
2. Train Stage: Splits the dataset, vectorizes features using TF-IDF, trains a Logistic Regression classifier, logs tracking data via MLflow, and serializes the model.

---

3. Experiment Tracking with MLflow

All experiment parameters, training runs, evaluation metrics (Accuracy, Precision, Recall, F1-Score), and serialized model artifacts are automatically tracked.

To launch the MLflow User Interface and compare your runs:

mlflow ui

Open your browser and navigate to http://localhost:5000 to view the comprehensive dashboard.

---

4. Running the Serving API (FastAPI)

Once the model is successfully trained and saved into models/sentiment_model.pkl, you can spin up the local prediction server:

# Navigate to the backend directory
cd backend

# Start the FastAPI server using Uvicorn
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload

API Endpoints

• GET /: Health check.
• POST /predict: Real-time sentiment prediction.

Example Request:

curl -X POST "http://localhost:8000/predict" \
     -H "Content-Type: application/json" \
     -d '{"text": "MLOps makes machine learning deployments incredibly smooth and reliable!"}'

Example Response:

{
  "text": "MLOps makes machine learning deployments incredibly smooth and reliable!",
  "sentiment": "Positive"
}

---

🐳 Docker Deployment & CI/CD

This application is fully containerized for simplified orchestration and deployment.

Pulling from Docker Hub

The pre-built Docker image for this pipeline is hosted on Docker Hub. You can pull the image directly:

docker pull grog68/mlops:latest

Docker Hub Repository URL: https://hub.docker.com/repository/docker/grog68/mlops/general

Building Locally

To build the Docker image locally:

docker build -t grog68/mlops:latest .

Running the Container

Start the FastAPI container, mapping the internal port 8000 to your host port 8000:

docker run -p 8000:8000 grog68/mlops:latest

Continuous Integration & Deployment (CI/CD)

Any push or pull request to the main or master branches triggers our automated GitHub Actions workflow (.github/workflows/ci-cd.yml).

The pipeline automatically:

1. Sets up Docker Buildx and QEMU.
2. Authenticats with Docker Hub using repository secrets.
3. Extracts standardized metadata and Git SHA tags.
4. Builds the production-ready Docker image.
5. Pushes the optimized image directly to the Docker Hub repository.