PolyGLUE: Multi-Task Text Engine

Siamese BiRNN + DistilBERT multi-task NLP system for sentiment analysis, paraphrase detection, duplicate question identification, and natural language inference, served with FastAPI and a sleek dark-mode web UI.

---

Overview

PolyGLUE is a production-ready multi-task NLP engine that trains and serves two model families simultaneously:

• Specialized Model: A Siamese Bidirectional RNN with attention pooling, trained end-to-end on 4 GLUE tasks
• Foundation Model: Fine-tuned distilbert-base-uncased with per-task classification heads

Both models are served through a unified FastAPI backend with a responsive, interactive web UI.

---

Web App Experience

Screenshots captured from a live end-user session, with both models fully loaded and serving predictions.

Hero — Landing Page

The landing page greets users with a sleek dark-mode hero, a live model status indicator ("All models loaded and ready."), and four task cards for instant navigation.

---

SST-2 — Sentiment Analysis

Input: "This movie was absolutely fantastic! The acting was brilliant and the story was deeply moving." Both models agree - Positive - with the BiRNN at 100.0% confidence and DistilBERT at 99.1%.

---

MRPC — Paraphrase Detection

Sentence A: "The dog chased the cat around the yard." vs Sentence B: "A cat was being pursued by a dog in the garden." Both models correctly predict Paraphrase - BiRNN at 100.0%, DistilBERT at 90.3%.

---

QQP — Duplicate Question Detection

Q1: "How do I learn machine learning?" vs Q2: "What is the best way to study machine learning?" Both models flag this as Duplicate - BiRNN at 97.0%, DistilBERT at 87.6%.

---

MNLI — Natural Language Inference

Premise: "A man is playing soccer in the park." / Hypothesis: "A man is outdoors." The BiRNN predicts Entailment at 82.2% confidence. The three-way task selector (Entailment / Neutral / Contradiction) is visible in the form.

---

Swagger API Docs

The auto-generated OpenAPI docs at /docs expose the full schema for both endpoints with interactive try-it-out support.

---

Supported Tasks

Task	Description	Labels
SST-2	Movie review sentiment analysis	Positive / Negative
MRPC	Microsoft Research Paraphrase Corpus	Paraphrase / Not Paraphrase
QQP	Quora duplicate question detection	Duplicate / Not Duplicate
MNLI	Multi-Genre Natural Language Inference	Entailment / Neutral / Contradiction

Dataset source: GLUE Benchmark on HuggingFace

---

Architecture

Specialized Model — Siamese BiRNN

Sentence A ──► Embedding ──► 2-layer BiRNN ──► Attention Pooling ──► u
                                                                         ──► [u, v, |u-v|, u*v] ──► Task Head
Sentence B ──► Embedding ──► 2-layer BiRNN ──► Attention Pooling ──► v

• Embedding: Word2Vec-initialized, dimension 200, fine-tuned during training
• Encoder: 2-layer bidirectional RNN (hidden size 256), shared weights for A and B
• Pooling: Attention-weighted sum (PAD tokens masked out)
• Pair tasks (MRPC, QQP, MNLI): head input is the interaction vector [u, v, |u-v|, u*v] (dim 2048)
• Single-sentence tasks (SST-2): head input is u only (dim 512)
• Heads: Linear(dim, 128) → ReLU → Dropout → Linear(128, num_classes)

Foundation Model — DistilBERT

• Backbone: distilbert-base-uncased (66M parameters)
• Sentence pairs concatenated with [SEP] and fed as a single sequence
• 4 independent classification heads fine-tuned on top of the [CLS] representation

---

Project Structure

PolyGLUE Multi-Task Text Engine/
├── assets/                     # Web UI
│   ├── index.html              # Single-page application
│   ├── style.css               # Dark-mode glassmorphic design
│   └── app.js                  # Task switching, fetch, result rendering
├── checkpoints/                # Trained model artifacts (gitignored)
│   ├── best_multitask_model.pt # Specialized BiRNN weights
│   ├── best_foundation_model.pt
│   ├── embedding_matrix.npy    # Word2Vec embedding matrix
│   └── vocab.json              # Word-to-index vocabulary
├── configs/
│   └── default.yaml            # All hyperparameters and paths
├── src/
│   ├── api/                    # FastAPI app, routes, schemas
│   ├── config/                 # Pydantic settings loader
│   ├── core/                   # Custom exception hierarchy
│   ├── data/                   # HuggingFace dataset loaders
│   ├── evaluation/             # Accuracy and macro-F1 metrics
│   ├── inference/              # Predictor wrappers (Siamese + Foundation)
│   ├── models/                 # BiRNN and DistilBERT model classes
│   ├── preprocessing/          # Regex tokenizer and vocabulary builder
│   ├── services/               # Model service singleton
│   ├── training/               # Trainer, MultiTaskDataset, FoundationDataset
│   ├── utils/                  # Logging, device detection
│   └── main.py                 # CLI entrypoint (train / serve)
├── tests/                      # 23 pytest unit tests
├── requirements.txt
└── .gitignore

---

Setup

Prerequisites

• Python 3.10+
• pip

Install Dependencies

pip install -r requirements.txt

Required Files in checkpoints/

The following files are not tracked by git (large binary files). Place them in the checkpoints/ directory before serving:

File	Description
best_multitask_model.pt	Trained Siamese BiRNN weights
best_foundation_model.pt	Fine-tuned DistilBERT weights
embedding_matrix.npy	Word2Vec embedding matrix (vocab_size x 200)
vocab.json	Word-to-index mapping

---

Training

Train the Siamese BiRNN specialized model:

python src/main.py train --model specialized

Fine-tune the DistilBERT foundation model:

python src/main.py train --model foundation

Key hyperparameters are in configs/default.yaml. Notable defaults:

Setting	Specialized	Foundation
Epochs	25 (early stop at patience=4)	3
Batch size	128	32
Learning rate	1e-3	2e-5
Max sequence length	80	80
Training subsample	Full	20,000 per task

---

Serving

Start the API server (default port 8003):

python src/main.py serve

Custom port:

python src/main.py serve --port 8080

Endpoint	URL
Web UI	http://localhost:8003
API docs (Swagger)	http://localhost:8003/docs
Health check	GET /api/health
Inference	POST /api/predict

---

API Usage

Health check:

curl http://localhost:8003/api/health

{
  "status": "ok",
  "specialized_ready": true,
  "foundation_ready": true,
  "message": "All models loaded and ready."
}

Single-sentence task (SST-2):

curl -X POST http://localhost:8003/api/predict \
  -H "Content-Type: application/json" \
  -d '{"task": "sst2", "text_a": "This movie was absolutely fantastic!"}'

{
  "task": "sst2",
  "specialized": { "label": "Positive", "confidence": 1.0, "low_confidence": false },
  "foundation":  { "label": "Positive", "confidence": 0.99, "low_confidence": false }
}

Pair task (MRPC):

curl -X POST http://localhost:8003/api/predict \
  -H "Content-Type: application/json" \
  -d '{
    "task": "mrpc",
    "text_a": "The dog chased the cat.",
    "text_b": "A cat was chased by a dog."
  }'

---

Testing

python -m pytest tests/ -v

23 passed in 4.35s

Tests cover: model forward passes (SST-2, MRPC, MNLI, mixed batch), Siamese head dimensionality, API validation, tokenizer edge cases, and evaluation metrics.

---

Tech Stack

Component	Technology
Specialized model	PyTorch, custom Siamese BiRNN
Foundation model	HuggingFace Transformers (DistilBERT)
Embeddings	Gensim Word2Vec
Dataset	HuggingFace Datasets (GLUE)
Backend	FastAPI + Uvicorn
Frontend	Vanilla HTML/CSS/JS
Config	Pydantic Settings + YAML
Testing	pytest