🏦 Banking Intent Classification — QLoRA Fine-tuned Mistral-7B

Fine-tuned Mistral-7B-Instruct using QLoRA (4-bit quantization + LoRA) to classify banking customer queries into 77 intent categories with 91.28% accuracy on the Banking77 benchmark. Integrated into a realistic UAE banking chatbot UI.

---

📋 Table of Contents

• Overview
• Results
• Tech Stack
• Project Structure
• Quick Start
• Training
• API Usage
• Dataset
• How It Works
• Author

---

🎯 Overview

This project demonstrates parameter-efficient fine-tuning (PEFT) of a large language model for real-world banking NLP. Instead of training all 7 billion parameters, QLoRA trains only 13.6M parameters (0.36%) while achieving competitive accuracy — making it feasible on a free Google Colab T4 GPU.

The system works like this:

1. Customer types a banking query into the NovaPay chatbot
2. Query is sent to a Flask API running the fine-tuned Mistral-7B model
3. Model classifies the intent in real time
4. Customer receives a friendly, context-aware response

Key achievements:

• 91.28% accuracy on Banking77 test set (3,080 examples)
• 92.82% weighted F1-score
• Training time: ~3 hours on a single T4 GPU (free Colab tier)
• LoRA adapter: only 53MB (vs ~14GB for full model weights)
• Only 0.36% of parameters are trainable

---

📊 Results

Metric	Score
Accuracy	91.28%
F1 Weighted	92.82%
F1 Macro	56.35%
Training Loss (Epoch 3)	0.3491
Validation Loss (Best)	0.4316
Samples Evaluated	1,353
Total Test Set	3,080

Training Progression:

Epoch	Train Loss	Val Loss
1	0.4887	0.4513
2	0.4260	0.4316	← Best model saved
3	0.3491	0.4475

Sample Predictions:

Query	Predicted Intent	Correct
"I lost my credit card yesterday"	lost_or_stolen_card	✅
"What is the exchange rate for USD?"	exchange_rate	✅
"My card payment was charged twice"	transaction_charged_twice	✅
"How do I activate my new card?"	activate_my_card	✅
"My top up failed"	top_up_failed	✅
"I want to cancel my transfer"	cancel_transfer	✅
"Can I use Apple Pay?"	apple_pay_or_google_pay	✅
"I forgot my PIN"	pin_blocked	✅

---

🛠️ Tech Stack

Component	Tool	Version
Base Model	Mistral-7B-Instruct-v0.2	v0.2
Fine-tuning Method	QLoRA (LoRA + 4-bit NF4)	—
PEFT Library	HuggingFace peft	0.10.0
Quantization	bitsandbytes	0.46.1
Training Framework	HuggingFace transformers + trl	4.40.0
Dataset	PolyAI/banking77	—
Evaluation	scikit-learn	≥1.3.0
API	Flask + Flask-CORS	≥2.3.0
Frontend	HTML + CSS + Vanilla JS	—
Training Environment	Google Colab T4 GPU (16GB)	—
Python	Python 3.11	3.11

---

📁 Project Structure

banking-intent-classification/
│
├── notebooks/
│   └── train_qlora.ipynb          # Google Colab training notebook (run on GPU)
│
├── src/
│   ├── __init__.py
│   ├── config.py                  # Central configuration — all hyperparams
│   ├── dataset.py                 # Data loading & prompt formatting
│   ├── model.py                   # QLoRA model loading (train + inference)
│   ├── train.py                   # Training script (GPU only)
│   ├── inference.py               # Prediction pipeline with fuzzy matching
│   └── evaluate.py                # Evaluation metrics & confusion analysis
│
├── app/
│   ├── api.py                     # Flask REST API (5 endpoints)
│   └── index.html                 # NovaPay banking chatbot UI
│
├── models/
│   └── qlora_adapter/             # Saved LoRA adapter (~53MB)
│       ├── adapter_model.safetensors
│       ├── adapter_config.json
│       └── tokenizer files...
│
├── results/
│   └── evaluation_results.json    # Saved evaluation metrics
│
├── assets/
│   └── demo.png                   # Demo screenshot
│
├── requirements.txt
├── .env.example
├── .gitignore
└── README.md

---

🚀 Quick Start

Prerequisites

• Python 3.11+
• CUDA GPU (for training and inference)
• Google Colab or Kaggle (recommended for training)

1. Clone the repository

git clone https://github.com/nithinraj49/banking-intent-classification.git
cd banking-intent-classification

2. Install dependencies

pip install -r requirements.txt

⚠️ bitsandbytes requires a CUDA GPU. For local testing without GPU, the Flask API runs in demo mode automatically.

3. Download the LoRA adapter

Download the trained adapter and place files in models/qlora_adapter/:

models/qlora_adapter/
├── adapter_model.safetensors   # ~53MB — trained LoRA weights
├── adapter_config.json
├── tokenizer_config.json
├── tokenizer.model
├── tokenizer.json
└── special_tokens_map.json

4. Run the Flask API

python app/api.py

Expected output:

✅ API ready at http://localhost:5000
   POST /predict    — classify intent
   GET  /health     — health check
   GET  /intents    — list all 77 intents
   GET  /categories — list categories

5. Open the frontend

# Serve locally (required to avoid CORS issues)
python -m http.server 8080

Open: http://localhost:8080/app/index.html

---

🎓 Training

Training requires a GPU. Use Google Colab (free T4) or Kaggle (free P100).

Open the notebook

notebooks/train_qlora.ipynb → Open in Google Colab → Set T4 GPU runtime → Run all

QLoRA Configuration

# 4-bit quantization — reduces model from ~14GB to ~5GB VRAM
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",            # NormalFloat4 — best for LLMs
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,       # Extra ~0.4 bits saved
)

# LoRA adapters — only train attention projection layers
lora_config = LoraConfig(
    r=16,                                  # Rank
    lora_alpha=32,                         # Scaling factor
    lora_dropout=0.05,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    task_type=TaskType.CAUSAL_LM,
)

Training Arguments

TrainingArguments(
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,        # Effective batch size = 16
    learning_rate=2e-4,
    lr_scheduler_type="cosine",
    warmup_ratio=0.03,
    fp16=True,
    optim="paged_adamw_8bit",             # Memory-efficient optimizer
    gradient_checkpointing=True,          # Trade compute for memory
)

GPU Memory Usage

Base model (4-bit quantized) : ~4.7GB VRAM
Training (with gradients)    : ~8-10GB VRAM
Available on free T4          : 15.6GB
Adapter saved to Drive        : ~53MB

---

🔌 API Usage

Predict single intent

curl -X POST http://localhost:5000/predict \
  -H "Content-Type: application/json" \
  -d '{"query": "I lost my credit card"}'

Response:

{
  "intent": "lost_or_stolen_card",
  "confidence": 0.91,
  "category": "Card Management",
  "query": "I lost my credit card",
  "latency_ms": 245
}

Batch predict

curl -X POST http://localhost:5000/batch \
  -H "Content-Type: application/json" \
  -d '{
    "queries": [
      "I lost my card",
      "Check exchange rate",
      "Activate my card"
    ]
  }'

All endpoints

Method	Endpoint	Description
POST	/predict	Classify single query
POST	/batch	Classify up to 50 queries
GET	/health	API health check
GET	/intents	List all 77 intent labels
GET	/categories	List intent categories

---

📦 Dataset

PolyAI/banking77 — a benchmark dataset for banking intent detection.

Split	Examples
Train	10,003
Test	3,080
Total	13,083
Intent Classes	77

Intent Categories:

Category	Example Intents
Card Management	lost_or_stolen_card, activate_my_card, card_arrival
Transfers	cancel_transfer, failed_transfer, pending_transfer
Top Up	top_up_failed, top_up_limits, topping_up_by_card
Exchange & Currency	exchange_rate, exchange_via_app, fiat_currency_support
Cash & ATM	atm_support, cash_withdrawal_charge, declined_cash_withdrawal
Account & Identity	verify_my_identity, edit_personal_details, terminate_account
Refunds	request_refund, refund_not_showing_up
Payments	transaction_charged_twice, card_payment_not_recognised

---

⚙️ How It Works

Architecture

Customer Query (text)
        ↓
Prompt Template:
"Classify the following banking customer query 
 into one of the 77 intent categories.
 Respond with ONLY the intent label.
 
 Query: {query}
 Intent:"
        ↓
Mistral-7B-Instruct (4-bit quantized)
+ QLoRA Adapter (13.6M trainable params)
        ↓
Generated token: "lost_or_stolen_card"
        ↓
Fuzzy label matching → Valid intent (1 of 77)
        ↓
Friendly response → Customer sees natural language

Why Generative Classification?

Instead of a traditional classifier head (softmax over 77 classes), we prompt Mistral-7B to generate the intent label as text. This approach:

• Leverages the model's rich language understanding
• Handles paraphrasing and ambiguous queries better
• Requires no custom classification head
• Produces interpretable, debuggable outputs

Why QLoRA?

Approach	VRAM Required	Trainable Params	Adapter Size
Full fine-tuning	~56GB	7B (100%)	~14GB
LoRA (fp16)	~14GB	13.6M (0.36%)	~53MB
QLoRA (4-bit)	~5GB	13.6M (0.36%)	~53MB

QLoRA makes fine-tuning Mistral-7B accessible on a free Colab T4 GPU.

---

🔧 Environment Variables

Copy .env.example to .env and configure:

cp .env.example .env

# HuggingFace token (optional for public models)
HF_TOKEN=your_hf_token_here

# Flask settings
API_HOST=0.0.0.0
API_PORT=5000
API_DEBUG=False

---

📄 License

MIT License — free to use for personal and commercial projects.

---

Built with Mistral-7B · QLoRA · HuggingFace · Flask · Banking77
Fine-tuning LLMs for real-world banking NLP