🧠 AI Fashion Tagger – Meesho Visual Taxonomy

⏱ Duration: Jan 25 – Apr 25
🔗 GitHub: GitHub Repo
🔗 Website link: Live Demo

---

🧩 The Fashion Attribution Challenge

Manual fashion tagging is a billion-dollar bottleneck for fashion e-commerce platforms like Meesho. With ever-expanding product catalogs, tagging new items accurately and consistently has become unscalable.

Manual tagging struggles with:

• 🐌 Speed Bottleneck: Products can take days to become searchable due to slow manual throughput (taking ~1 minute per product).
• 📈 Scalability Nightmare: Tagging needs grow linearly with SKUs, causing unsustainable operational costs.

---

🚀 What We Have Done (Our Impact)

We have built a robust, full-stack web application to auto-tag fashion product images using a deep learning pipeline, featuring real-time predictions and scalable REST APIs.

📈 Performance & Impact Highlights

• Speed: Inference time is < 1s per image, dramatically reducing the time-to-platform for new products.
• Scale: Handles 5 - 10 fashion attributes across 5 distinct clothing categories in a single pass.
• Accuracy: Achieved a 73% harmonic F1 score on the evaluation metric.
• Session-based Tracking: Users can upload multiple images in a session, and the backend tracks their predictions efficiently.

---

💡 What is it Solving Differently?

Most traditional tagging systems rely on slow human-in-the-loop pipelines or basic single-label image classification models that can only predict a product's main category.

Our solution innovates by:

1. Multi-Head Deep Learning Architecture: Instead of multiple models for different attributes, we use a single EfficientNet-b0 backbone with 10 custom classification heads. This allows the model to predict the main category and multiple fine-grained attributes (like color, pattern, fit, material, neckline) simultaneously.
2. Dynamic Slot Mapping: We use custom parquet and JSON mappings (category_attributes.parquet, category_slot_to_labels.json) to dynamically map specific attribute predictions to the correct category slots, ensuring predictions are highly context-aware for each specific clothing type.
3. End-to-End Automation: From the sleek React frontend to the PyTorch inference engine in the Django backend, the entire pipeline is automated. Users simply upload images, and the system handles the heavy lifting, saving processed data seamlessly.

---

🛠 Tech Stack

Frontend

• Framework: React (via Vite) with TypeScript
• Styling: Tailwind CSS for utility-first styling
• UI Components: shadcn/ui & Radix UI primitives for accessible, modern interfaces
• Routing & State: React Router DOM, React Query, and Context-based state management
• Build Tool: Vite (Lightning fast HMR and optimized builds)

Backend

• Framework: Django & Django REST Framework (DRF)
• Database: SQLite (Default)
• Machine Learning: PyTorch & Torchvision
• Data Processing: Pandas, PyArrow, FastParquet
• Server: Gunicorn & Whitenoise (for static file serving in production)

Machine Learning Model

• Base Architecture: EfficientNet_b0 (Pre-trained)
• Customization: Modified classifier with a multi-head dense layer setup to predict across 10 distinct attribute slots.

---

🌐 System Architecture

Frontend (React + Tailwind) 
       │
       ▼ (REST API / Multi-part form data)
       │
Backend (Django REST Framework)
       │
       ├─► Session Management (Tracks user uploads)
       │
       ▼
Model Server (PyTorch - EfficientNet-b0)
       │
       ├─► Feature Extraction (Backbone)
       ├─► Multi-head Classification (10 Heads)
       │
       ▼
Attribute Formatting (Pandas / Parquet / JSON Mapping)
       │
       ▼
Response (JSON Output & Predictions Data)

---

⚙️ Local Development Setup

Backend

cd backend
python -m venv venv
source venv/bin/activate  # On Windows use: venv\Scripts\activate
pip install -r requirements.txt
python manage.py runserver

Frontend

cd meesho-version-re2
npm install
npm run dev