DocMind is an intelligent document analysis platform that allows users to upload files, ask questions, and receive precise per document answers and citations along with theme-based insights. It supports PDFs, images, Word documents, and text files, and is powered by OCR, vector search, and Groq LLM.
Upload → Ask → Analyze → Understand
DocMind was built to demonstrate how Large Language Models (LLMs) can power scalable, document-centric AI systems capable of processing multi-format, unstructured content in real-world settings.
It focuses on:
- Answering high-level, insight-oriented questions that are relevant across documents—such as "What key concerns were raised?" or "What risks are commonly mentioned?"
- Delivering per-document responses, ensuring source traceability with paragraph-level citations
- Organizing outputs into thematic clusters, helping users navigate and synthesize complex document sets
- Handling scanned and text-based documents via integrated OCR and dynamic preprocessing
- Prioritizing production readiness, with modular FastAPI endpoints, Dockerized deployment, MongoDB-based metadata storage, and built-in rate limiting
DocMind is best suited for scenarios like compliance review, research synthesis, operational audits, or knowledge extraction—where interpretability and traceability across multiple documents are critical.
┌────────────────────────┐
│ Frontend UI │
│ (Upload files) │◄─────────────────────────┐
└────────────┬───────────┘ │
│ │
▼ │
┌───────────────┐ │
│ /upload API │ │
└──────┬────────┘ │
▼ │
┌─────────────────────┐ │
│Store file in uploads│ │
│ folder on backend │ │
└─────────┬───────────┘ │
│ │
▼ │
┌────────────────────────┐ │
│ Frontend UI │◄─────────────────────────│
│ (Asks question) │ │
└──────────┬─────────────┘ │
│ │
▼ │
┌────────────────────┐ │
│ /run-pipeline API │ │
└────────┬───────────┘ │
│ │
▼ │
┌────────────────────────────────────────────┐ │
│ For each uploaded document: │ │
│ Check if metadata exists in MongoDB │ │
└─────────────┬──────────────┬───────────────┘ │
│ │ │
Yes ▼ No ▼ │
┌────────────────────────┐ ┌──────────────────────────────┐ │
│ Skip preprocessing and │ │ Preprocess document: │ │
│ embedding (already in │ │ - OCR/text extraction │ │
│ vector store) │ │ - Chunking + Embedding │ │
└────────────┬───────────┘ │ - Store chunk and embeddings│ │
│ │ in metadata │ │
│ └─────────────┬────────────────┘ │
│ │ │
│ │ │
▼ ▼ │
┌────────────────────────────────────┐ │
│ Embed the user's question │ │
│ using sentence-transformers │ │
└─────────────┬──────────────────────┘ │
▼ │
┌────────────────────────────────────────────────┐ │
│ Semantic search using FAISS │ │
│ (find top-k chunks per doc based on Question) │ │
└─────────────┬──────────────────────────────────┘ │
▼ │
┌─────────────────────────────────────┐ │
│ Groq LLM: Answer Generator │ │
│ (answers using retrieved chunks) │ │
└─────────────┬───────────────────────┘ │
▼ │
┌─────────────────────────────────────┐ │
│ Groq LLM: Theme Identifier │ │
│ (extracts themes from all answers) │ │
└──────────────┬──────────────────────┘ │
▼ │
┌────────────────────────────────────────────────────┐ │
│ Response to frontend: answers + citations + themes │─────│
└─────────────────────┬──────────────────────────────┘ │
│ │
▼ │
┌────────────────────────┐ │
│ Frontend UI │ │
│ (Delete files) │ │
└────────────┬───────────┘ │
▼ │
┌──────────────┐ │
│ /delete API │ │
└──────┬───────┘ │
▼ │
┌───────────────────┐ │
│ Remove file and │ │
│ its metadata │────────────────────────────┘
└───────────────────┘
| File | Responsibility | Link |
|---|---|---|
document_routes.py |
Backend route for uploading, listing, and deleting documents from storage | View Docs |
pipeline_routes.py |
Backend route that triggers the full QA + theme extraction pipeline | View Docs |
document_preprocessor.py |
Extracts text from PDFs, images, DOCX, and TXT | View Docs |
meta_store.py |
Manages per-document metadata in MongoDB, including chunk texts and vector embeddings. Enables FAISS-based top-k semantic search using this data. | View Docs |
query_engine.py |
Uses Groq LLM to extract detailed answers and citations per document | View Docs |
theme_identifier.py |
Clusters answers into themes using Groq LLM | View Docs |
models.py |
Defines Pydantic models for API requests and responses | View Docs |
main.py |
Initializes the FastAPI app, registers routes, sets up static file serving | View Docs |
config.py |
Centralizes environment and file path configs; loads variables via .env |
View Docs |
logger.py |
Sets up structured application-wide logging | View Docs |
limiter.py |
Defines and applies a global shared rate limiter for all API routes using SlowAPI. | View Docs |
-
Document Uploading
- Upload multiple file types: .pdf, .docx, .txt, .jpg, .png
-
Smart Text Extraction
-
OCR-based parsing using Tesseract for images and scanned PDFs
-
Text-based parsing for PDF, Word, and TXT documents
-
Paragraph-aware formatting for clean chunking
-
-
Retrieval-Augmented Generation (RAG)
-
Embeds document chunks with sentence-transformers
-
Stores document chunks and embeddings as structured metadata in MongoDB, and constructs a temporary FAISS index at query time for semantic retrieval.
-
Prevents redundant processing by checking if the document is already indexed in the MongoDB metadata.
-
-
LLM-Powered QA and Insights
-
Uses Groq (OpenAI-compatible) to answer a question per document and provide citations
-
Identifies themes across all documents via prompt-engineered summarization
-
-
Paragraph-Level Citations
-
Each answer is linked back to the exact paragraph(s) that supported it
-
Enhances traceability and transparency of AI-generated responses
-
-
Selective Document Processing
- Users can choose specific uploaded documents to include in analysis
-
Modular & Extensible Backend
-
Clear separation of API routes
-
Built with FastAPI and fully containerized with Docker
-
Config-driven paths, logging, and API schema
-
-
Rate Limiting & Abuse Protection
-
Shared global request limit across all routes (100 requests/minute per IP)
-
Automatic redirect to
/rate_limit.htmlif rate limit is exceeded
-
-
Session-Based Isolation
-
Each user session is uniquely tracked using localStorage and a generated session ID
-
Uploaded files are stored in per-session directories for logical separation
-
Each session maintains its own metadata
-
Access the app at:
Best viewed on desktop.
Metadata stored in a managed MongoDB instance hosted on Railway.
Upload your documents, ask a question, and view:
- Per Document answers
- Paragraph-Level Citations
- Theme-based insights across documents
You can run DocMind either via Docker (recommended) or directly using Uvicorn with a local Python environment.
git clone https://github.com/larrymargerum01/Document-IntelligenceSet the following in your .env file.
GROQ_API_KEY=your_groq_key_hereMONGO_URL=your_public_mongo_connection_string
docker build -t docmind .
docker-compose up -dVisit http://localhost:8000/ in your browser.
conda create -n docmind python=3.10
conda activate docmindpip install -r requirements.txtSet GROQ_API_KEY=your_groq_key_here in your .env file.
uvicorn backend.app.main:app --host 0.0.0.0 --port 8000 --reloadVisit http://localhost:8000/
root_dir/
├── backend/
│ ├── app/
│ │ ├── main.py # Initializes FastAPI app and routes
│ │ ├── api/
│ │ │ └── routes/
│ │ │ ├── document_routes.py # Document-related API endpoints
│ │ │ └── pipeline_routes.py # Pipeline trigger API
│ │ ├── core/
│ │ │ ├── config.py # Configuration and constants
│ │ │ ├── logger.py # Application-wide logging
│ │ │ └── limiter.py # Rate Limiting
│ │ ├── models/
│ │ │ └── models.py # Pydantic request/response models
│ │ ├── services/
│ │ │ ├── document_preprocessor.py # OCR and document parsing
│ │ │ ├── query_engine.py # Groq-based per document answer generation
│ │ │ ├── theme_identifier.py # Extracts themes from answers
│ │ │ └── meta_store.py # Manages metadata storage and temporary FAISS search
│ ├── data/
│ └── uploads/{session_id}/ # Uploaded files stored per user session
│
│
├── demo/
│ ├── index.html # Frontend UI (HTML)
│ ├── style.css # Custom styles
│ └── script.js # JS for API interaction
│
├── docs/ # Module-level documentation
├── tests/ # Manual test files
├── .dockerignore # Ignore files for Docker builds
├── .env # Environment variables (not tracked)
├── .gitignore # Git ignore file
├── Dockerfile # Dockerfile for building docker image
├── README.MD # Project documentation
└── requirements.txt # Python dependencies
- PyMuPDF – Text extraction from PDFs
- Tesseract OCR – Open-source OCR engine for scanned documents
- pytesseract – OCR for scanned documents and images
- python-docx – Reading text from DOCX files
- sentence-transformers – Embeds document chunks into dense vectors
- FAISS – Temporary in-memory FAISS indexes are created at query time to support fast semantic retrieval using per-document metadata.
- RAG Architecture – Retrieval-Augmented Generation: retrieve relevant content, generate answers
- Groq API (OpenAI-compatible) – Answer generation and theme extraction based on retrieved context
- Docker – Containerization for consistent deployment
- MongoDB – Stores per-session, per-document chunk metadata and embeddings
- pymongo – MongoDB client for metadata operations
- FastAPI – Web framework for building APIs
- Pydantic – Data validation and schema modeling
- Uvicorn – ASGI server for running FastAPI
- Slowapi – For Rate Limiting
- HTML + CSS + Vanilla JS – Lightweight frontend for uploading documents, triggering analysis, and displaying results with built-in rate-limit handling.
- Logging (Python logging module) – Structured log output across modules
- dotenv – Environment variable loading
- Pathlib / os – File path management and directory handling
-
Per-User Authentication & Encrypted Storage
-
Implement OAuth2/JWT-based login for secure user access
-
Enable encrypted, user-isolated file storage and query history
-
-
Redis-Based Rate Limiting (Per-User/IP)
-
Integrate Redis for scalable, distributed rate limiting across API endpoints
-
Enforce request quotas per session or user to prevent abuse
-
-
Redis-Based Session Locks & Query Control
-
Prevent overlapping executions from the same session using Redis locks
-
Enable queued or rejected requests if a session is already processing
-
Manage parallel execution across multiple sessions while preserving per-session consistency
-
-
Concurrent Session Management
- Improve execution tracking to safely support simultaneous queries from multiple sessions
-
Parallel Query Execution
- Scale backend to support concurrent document analysis for multiple users using multi-worker setups
-
Responsive Frontend for All Devices
- Optimize the UI for mobile, tablet, and smaller screens