📖 DocuMind AI

Precision-Aware Document Intelligence — RAG that tells you how much to trust it

Most RAG demos wrap an LLM around a vector store and call it done. This one questions that — and scores its own confidence for every answer it gives.

Features · Architecture · Run locally

🎯 The Problem With Naive RAG

Ask a typical PDF chatbot a question and it will always give you a confident-sounding answer — whether the document actually supports it or not. There's no signal for "I'm not sure," no visibility into why it retrieved what it retrieved, and one fixed chunk size that's always a compromise between precision and context.

DocuMind AI was built to fix all three.

✨ What Makes This Different

🔍 Multi-Granularity Retrieval Instead of picking one chunk size and living with the trade-off, three parallel FAISS indexes are built per document — fine (300 chars), medium (700 chars), coarse (1800 chars). A cross-encoder reranker picks the best granularity per question, automatically.	🎯 Confidence-Scored Answers Every answer ships with a 0–100% precision score, derived from reranker relevance logits via sigmoid transform, plus a 🟢 HIGH / 🟡 MEDIUM / 🔴 LOW label. Weak evidence gets flagged — not hidden behind confident phrasing.
🧩 Query Decomposition Complex or multi-part questions are automatically broken into sub-questions, answered independently, then synthesized — with a "show your reasoning" panel exposing every intermediate step and its own confidence score.	🔒 Privacy-Preserving Local Mode A toggle routes every LLM call through local Ollama instead of the cloud. Combined with local embeddings + local reranking, a live Privacy Audit panel shows exactly which component runs where — zero-network-transmission, formally auditable.
⚡ Instant Query Expansion Questions are auto-reformulated into 2–3 alternative phrasings (synonym substitution + structural rewrites) before retrieval — boosting recall with zero extra LLM calls, pure local logic, near-zero latency cost.	📅 Proactive Structure Extraction Beyond reactive Q&A: auto-extracted document timelines (dated events, visualized chronologically) and structured summaries (topic, doc type, key facts) — generated the moment a document is processed.

🏗️ Architecture

flowchart TD
    A["📄 Documents<br/>PDF · DOCX · TXT · URL · Text"] --> B["Multi-doc parsing<br/>+ per-source tagging"]
    B --> C["Chunking<br/>3 granularities in parallel"]
    C --> D1["FAISS · Fine<br/>300 chars"]
    C --> D2["FAISS · Medium<br/>700 chars"]
    C --> D3["FAISS · Coarse<br/>1800 chars"]
    E["❓ User Question"] --> F["Query Expansion<br/>local, instant"]
    F --> G["Retrieval across<br/>all 3 granularities"]
    D1 --> G
    D2 --> G
    D3 --> G
    G --> H["Cross-Encoder<br/>Reranking"]
    H --> I{"Complex<br/>question?"}
    I -->|Yes| J["Decompose →<br/>sub-Q answering →<br/>synthesis"]
    I -->|No| K["Direct answer"]
    J --> L["🎯 Confidence-scored<br/>answer + reasoning trace"]
    K --> L
    L --> M["LLM: HuggingFace API<br/>⇄ Local Ollama"]

    style A fill:#1a1d2e,stroke:#3a5bf0,color:#fff
    style L fill:#0d2c1a,stroke:#3dcc7e,color:#fff
    style M fill:#1a1420,stroke:#a07ee0,color:#fff

Loading

Multi-document balancing: during retrieval, chunks are grouped by source and capped per document before reranking — so one large PDF can't drown out a smaller one in the candidate pool.

🛠️ Tech Stack

Layer	Technology
UI	Streamlit — fully custom CSS, zero default theme
Orchestration	LangChain
Vector Store	FAISS (IndexFlatIP, in-memory)
Embeddings	sentence-transformers/all-mpnet-base-v2
Reranking	cross-encoder/ms-marco-MiniLM-L-6-v2
Cloud LLM	HuggingFace Inference API — Meta-Llama-3-8B-Instruct
Local LLM	Ollama — Qwen2.5 / Llama3 (auto-detected at runtime)
Document Parsing	PyPDF2 · python-docx · LangChain WebBaseLoader

🚀 Running It Locally

git clone https://github.com/Sahoo999/RAG-BASED-Q-A-SYSTEM.git
cd RAG-BASED-Q-A-SYSTEM
pip install -r requirements.txt

Set your HuggingFace API key:

# macOS / Linux
export HUGGINGFACE_API_KEY="hf_your_key_here"

# Windows PowerShell
$env:HUGGINGFACE_API_KEY = "hf_your_key_here"

Launch:

streamlit run app.py

🔒 Want Privacy Mode? Install Ollama, then ollama pull qwen2.5:0.5b. The app auto-detects it and unlocks the local-inference toggle in the sidebar — no config needed.

📁 Project Structure

├── app.py               # Full application — UI, RAG pipeline, LLM routing
├── requirements.txt      # Dependencies
├── secret_api_keys.py    # Reads HUGGINGFACE_API_KEY from env/Streamlit secrets
└── README.md

💡 Design Decisions Worth Knowing

Why three granularities instead of one "optimal" chunk size?

Because there isn't one — the right chunk size depends on the question, not the document. A factual one-liner benefits from tight, precise chunks; a "summarize the section" question needs broad context. Building all three upfront and letting a reranker choose per-query removes the need to guess.

Why a cross-encoder reranker on top of embedding similarity?

Cosine similarity between independently-encoded vectors is a weak relevance signal — it can't model interaction between the query and the passage. A cross-encoder jointly attends to both, catching relevance that embedding search alone misses.

Why expose confidence scores instead of just answering?

An LLM will answer fluently and confidently even when the retrieved context doesn't actually support the claim. Surfacing a calibrated score turns "trust me" into "here's the evidence strength — verify if it's LOW," which is a fundamentally more honest contract with the user.

Why offer a local-inference mode at all?

Sensitive documents — medical, legal, financial, proprietary — shouldn't have to leave the user's machine just to be queried. Privacy Mode makes that a real, auditable guarantee rather than a marketing claim: the sidebar's Privacy Audit panel names every pipeline stage and whether it ran locally or in the cloud.

---

📬 Let's Connect

Built with a focus on the parts of RAG that most demos skip: honesty about confidence, and respect for user privacy.