Upload PDFs, Word documents, PowerPoint decks, or CSVs, then ask questions across all of them at once. Every answer streams in real time, cites the exact document and page it came from, and is checked for whether those citations are actually grounded in what was retrieved.
This isn't the "vector search → LLM" version of RAG you'll find in most tutorials. It's built around the same pipeline shape used in real production retrieval systems: query rewriting, hybrid search, cross-encoder reranking, context compression, streaming generation, citation verification, and a retrieval evaluation dashboard.
User question
│
▼
Query Rewriter — resolves follow-ups using chat history
│ ("what about the second one?" → self-contained question)
▼
Query Expansion — generates alternate phrasings to widen recall
│
▼
Hybrid Retrieval — BM25 (keyword) + vector (semantic) search,
│ fused with Reciprocal Rank Fusion
▼
Cross-Encoder Rerank — re-scores top candidates with a (query, chunk)
│ pair model, far more precise than embedding similarity alone
▼
Context Compression — dedupes overlapping sentences, trims to a token budget
│
▼
LLM Generation (streamed) — answer streams token-by-token in the UI
│
▼
Citation Verification — flags any cited source that wasn't actually retrieved
Retrieval quality
- Hybrid search: BM25 keyword search + vector semantic search, fused via Reciprocal Rank Fusion (not naive score averaging, which breaks because BM25 and cosine-similarity scores live on incompatible scales)
- Cross-encoder reranking as a second, more accurate pass over the top candidates from hybrid search
- Query rewriting for conversational follow-ups
- Query expansion for terminology mismatches ("cost" vs. "pricing" vs. "fees")
- Metadata filtering — scope a question to a single uploaded document
- Context compression — sentence-level dedup + token-budget trimming
Generation
- Streamed responses (token-by-token, not wait-for-the-whole-answer)
- Multi-provider LLM support — Groq, OpenAI, Anthropic, Gemini, Ollama — switchable from the sidebar or via one config value
- Explicit hallucination guardrail: told to say "I don't know" when the retrieved context doesn't cover the question
- Citation verification: cross-checks that cited sources actually appear in the retrieved context, with a visible groundedness badge in the UI
Evaluation (separate dashboard page)
- Precision@K, Recall@K, and MRR against a hand-authored eval set
- Live per-request latency breakdown (query rewrite, expansion, hybrid retrieval, reranking, compression, LLM generation)
- Token usage and estimated cost per request
Production scaffolding
- Central
config.py— every tunable value overridable via.env, no magic numbers scattered through the codebase - Structured logging with per-stage timing
- Unit tests (
pytest) covering chunking, hybrid search, and evaluation metrics - Dockerfile + docker-compose for containerized deployment
- GitHub Actions CI (lint + test + Docker build on every push)
| Layer | Choice |
|---|---|
| Orchestration | LangChain (chunking) + custom pipeline (retrieval/generation) |
| Keyword search | BM25 (rank-bm25) |
| Vector DB | ChromaDB (persistent, per-session isolated) |
| Embeddings | Sentence-Transformers (all-MiniLM-L6-v2) |
| Reranker | Cross-Encoder (ms-marco-MiniLM-L-6-v2) |
| LLM providers | Groq / OpenAI / Anthropic / Gemini / Ollama (pluggable) |
| Frontend | Streamlit (multipage: chat + evaluation dashboard) |
| Testing | pytest |
| CI/CD | GitHub Actions |
| Deployment | Docker / docker-compose |
git clone https://github.com/<your-username>/multi-doc-rag-chatbot.git
cd multi-doc-rag-chatbot
python -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env # fill in the API key for your chosen provider
streamlit run app.pycp .env.example .env # fill in your API key
docker compose up --buildThen open http://localhost:8501.
pytest tests/ -vrag-chatbot/
├── app.py # Main Streamlit chat UI
├── pages/
│ └── 1_Evaluation_Dashboard.py # Retrieval metrics + latency/cost dashboard
├── backend/
│ ├── config.py # Central, env-driven configuration
│ ├── document_loader.py # PDF / DOCX / PPTX / CSV parsing
│ ├── chunking.py # Text splitting
│ ├── vectorstore.py # ChromaDB wrapper + metadata filtering
│ ├── hybrid_search.py # BM25 + vector fusion (RRF)
│ ├── reranker.py # Cross-encoder reranking
│ ├── query_processing.py # Query rewriting + expansion
│ ├── context_compression.py # Dedup + token-budget trimming
│ ├── citation_verification.py # Groundedness checking
│ ├── llm_providers.py # Multi-provider LLM abstraction + streaming
│ ├── evaluation.py # Precision@K / Recall@K / MRR / cost estimation
│ └── logging_config.py # Structured logging + stage timing
├── tests/ # pytest unit tests
├── .github/workflows/ci.yml # Lint + test + Docker build on push
├── Dockerfile / docker-compose.yml
├── .env.example
└── requirements.txt
- Why Reciprocal Rank Fusion instead of averaging scores? BM25 scores and cosine similarity live on incompatible scales — a 0.8 from one method doesn't mean the same thing as a 0.8 from the other. RRF only needs each method's rank ordering, sidestepping that problem entirely.
- Why rerank at all if vector search already ranks results? Bi-encoder retrieval embeds the query and each chunk independently, which is fast but approximate. A cross-encoder reads the (query, chunk) pair together in one pass — much more accurate, but too slow to run over an entire corpus, hence the two-stage "retrieve cheap, then rerank precisely" pattern.
- Why verify citations instead of trusting the LLM? LLMs can cite a source that sounds plausible but wasn't actually retrieved. A deterministic (non-LLM) check catches this cheaply and flags it in the UI rather than silently trusting every citation.
- Why measure retrieval quality separately from the LLM's answer quality? A great LLM can't fix bad retrieval — if the right chunk was never retrieved, no amount of prompting recovers it. Precision@K/Recall@K/MRR isolate whether the retrieval stage is working before blaming generation.
- Swap the exact-match citation check for a lightweight NLI-based entailment check (does the chunk actually support the claim, not just get mentioned)
- Add RAGAS or a similar framework for LLM-graded answer quality, on top of the retrieval-only metrics already implemented
- Add authentication + multi-tenant persistent storage (S3-backed uploads) instead of per-session in-memory processing, for a true multi-user deployment
MIT