Overview

A retrieval-augmented generation chatbot for querying a corpus of PDF documents, built to correct a specific, observed failure mode in its predecessor.

The earlier Vectorless RAG Chatbot proved that a government document repository could be made conversationally searchable with keyword retrieval alone. Evaluating it also showed exactly where that approach breaks: queries phrased differently from the source text, indirect references, and non-English content.

This system addresses those gaps directly — semantic vector search and keyword search run in parallel and are fused into a single ranking, with OCR for scanned pages and multilingual embeddings so one index serves cross-lingual search. Every answer carries citations back to its source passages.

Problem Statement

Keyword retrieval and semantic retrieval fail in opposite directions.

Keyword (lexical) retrieval is precise on exact terms — policy numbers, regulation codes, defined terminology — and is cheap to deploy and easy to audit. It has no understanding of meaning, so a question phrased differently from the source text may retrieve nothing useful.

Semantic (vector) retrieval represents query and passage as embeddings and matches on meaning, handling paraphrase and cross-lingual similarity well. It requires additional infrastructure, and can occasionally under-rank a passage that is an exact but non-obvious match.

Choosing one means accepting the other's weakness. Fusing both rankings, rather than choosing between them, was the central design decision here — and it was driven by an observed failure pattern in a working baseline, not by a generic preference for more advanced retrieval.

Architecture

Five layers — the vectorless system's four, plus a dedicated embedding layer, with retrieval becoming a dual path.

Component	Implementation
Ingestion (ingestion/)	Loads PDFs (loader.py), OCRs scanned pages with Tesseract (ocr.py), extracts tables with pdfplumber, chunks text (chunker.py)
Embedding (ingestion/embedder.py)	intfloat/multilingual-e5-base embeddings written to ChromaDB and a FAISS index (vector_store.py)
Retrieval (retrieval/search.py)	Hybrid search combining FAISS/Chroma vector similarity with BM25 keyword search, plus lightweight query spell-correction
Generation (generation/llm.py)	Local LLM via Ollama (gemma2:2b-instruct-q4_K_M by default), prompted with the retrieved context
API (api/main.py)	FastAPI server — query, file upload/ingestion, health, and index-stats endpoints
Frontends	react_frontend/ (Vite + React + Tailwind) and streamlit/app.py
CLI (chatbot/chat.py)	Minimal terminal chat loop

Configuration lives in config/settings.py (embedding model, LLM model, chunk size/overlap).

Workflow

Documents are OCR'd and cleaned, then chunked. Embeddings and a keyword index are built in parallel over the same chunks. At query time semantic and keyword search run against their respective indexes independently, and their results are merged through rank fusion before context construction. The fused context builds the prompt, and the response is paired with generated citations before it reaches the interface.

Screenshots

_{Conversational interface — indexed query templates, active guardrails, and engine state}

_{Citation trace — every retrieved chunk with its source document and relevance score}

Features

• Hybrid retrieval — FAISS/Chroma vector similarity fused with BM25 keyword ranking, so neither paraphrase nor exact-term precision is sacrificed.
• Multilingual embeddings — intfloat/multilingual-e5-base lets a single index serve cross-lingual queries instead of one store per language.
• OCR ingestion — Tesseract (English + Gujarati) brings scanned pages into the same index as machine-readable documents.
• Table extraction — pdfplumber preserves tabular content that naive text extraction flattens.
• Citation-backed answers — citations are a first-class pipeline output, not a post-hoc addition.
• Retrieval transparency — the interface shows which passages contributed, via which retrieval path, with scores.
• Query spell-correction — lightweight correction ahead of retrieval, recovering matches lost to typos and OCR noise.
• On-premise by default — the LLM runs locally through Ollama; document content never leaves the host.
• Two frontends — a React chat UI and a Streamlit app over the same FastAPI backend.

Tech Stack

Python · FastAPI · React · Vite · Tailwind CSS · Streamlit · ChromaDB · FAISS · BM25 · multilingual-e5-base · Ollama · Tesseract OCR · pdfplumber

Installation

Prerequisites

• Python 3.10+
• Node.js 18+ (for the React frontend)
• Ollama running locally with the target LLM pulled:

  ollama pull gemma2:2b-instruct-q4_K_M

• Tesseract OCR (English + Gujarati packs) for scanned PDFs:

  brew install tesseract tesseract-lang

• Poppler (for pdf2image):

  brew install poppler

Install

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Add your PDF source documents to data/.

Usage

Build the index — this populates chroma_db/ and faiss_index/:

python -c "from ingestion.vector_store import run_ingestion; run_ingestion()"

Run the API:

uvicorn api.main:app --reload

Run a frontend — React:

cd react_frontend
npm install
npm run dev

Streamlit:

streamlit run streamlit/app.py

Terminal chat:

python -m chatbot.chat

Notes

• ingestion/store.py is an older MongoDB-backed ingestion path, unused by the current API (which uses vector_store.py with Chroma + FAISS). Kept for reference.
• Downloaded embedding weights (models/), the vector stores (chroma_db/, faiss_index/) and source PDFs (data/) are git-ignored — regenerate or re-add them locally.

Future Work

• Quantitative retrieval evaluation (NDCG over a labelled query set) to measure the fusion gain rather than infer it.
• Tunable fusion weighting between the lexical and semantic rankings per query class.
• Incremental re-indexing on document upload instead of full rebuilds.
• Broader OCR language coverage beyond English and Gujarati.

---

_{Developed during an AI engineering internship at Ernst & Young LLP (Technology Consulting, CNS — Technology Strategy & Transformation). Per enterprise confidentiality, this repository contains no client data, prompt text or internal configuration values. Architecture and workflow diagrams are redrawn from my Project Semester Report.}