Overview

DOCINT is a document intelligence platform built around a vectorless Retrieval-Augmented Generation architecture. Instead of embeddings and a vector database, it combines BM25 lexical retrieval with cross-encoder reranking, then generates grounded answers with a local LLM via Ollama — no external API calls, fully offline-capable.

Built during an AI engineering internship at Ernst & Young for government document workflows, with native OCR and query support for English and Gujarati.

This was a deliberate architectural experiment: start at the keyword-only end of the retrieval spectrum and find out whether the simplest possible system could carry a real government policy repository. It could — and where it could not is documented below, because that finding is what motivated the Hybrid RAG Chatbot that followed.

Problem Statement

Public-sector departments maintain large repositories of policy documents, circulars, technical reports and regulatory filings — often hundreds of pages, in multiple languages, with inconsistent formatting, including scanned documents that carry no machine-readable text layer.

An officer answering a single policy question may need to search that entire repository by hand. This is slow, inconsistent between reviewers, and difficult to audit afterwards: a manual answer carries no direct trace back to the source passage that justified it.

The conventional response is a vector database. That brings embedding infrastructure, index maintenance and — for air-gapped government deployments — a meaningful operational burden. The question this project answers is whether that burden is necessary before it is proven necessary.

Why vectorless

Aspect	This project (BM25 + cross-encoder)	Traditional vector RAG
Infrastructure	Local filesystem only	Vector database required
Cost	Zero (fully local)	Recurring cloud costs
Auditability	High — interpretable BM25 scores	Lower — cosine similarity is opaque
Offline capable	Yes	Limited
GPU requirement	Optional	Often required

Architecture

Four layers, with no vector index anywhere in the path.

Layer	Technology
Frontend	Streamlit
Backend API	FastAPI (async)
Retrieval	rank-bm25 (BM25 Okapi) + sentence-transformers cross-encoder (ms-marco-MiniLM-L-6-v2)
Generation	Ollama, default model llama3.1:8b-instruct-q4_K_M
OCR	Tesseract (English + Gujarati)
Caching	cachetools TTL in-memory cache
Storage	Local filesystem, pickle-serialized BM25 index

Workflow

User Query
   → Query Expansion (OCR-error correction, synonyms, acronyms)
   → BM25 Lexical Retrieval (top 20 candidates)
   → Cross-Encoder Reranking (top 5 results)
   → Context assembly + citation tracing
   → Local LLM generation (Ollama, temperature 0.1 for grounded answers)

Screenshots

_{Conversational interface — indexed query templates and active retrieval guardrails}

_{Generated response — performance metrics and the full citation trace of source chunks used}

Features

• Vectorless retrieval — BM25 Okapi over a pickle-serialized local index; no embedding model or vector store at query time.
• Cross-encoder reranking — the top 20 lexical candidates are rescored down to the 5 passages actually placed in context.
• Query expansion — OCR-error correction, synonym and acronym expansion, applied before retrieval to recover matches that raw keyword search would miss.
• Citation tracing — every answer is returned with the source chunks and BM25/rerank scores that produced it.
• Retrieval transparency — per-query timing for retrieval and generation, plus a cache hit/miss indicator, shown in the interface.
• Bilingual OCR — Tesseract with English and Gujarati language packs, so scanned documents index alongside machine-readable ones.
• Grounded generation — prompts constrain the model to answer strictly from retrieved context, at temperature 0.1.
• Fully offline — no external API calls anywhere in the request path.

Tech Stack

Python · FastAPI · Streamlit · rank-bm25 · sentence-transformers · Ollama · Tesseract OCR · cachetools

Installation

Prerequisites

• Python 3.9+
• Tesseract OCR with English + Gujarati packs

  brew install tesseract tesseract-lang          # macOS
  apt-get install tesseract-ocr tesseract-ocr-guj # Linux

• Ollama with the target model pulled

  ollama pull llama3.1:8b-instruct-q4_K_M

Install

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Add your PDF source documents to data/.

Usage

Build the BM25 index over the documents in data/:

python scripts/ingest.py

Run the FastAPI backend:

uvicorn api.main:app --reload

Run the Streamlit interface:

streamlit run streamlit_app/app.py

Or start both together, and check the system is healthy:

bash scripts/start_docint.sh
python scripts/health_check.py

Project structure

api/               FastAPI backend (REST endpoints)
streamlit_app/     Streamlit chat UI
ingestion/         PDF parsing, OCR, chunking, tokenization
retrieval/         BM25 retrieval, query expansion, reranking
generation/        Ollama client + prompt templates
utils/             Citation engine, cache manager
config/            Settings and logging configuration
scripts/           Setup, ingestion, health-check, and start scripts
data/              PDF documents to index (gitignored — add your own)
index_store/       Generated BM25 indices (gitignored)

See TECH_STACK.txt for the full technical specification.

Observed Limitations

These are recorded rather than hidden — they are the finding that motivated the next system.

• Retrieval quality degrades for queries that paraphrase the source material or reference concepts indirectly, because keyword matching has no notion of meaning.
• The same weakness appears for non-English queries against English content and vice versa.
• Responsiveness scales with document volume and query length, as expected for a pipeline whose cost is proportional to the text searched.

Future Work

• Evaluate retrieval quality quantitatively (NDCG against a labelled query set) rather than by inspection.
• Incremental index updates instead of full rebuilds on ingestion.
• Configurable reranking depth per query class.

The semantic and multilingual gaps above are addressed directly in the successor system, the Hybrid RAG Chatbot, which fuses this lexical path with semantic vector search.

---

_{Developed during an AI engineering internship at Ernst & Young LLP (Technology Consulting, CNS — Technology Strategy & Transformation). Per enterprise confidentiality, this repository contains no client data, prompt text or internal configuration values. Architecture and workflow diagrams are redrawn from my Project Semester Report.}