RAG Exploration

Repository to explore different approaches to document retrieval and generation. The UI is a simple web app that lets you:

• Upload a PDF.
• Choose a retrieval strategy.
• Chat about the document.

Overview

1. Parse & embed — upload a document → parse with Apache Tika → chunk → embed with OpenAI text-embedding-3 → store vectors in Pinecone. The UI streams each step live.
2. Query & retrieval — chat about the document with two selectable hybrid strategies:
   • Approach 1 — Hybrid + Reciprocal Rank Fusion: dense (cosine) + lexical (BM25) retrieval, fused with RRF, top-k → LLM answer.
     
     
     
     
   • Approach 2 — Hybrid + Cross-encoder rerank: union of dense + lexical candidates, reranked by a cross-encoder, top-k → LLM answer.
     
     

document-processing-pipeline/
├── backend/   # Python · FastAPI · Tika · OpenAI · Pinecone (the AI parts)
└── frontend/  # Vite · vanilla JS (upload + step progress + chat)

Architecture notes

• Dense retrieval runs in Pinecone (cosine similarity).
• Lexical retrieval uses BM25 (rank_bm25) over chunk text persisted locally in backend/data/chunks.json, indexed on the exact same chunks stored in Pinecone — so the two retrievers stay aligned.
• Cross-encoder uses sentence-transformers (ms-marco-MiniLM-L-6-v2), downloaded on first use (Approach 2 only).
• The final answer is generated by an OpenAI chat model, grounded strictly in the retrieved passages, with [n] citations.

Prerequisites

• Python 3.10+ and Java (Tika downloads and runs a local server jar).
• Node 18+.
• OpenAI API key and Pinecone API key.

Setup

Backend

cd backend
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env     # then fill in OPENAI_API_KEY and PINECONE_API_KEY
uvicorn app.main:app --reload --port 8000

The Pinecone index is created automatically on first use (serverless, cosine, dimension from EMBEDDING_DIM).

Make sure EMBEDDING_DIM matches your model: text-embedding-3-small → 1536, text-embedding-3-large → 3072.

Frontend

cd frontend
npm install
npm run dev      # http://localhost:5173 (proxies /api to :8000)

Using it

1. Open http://localhost:5173.
2. Choose a PDF and click Start pipeline — watch parse → chunk → embed → upsert complete step by step.
   
   
   
   
3. Pick a retrieval Approach, set Top K, and chat about the document. Expand each answer's sources to inspect the retrieved chunks and scores.
   
   

API

Method	Endpoint	Description
POST	/api/upload	Multipart file. Streams SSE step progress.
POST	/api/query	{query, doc_id?, approach, top_k} → answer.
GET	/api/documents	List ingested documents.
POST	/api/clear	Delete all vectors from Pinecone + local store.
GET	/api/health	Health check.

approach is "rrf" (Approach 1) or "cross_encoder" (Approach 2).

Future work

This project is a personal sandbox for experimenting with document parsing and retrieval. A few directions I'd like to explore next:

1 · Parsing & chunking

The current flow is Tika text extraction + a RecursiveCharacterTextSplitter. Ways to evolve it:

• Layout-aware parsing — swap/augment Tika with tools that preserve document structure (headings, tables, columns, reading order), e.g. unstructured, Docling, LlamaParse, or AWS Textract / Azure Document Intelligence for scanned PDFs.
• Structure-aware chunking — chunk along semantic boundaries (sections, paragraphs, list items, table rows) instead of a fixed character window, and keep tables/code blocks intact.
• OCR for image-only PDFs — fall back to Tesseract / PaddleOCR when Tika returns little or no text, so scanned documents are supported.
• Semantic / recursive chunking — split on embedding-similarity breakpoints, or build hierarchical "parent–child" chunks (small chunks for retrieval, larger parents for context).
• Richer metadata — capture page numbers, section titles, and headings per chunk to power citations, filtering, and better grounding.

2 · Querying & retrieving

Today: hybrid dense + BM25 with RRF or a cross-encoder reranker. More sophisticated retrieval to try:

• Query transformation — query rewriting, HyDE (hypothetical document embeddings), and multi-query expansion to improve recall on vague questions.
• Contextual / late-interaction retrieval — ColBERT-style multi-vector retrieval, or Anthropic-style "contextual retrieval" that prepends context to each chunk before embedding.
• Multi-hop & agentic retrieval — let the LLM decompose a question, issue several retrieval calls, and synthesize — for questions that span sections.
• Parent-document / auto-merging retrieval — retrieve small chunks but feed their larger parent passages to the LLM for fuller context.
• Conversational memory — condense chat history into standalone queries so follow-up questions retrieve correctly.