RAG: 5K Running Coach

An expert-level RAG (Retrieval-Augmented Generation) system providing specialized guidance for 5K running. This project demonstrates a production-ready approach to information retrieval, combining dense and sparse search techniques with rigorous evaluation frameworks.

Engineering Highlights

• Hybrid Retrieval (RRF): Implements Reciprocal Rank Fusion to merge results from ChromaDB (Dense) and BM25 (Sparse). This solves the "incompatible score" problem between vector distance and lexical frequency without requiring manual weight tuning.

• Inference Optimization: Utilizes GPT-5-Nano to achieve high-quality reasoning with low latency, making it ideal for real-time assistant applications.

• Evaluation-Driven Development: Moving beyond "vibes-based" AI. The system is benchmarked using Hit Rate and MRR to ensure the retrieval engine provides the highest quality context to the LLM.

• Automated Ingestion: A dedicated pipeline (index.py) handles document chunking, embedding generation, and multi-index synchronization for both search engines.

• Containerized Microservices: Fully dockerized FastAPI backend, ensuring environment parity and seamless horizontal scaling.

Tech Stack

• LLM: GPT-5-Nano
• Core Logic: Python
• Vector Database: ChromaDB (Dense)
• Lexical Search: BM25 (Sparse)
• API Framework: FastAPI
• DevOps: Docker, Docker Compose
• Evaluation: LLM-as-a-Judge (Answer Relevance)

Performance Benchmarking

This project uses a two-stage evaluation process:

1. Retrieval Stage

The "Ground Truth" retrieval quality is measured to ensure the LLM has the best possible context.

Metric	Purpose
Hit Rate (Recall)	Percentage of queries where the correct context is within the top-$k$ results.
MRR	Evaluates the rank of the relevant document; higher values indicate better precision.

2. Generation Stage

Using an LLM-as-a-Judge framework, we programmatically score the generated response against the retrieved context to measure Relevance and Faithfulness, minimizing hallucinations.

Getting Started

1. Ingestion Pipeline

Build the hybrid index from the data/ directory:

python app/index.py

2. Ingestion Pipeline

Spin up the FastAPI backend and infrastructure:

docker compose up --build

3. Client Interaction

Launch the CLI assistant to start chatting:

python chat_cli.py

Development & Prototyping

The core RAG logic was developed and validated in a research environment to ensure optimal retrieval and generation performance before being ported to the FastAPI backend.

• Development Notebook: rag-notebook.ipynb
  • Indexing & Orchestration: Full implementation of document chunking and hybrid index creation (ChromaDB + BM25).
  • Retrieval Evaluation: Quantitative benchmarking using Hit Rate (Recall) and MRR to tune the retrieval parameters.
  • RAG Evaluation: Implementation of LLM-as-a-Judge to evaluate answer faithfulness and relevance.
  • End-to-End Pipeline: Initial prototyping of the RAG pipeline.

Roadmap & Best Practices

• Hybrid Search: RRF Implementation
• Containerization: Docker & Compose
• Document Reranking: (Planned: Cross-Encoder integration)
• Query Rewriting: (Planned: HyDE or Multi-query expansion)

Acknowledgments

This project was developed as part of the LLM Zoomcamp provided by DataTalks.Club. Special thanks to the instructors for the comprehensive curriculum on production-grade RAG systems.

---

Danny Khant Engineer | AI & Data Engineering