Haystack 2.0 Local RAG Assistant

A fully local Retrieval-Augmented Generation (RAG) assistant built to query and converse about the Haystack 2.0 documentation (Limited knowledge base currently). This project utilizes an entirely local AI stack, ensuring zero data leakage and avoiding API costs, while implementing a full evaluation loop to measure hallucination and answer relevancy.

Architecture

This project is built using:

• Orchestration: Haystack 2.0 for pipeline generation.
• Language Model: llama3 running entirely locally via Ollama.
• Embeddings: nomic-embed-text for highly accurate vector representations.
• Vector Database: ChromaDB for persistent document storage and retrieval.
• Frontend: Streamlit for an interactive, memory-aware conversational UI.
• Evaluation: Ragas framework acting as an LLM-as-a-judge to grade the pipeline on Faithfulness and Answer Relevancy.

Project Structure

• ingest.py: Handles web scraping of Haystack documentation, document cleaning, chunking (250 words w/ 30 overlap), embedding, and indexing into ChromaDB.
• query.py: Contains the core components to build the Haystack pipeline, acting as the single source for both the UI and evaluation scripts.
• app.py: A Streamlit web application that provides a chat interface, maintaining short-term conversation history for context-aware follow-up questions.
• evaluate_rag_v2.py: Uses the Ragas framework to test the pipeline against a (hard-coded) dataset of ground-truth Q&A pairs, utilizing LLM(Llama 3) as a judge approach.