An enterprise-grade, localized Retrieval-Augmented Generation (RAG) platform optimized for private context retrieval, semantic indexing, and multi-stage query optimization. Built entirely on an open-source, local-first architecture to maintain complete data privacy.
This project implements an advanced pipeline designed to maximize context precision over standard RAG implementations:
- Multi-Query Expansion: Utilizes
llama3.2to rewrite raw chat intents into highly specific vector search terms. - Context Deduplication & Merging: Aggregates original inputs and expanded query variations, resolving semantic duplicate boundaries.
- LLM-Based Reranking: Dynamically orders extracted segments via a cross-encoder model paradigm to fit strict token budgets.
- Automated Evaluation: Continuous tracing and dataset benchmarking hooked up with LangSmith.
- LLM Core Framework: Ollama (
llama3.2) - Vector Vectorstore: ChromaDB
- Embeddings: HuggingFace (
all-MiniLM-L6-v2) - Orchestration: LangChain & LiteLLM
- UI Interface: Gradio 6.0
- Observability: LangSmith Trace Integration
Ensure you have the uv package manager installed, then configure your environment:
uv venv
.venv\Scripts\activate
uv pip install -r requirements.txt