A production-grade Retrieval-Augmented Generation pipeline with a built-in multi-agent adjudication layer that independently audits every answer it produces before it reaches the user.

📖 Description of Project

Every RAG portfolio project answers questions. None of them check their own work. In production, the failure mode that actually costs companies money isn't "RAG can't find the answer" — it's "RAG confidently returns a wrong answer with a citation that doesn't actually say that."

Veritas closes that gap: it's a RAG system where every generated answer is routed through a panel of independent critic agents (running on different model personas, so they don't share blind spots) before it's marked "safe to ship" to the end user. This reframes the architecture from simple retrieval to true evaluation-engineering.

🌍 Real World Use

• Legal & Compliance: Verifying that a generated brief correctly cites the actual uploaded contracts.
• Medical Triage: Ensuring that AI summaries of patient records do not hallucinate diagnoses that aren't present in the source files.
• Customer Support Automation: Auditing automated responses against strict company policy documents to prevent rogue claims.

🏗️ Architecture

flowchart TD

    subgraph ING["Phase 1 — Ingestion & Chunking"]
        A1[Raw Docs: MD / TXT / HTML / PDF] --> A2[Multi-format Loader<br/>+ Metadata: source, heading, page]
        A2 --> A3{Chunking Strategy}
        A3 -->|Fixed-size + overlap| A4[Chunks]
        A3 -->|Structure-aware| A4
        A3 -->|Semantic boundary| A4
        A4 --> A5[Dedup Check<br/>cosine sim > 0.95 → skip]
        A5 --> A6[(Embeddings<br/>text-embedding-3-small)]
        A5 --> A7[(BM25 Index)]
        A6 --> A8[(Vector Store<br/>ChromaDB / Qdrant)]
    end

    subgraph RET["Phase 2 — Hybrid Retrieval"]
        B1[User Question] --> B2[Dense Retrieval<br/>top-k=10 cosine sim]
        B1 --> B3[Sparse Retrieval<br/>top-k BM25]
        A8 --> B2
        A7 --> B3
        B2 --> B4[Reciprocal Rank Fusion<br/>0.7 dense / 0.3 sparse]
        B3 --> B4
        B4 --> B5[Cross-Encoder Rerank<br/>top 20 → top 5]
    end

    subgraph GEN["Phase 3 — Grounded Generation"]
        C1["Groq API<br/>llama-3.3-70b-versatile<br/>(temp ~0.1)"]
        B5 --> C1
        C1 --> C2["Answer + Bracketed Citations<br/>e.g. claim... ref 1, ref 2"]
        C2 --> C3[Citation Parser<br/>extract claim ↔ chunk pairs]
    end

    subgraph ARB["Phase 4 — Arbitration Layer (parallel fan-out)"]
        C3 --> D1[Factual Accuracy Critic<br/>Groq: llama-3.3-70b-versatile<br/>checks claims vs retrieved chunks]
        C3 --> D2[Logical Consistency Critic<br/>Groq: alt model / persona<br/>checks reasoning coherence]
        C3 --> D3[Completeness Critic<br/>Groq: llama-3.1-8b-instant<br/>checks question fully addressed]
        B5 -. source chunks .-> D1
        B5 -. source chunks .-> D2
        B5 -. source chunks .-> D3
    end

    subgraph ADJ["Phase 5 — Disagreement Detection & Adjudication"]
        D1 --> E1{Disagreement Detector<br/>severity gap > 2? missed issue?}
        D2 --> E1
        D3 --> E1
        E1 -->|No disagreement| E2[Short-circuit:<br/>High-confidence PASS]
        E1 -->|Disagreement found| E3["Adjudicator Agent<br/>Groq: llama-3.3-70b-versatile<br/>re-checks disputed claims vs source chunks"]
        E3 --> E4[Final Verdict:<br/>score, confirmed issues,<br/>dismissed flags, confidence]
        E2 --> E4
    end

    subgraph OUT["Output & Persistence"]
        E4 --> F1[Answer + Confidence Score<br/>+ Annotated Citations]
        E4 --> F2[(SQLite + JSON<br/>Full Audit Trail)]
        F1 --> F3[FastAPI<br/>/v1/ask · /v1/arbitrate · /v1/ingest]
        F3 --> F4[Verdict Explorer UI<br/>Streamlit / React]
        F2 --> F5[Analytics Dashboard<br/>critic agreement rate,<br/>override rate, chunking comparison]
    end

    style ARB fill:#2b2b3d,stroke:#8888ff
    style ADJ fill:#2b2b3d,stroke:#ff8888
    style GEN fill:#2b2b3d,stroke:#88ff88

Loading

🛠️ Tech Stack

• Language Models: Inference is handled via Groq utilizing Llama 3 models (llama-3.3-70b-versatile & llama-3.1-8b-instant). Groq is used due to its deterministic, high-throughput LPU engines giving single-digit millisecond token latencies to run concurrent multi-agent critique graphs smoothly.
• Backend Infrastructure: Python 3.12 with FastAPI handles asynchronous I/O and routing. uvicorn acts as the ASGI server for high concurrency.
• Embedding & Vector DB: HuggingFace's all-MiniLM-L6-v2 is utilized for robust semantic representations. Vectors and their associative chunk metadata are stored in ChromaDB (SQLite-backed) mapped to local disk for persistence.
• Sparse Indexing: Lexical/keyword search relies on the rank_bm25 module using the Okapi BM25 algorithm to score document relevance for discrete technical queries that dense vectors often fail to encapsulate.
• Frontend Stack: Built on Next.js 16 App Router. React Server Components and client-side hooks interface directly with the FastAPI endpoints. The UI is designed with Tailwind CSS focusing heavily on dynamic, glassmorphic interactions and lucide-react for iconography.
• Agent Orchestration: Structured outputs and tool binding are strictly typed using Pydantic. Agent personas are systematically managed directly on top of raw API bindings with highly specialized system prompts defining their role (Generation, Criticism, Adjudication).

🧠 RAG System

The generation system operates on a state-of-the-art Hybrid Retrieval Engine configured to eliminate knowledge gaps:

1. Multi-Strategy Ingestion: Documents (PDF, MD, TXT, HTML) are digested via multi-strategy parsers, divided into normalized chunks while retaining document metadata, and then parallelized into dense and sparse representations.
2. Dense Retrieval Pass: Cosine similarity via HuggingFace's all-MiniLM-L6-v2 retrieves semantically correlated context from ChromaDB.
3. Sparse Retrieval Pass: BM25 handles pure lexical mapping to cover esoteric edge cases, proprietary IDs, and structural keywords.
4. Fusion via RRF: Reciprocal Rank Fusion aggregates the dense and sparse topologies to formulate a single candidate set.
5. Generative Grounding: Groq leverages llama-3.3-70b-versatile utilizing highly constrained system prompts restricting it to ONLY draw from the finalized retrieval context. It injects literal bracketed citations [n] referencing the explicit document chunk used.

⚖️ Arbitration System

Veritas doesn't just answer; it self-audits.

• Parallel Critics: Three independent Pydantic-typed agents evaluate Accuracy, Logic, and Completeness against the actual retrieved chunks.
• Disagreement Detection: If the critics conflict, the system flags the severity.
• Adjudicator Agent: An overarching Llama 3 instance re-checks disputed claims directly against the source text to resolve conflicts and output a final, confidence-scored verdict.

💻 Front End

Built entirely in Next.js (App Router) and Tailwind CSS. The interface features a dark, glassmorphic UI, smooth micro-animations, and a highly interactive layout. It allows users to:

• Upload and manage multi-format documents (PDF, MD, HTML, TXT) seamlessly.
• Chat with the RAG system and receive dictated answers via TTS.
• Expand a detailed Arbitration Log to see exactly how the LLM critics graded the response behind the scenes.

🔌 API Design

The backend is driven by FastAPI exposing the following core endpoints:

• POST /v1/ingest: Accepts files, extracts text, chunks, embeds, and pushes to ChromaDB & BM25 indexes in safe background batches.
• GET /v1/documents: Lists currently indexed files.
• DELETE /v1/documents/{filename}: Un-indexes specific files.
• POST /v1/ask: Triggers the end-to-end pipeline (Retrieve → Generate → Arbitrate → Adjudicate) returning a deeply structured payload with citations and verdicts.
• GET /v1/tts: A fast synchronous endpoint that converts the final response text into a playable audio stream.

🗂️ File Structure

Veritas/
├── backend/
│   ├── api.py                 # FastAPI server & route definitions
│   ├── ingestion.py           # Parsing, chunking, and ChromaDB DB ops
│   ├── retrieval.py           # Hybrid search & Reciprocal Rank Fusion
│   ├── generation.py          # Grounded LLM answer generation
│   ├── arbitration.py         # Multi-agent critic panel & Adjudicator
│   ├── tts.py                 # Text-To-Speech integration
│   └── requirements.txt
├── frontend/
│   ├── src/
│   │   ├── app/
│   │   │   ├── layout.tsx     # Root Next.js layout
│   │   │   ├── page.tsx       # Landing page
│   │   │   └── rag/page.tsx   # Main RAG Chat & Arbitration UI
│   │   └── components/        # Reusable UI components
│   ├── package.json
│   └── tailwind.config.ts
└── README.md

🎥 Project Demo

To see Veritas in action:

1. Upload a source document via the Docs manager in the top right.
2. Ask a question about the document's content.
3. Once the answer generates, click "View Arbitration Log" below the message to watch the Accuracy, Logic, and Completeness critics grade the output in real time.
4. Click the Dictate (speaker) button to hear the answer spoken back to you!

🚀 Cloning & Setup

Prerequisites

• Python 3.12+
• Node.js 18+
• API Keys: GROQ_API_KEY and HF_TOKEN

Backend Setup

cd backend
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# Create your .env file
echo "GROQ_API_KEY=your_key_here" > .env
echo "HF_TOKEN=your_token_here" >> .env

# Run the API
uvicorn api:app --reload

Frontend Setup

cd frontend
npm install

# Run the development server
npm run dev

Visit http://localhost:3000 to interact with Veritas!