RAG-Powered Document Intelligence System

Overview

This project is a production‑oriented Retrieval‑Augmented Generation (RAG) system designed to transform unstructured documents into an interactive, queryable knowledge interface. Users can upload their own documents and ask natural‑language questions, receiving grounded, source‑aware answers generated by a large language model.

It combines local document search with optional web augmentation and returns citation-aware answers.

---

Features

• RAG core: uses LangChain chains and a local FAISS vector store for semantic retrieval.
• Embeddings: HuggingFace sentence-transformers for local embeddings.
• Web augmentation: optional web search integration to supplement local context.
• Citation-aware answers: generated responses include clear source markers (document and web).
• Streamlit UI: upload documents, chat, and view concise evidence and relevance scores.

This enables accurate answers that are contextual, explainable, and traceable to source documents.

---

How It Works (High Level)

1. Document Ingestion Uploaded files are parsed and split into overlapping text chunks to preserve semantic continuity.

2. Embedding & Indexing Each chunk is converted into a dense vector using a local embedding model and stored in a FAISS index for fast similarity search.

3. Query Processing User queries are embedded and matched against the vector index to retrieve the most relevant document segments.

4. Context‑Aware Generation Retrieved content is injected into a structured prompt and passed to a large language model for answer generation.

5. Web Search Routing If local context is insufficient, the system dynamically invokes web search to supplement responses with up‑to‑date information. This happens only when the web search is toggled on.

6. Streaming Output Responses are streamed token‑by‑token to the UI for a responsive chat experience.

---

Architecture Highlights

• Separation of concerns across configuration, core RAG logic, tools, and UI
• Local embeddings + vector store for privacy and cost control
• Pluggable LLM and search providers
• Session‑aware UI state for multi‑turn conversations
• Persistent vector index to avoid reprocessing documents

The system follows SOLID principles and is designed to be easily extended or productionized.

---

Tech Stack

• Language: Python 3.11+
• LLM Inference: Groq (Llama 3.1 70B)
• Embeddings: HuggingFace sentence‑transformers (local execution)
• Vector Store: FAISS
• RAG Orchestration: LangChain
• UI: Streamlit
• Web Search: Tavily API

All core retrieval and embedding operations run locally, minimizing external dependencies.

---

Key Design Decisions

• Local‑first embeddings to eliminate recurring embedding costs
• FAISS over hosted vector DBs for simplicity and speed
• Streaming generation for better UX
• Hybrid search strategy instead of unconditional web queries
• Explicit configuration validation to fail fast on misconfiguration

---

Typical Use Cases

• Chat with internal documents (research papers, legal texts, manuals)
• Build a private knowledge assistant
• Prototype RAG systems without paid vector databases
• Learn production‑grade RAG architecture through real code

---

Deployment

The application is designed for secure deployment using environment variables or secret managers. It runs reliably on:

• Streamlit Cloud
• Docker‑based environments
• Any cloud platform supporting Python web apps

No API keys or secrets are committed to source control.

---

Scalability & Extension Paths

While currently optimized for single‑user or small‑scale usage, the architecture can be extended to support:

• Multi‑user document isolation
• Hosted vector databases (Pinecone, Weaviate)
• Authentication and access control
• Async ingestion pipelines
• API‑based (non‑UI) access

---

Quickstart

• Create / activate venv:

python -m venv chatbot2
# Windows
chatbot2\Scripts\activate
# macOS / Linux
source chatbot2/bin/activate

• Install dependencies:

pip install -r requirements.txt

• Environment variables: set API keys required by the LLM and web search (see config/settings.py):

set GROQ_API_KEY=your_groq_key_here   # Windows (PowerShell)
export GROQ_API_KEY=your_groq_key_here  # macOS / Linux

set TAVILY_API_KEY=your_tavily_key_here

• Run the app:

streamlit run app.py

Project layout (high level)

• core/: ingestion, embeddings, vector store, RAG chain, query routing and relevance checker.
• tools/: web search adapters and hybrid search manager.
• ui/: Streamlit components and chat interface.
• input_data/: uploaded documents (persisted between runs).

Developer notes

• To add documents: place files in input_data/ or use the UI uploader; the vector DB persists to disk.
• The system performs an initial local search and evaluates local relevance; when local context is insufficient it can augment with web search.
• Relevant modules to inspect: core/router.py, core/relevance_checker.py, core/vector_store.py, tools/tavily_search.py, ui/chat_interface.py.

Testing & Maintenance

• Restart the Streamlit app after changing settings or adding models.
• For debugging web results, inspect raw returns from tools/tavily_search.py.

---

Generated to reflect current workspace: see app.py for the main entrypoint.