📘 Retrieval-Augmented Generation (RAG) From Scratch

This project implements a Retrieval-Augmented Generation (RAG) pipeline from scratch, without using LangChain or LlamaIndex abstractions. It demonstrates a clear, modular, and educational implementation of RAG using Dense Passage Retrieval (DPR), FAISS, and GPT-2.

The project is designed to help learners and practitioners understand the core mechanics of RAG systems used in modern LLM-based applications. 🚀 Key Features

🔍 Dense Retrieval using Facebook DPR (context & question encoders)

📦 Vector Indexing with FAISS

🧠 Text Generation using GPT-2

🧩 Fully modular Python architecture

💻 Runs locally on CPU

📚 Clean separation of:

data loading

embedding

retrieval

generation

🧠 RAG Architecture (Conceptual) User Question │ ▼ Question Encoder (DPR) │ ▼ FAISS Vector Search │ ▼ Top-K Relevant Contexts │ ▼ Prompt Construction │ ▼ GPT-2 Generator │ ▼ Final Answer 📂 Project Structure

rag-from-scratch/ │ ├── data/ │ └── companyPolicies.txt # Input knowledge base │ ├── src/ │ ├── data_loader.py # Download & chunk documents │ ├── embeddings.py # Context encoder logic │ ├── retrieval.py # Question encoder + FAISS search │ ├── index.py # FAISS index creation │ ├── generation.py # GPT-2 loading & answer generation │ ├── config.py # Device configuration (CPU/GPU) │ └── init.py │ ├── run_rag.py # Main execution script ├── requirements.txt # Python dependencies ├── README.md # Project documentation └── .gitignore 🛠️ Technologies Used

Python 3.9+

PyTorch

Hugging Face Transformers

FAISS

Dense Passage Retrieval (DPR)

GPT-2

⚙️ Installation & Setup 1️⃣ Clone the Repository git clone https://github.com//rag-from-scratch.git cd rag-from-scratch

2️⃣ Create Virtual Environment python -m venv rag-env rag-env\Scripts\activate # Windows

3️⃣ Install Dependencies pip install -r requirements.txt

▶️ Running the Project

Execute the main pipeline:

python run_rag.py

You should see:

Document loading

Embedding creation

FAISS index building

Context retrieval

Generated answer

Example output:

Question: What is the mobile phone policy? Answer: The mobile phone policy outlines acceptable use, security, compliance, and responsibilities related to company-issued mobile devices...