LangChain Docs RAG

A production-grade Retrieval-Augmented Generation system that answers questions over LangChain's documentation — built entirely on free infrastructure. No paid APIs, no paid hosting.

Why this project

Most portfolio RAG projects are a notebook calling an LLM API on top of a FAISS index, with no evaluation, no API layer, and no idea how they'd behave under real traffic. This one is built the way an actual retrieval service is: hybrid search + reranking, an evaluated pipeline, a typed API, structured logging and metrics, containerized, with CI and a real deployment path — without spending anything.

Architecture

                    ┌─────────────┐
   docs.langchain   │  Ingestion   │  crawl + clean + chunk
   .com  ─────────▶ │  (Phase 2)   │
                    └──────┬──────┘
                           │ chunks.jsonl
                           ▼
                    ┌─────────────┐
                    │  Embeddings  │  sentence-transformers (local)
                    │  (Phase 3)   │
                    └──────┬──────┘
                           ▼
                    ┌─────────────┐
                    │   Qdrant     │  self-hosted vector store
                    └──────┬──────┘
                           │
   User ── question ──▶ ┌─────────────┐
                        │   Hybrid     │  dense + BM25 → RRF fusion
                        │  Retrieval   │  → cross-encoder rerank
                        │  (Phase 4)   │
                        └──────┬──────┘
                               ▼
                        ┌─────────────┐
                        │  Generation  │  Groq (Llama 3.3), grounded
                        │  (Phase 5)   │  + cited answer
                        └──────┬──────┘
                               ▼
                        ┌─────────────┐
                        │   FastAPI    │  /query /health /metrics
                        │  (Phase 6)   │
                        └─────────────┘

Project structure

langchain-docs-rag/
├── src/
│   ├── config.py              # typed, validated app settings
│   ├── ingestion/              # crawl LangChain docs, clean, chunk
│   │   ├── loader.py
│   │   ├── chunker.py
│   │   └── pipeline.py
│   ├── embeddings/
│   │   └── embedder.py         # local sentence-transformers
│   ├── vectorstore/
│   │   └── qdrant_store.py     # Qdrant client wrapper
│   ├── retrieval/
│   │   ├── bm25_retriever.py
│   │   ├── reranker.py         # cross-encoder
│   │   └── hybrid_retriever.py # RRF fusion + rerank
│   ├── generation/
│   │   ├── prompts.py
│   │   └── generator.py        # Groq free-tier LLM call
│   ├── observability/
│   │   ├── logging_config.py   # structured JSON logs
│   │   └── metrics.py          # Prometheus metrics
│   └── api/
│       ├── main.py             # FastAPI app + lifespan
│       ├── routes.py
│       └── schemas.py
├── scripts/
│   ├── ingest.py                # CLI: crawl + chunk
│   └── build_index.py           # CLI: embed + index
├── eval/
│   ├── qa_dataset.json          # hand-written eval questions
│   └── run_eval.py              # RAGAS scoring
├── tests/                       # unit tests, no live services required
├── deploy/
│   ├── ec2_setup.sh
│   └── DEPLOY.md                # AWS free-tier deployment guide
├── docker/Dockerfile             # multi-stage, non-root, healthcheck
├── docker-compose.yml            # api + qdrant
├── .github/workflows/ci.yml      # lint, test, docker build
└── pyproject.toml

Tech stack (all free tier / self-hosted)

Layer	Choice	Cost
Embeddings	sentence-transformers (BAAI/bge-small-en)	$0 — runs locally
Vector store	Qdrant, self-hosted via Docker	$0
Retrieval	Hybrid dense + BM25 (RRF fusion) + cross-encoder rerank	$0
LLM	Groq free tier (Llama 3.3 70B)	$0 — rate-limited, no expiry
API	FastAPI + uvicorn	$0
Eval	RAGAS (faithfulness, relevancy, precision, recall)	$0
Observability	Structured JSON logs + Prometheus metrics	$0
Deployment	AWS EC2 t2.micro/t3.micro free tier + S3	$0 within free-tier limits

Getting started (local dev)

# 1. Install dependencies
pip install -e ".[dev]"

# 2. Configure
cp .env.example .env
# edit .env: set GROQ_API_KEY (free at https://console.groq.com)

# 3. Start the vector store
docker compose up -d qdrant

# 4. Crawl + chunk the docs (capped for a fast first run)
python scripts/ingest.py --max-docs 100

# 5. Embed and index
python scripts/build_index.py

# 6. Run the API
uvicorn src.api.main:app --reload

# 7. Ask it something
curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "How do I create a PromptTemplate?"}'

Running the full stack in Docker

docker compose up -d --build

Tests and evaluation

pytest -m "not integration"     # fast unit tests, no live services
pytest                          # everything, needs Qdrant + models + GROQ_API_KEY
python eval/run_eval.py         # RAGAS scores against eval/qa_dataset.json

Deployment

See deploy/DEPLOY.md for the full AWS free-tier walkthrough (EC2 provisioning, security groups, S3 backup of the indexed chunks).

Observability

• GET /health — liveness + Qdrant connectivity
• GET /metrics — Prometheus format: query count, grounded-vs-refused ratio, retrieval/generation/end-to-end latency histograms
• Logs are structured JSON to stdout — pipe into CloudWatch Logs, Loki, or any log aggregator that accepts JSON lines