A production-style Retrieval-Augmented Generation (RAG) answering system that retrieves real scientific documents, generates grounded answers with citations, validates them, caches results, and traces everything — all through a clean FastAPI endpoint.
Built with the SciFact dataset as the knowledge base.
Table of Contents
3. Docker Quick-Start Guide
3.2. Set Up the Project
3.3. Understand the Dockerfile
3.4. Understand Docker Compose
3.5. Build and Run Containers
3.6. Verify the Environment
3.7. Attach VS Code to a Container
3.8. Work with Jupyter Notebooks
3.9. Stop and Remove Containers
3.10. Keep Your Environment Up-to-Date
This repository implements a Docker-first, production-style RAG answering API over the SciFact scientific-claim dataset. It covers the full pipeline — from data preparation and vector indexing through hybrid retrieval, grounded generation, citation building, validation, caching, and observability — exposed as a FastAPI service with phase-by-phase Jupyter notebooks for learning and experimentation.
The system is designed for real workloads: no mocked retrievers, no hardcoded answers, and no fake citations. Every component runs inside containers orchestrated by Docker Compose.
This project is intended for:
- ML / NLP engineers who want to learn how a production RAG pipeline is structured end-to-end
- Backend developers building FastAPI services with retrieval, caching, and observability
- Students and researchers exploring scientific-claim verification with real embeddings and LLMs
- DevOps-minded developers who prefer containerized, reproducible environments over host installs
You should be comfortable reading Python, running terminal commands, and working with Docker. Prior exposure to transformers, vector databases, or RAG concepts is helpful but not required — the notebooks walk through each phase.
By working through this repository you will learn how to:
- Download and prepare the SciFact dataset for retrieval and evaluation
- Build dense (Qdrant) and sparse (BM25) indexes with real embeddings
- Implement dense, BM25, and hybrid RRF retrieval strategies
- Construct prompt-ready context from retrieved documents
- Generate grounded JSON answers with citations using Phi-3.5-mini
- Validate answers with rule-based checks before returning them
- Cache query results in SQLite to reduce latency and cost
- Trace the full pipeline with Arize Phoenix (OpenTelemetry)
- Expose the pipeline through a FastAPI
/queryendpoint - Evaluate answering quality on a SciFact subset
Software (required on the host):
- Docker Desktop with WSL2 integration (Windows) or Docker Engine (Linux/macOS)
- VS Code or Cursor with the Docker and Dev Containers extensions
- NVIDIA GPU + NVIDIA Container Toolkit (recommended for embedding and generation; CPU fallback is possible but slow)
- Git
Knowledge levels:
- Experienced with RAG and FastAPI — Clone the repo, run
make build && make up, then jump to notebook09_api_query_flow.ipynbor call the API directly. - Familiar with Python but new to RAG — Start with notebooks
00–03to understand indexing and retrieval, then continue in order. - Complete beginners — Begin with Section 2. Quick Start and Section 3. Docker Quick-Start Guide, then run notebook
00_environment_check.ipynbbefore proceeding phase by phase.
Folder PATH listing
+---configs <-- YAML configuration files
│ README.md <-- Config folder documentation
│ settings.yaml <-- Application settings
│
+---data <-- Data storage (raw, processed, cache)
│ +---cache <-- SQLite cache and BM25 index
│ │ .gitkeep <-- Keeps empty directory in git
│ │ README.md <-- Cache folder documentation
│ │
│ +---processed <-- Prepared SciFact dataset
│ │ .gitkeep <-- Keeps empty directory in git
│ │ README.md <-- Processed data documentation
│ │
│ +---raw <-- Raw downloaded SciFact files
│ │ .gitkeep <-- Keeps empty directory in git
│ │ README.md <-- Raw data documentation
│ │
│ +---reports <-- Evaluation reports
│ │ README.md <-- Reports folder documentation
│ │
│ README.md <-- Data directory documentation
│
+---docker <-- Docker helper scripts
│ README.md <-- Docker folder documentation
│ entrypoint.sh <-- Container entrypoint script
│ healthcheck.py <-- Container health check
│
+---docs <-- Project documentation
│ Project_Goal.md <-- Project goals and scope
│ README.md <-- Docs folder documentation
│
+---notebooks <-- Phase-by-phase Jupyter notebooks
│ 00_environment_ch… <-- Environment and GPU verification
│ 01_download_and_pr… <-- Download and prepare SciFact
│ 02_build_qdrant_in… <-- Build embeddings and Qdrant index
│ 03_retrieval_orche… <-- Dense, BM25, hybrid retrieval
│ 04_context_constru… <-- Build prompt-ready context
│ 05_generation_stra… <-- Stuff and grounded JSON generation
│ 06_citation_and_va… <-- Citations and validation
│ 07_sqlite_cache.ip… <-- SQLite caching layer
│ 08_observability_p… <-- Phoenix tracing and metrics
│ 09_api_query_flow.… <-- Full API query flow
│ 10_answering_evalu… <-- Answering evaluation
│ README.md <-- Notebooks folder documentation
│
+---scripts <-- CLI utility scripts
│ +---notebook_patches <-- Notebook patch templates
│ │ README.md <-- Patches folder documentation
│ │
│ README.md <-- Scripts folder documentation
│ evaluate_answers.py <-- Evaluation CLI script
│ index_corpus.py <-- Indexing CLI script
│ prepare_data.py <-- Data preparation script
│ run_query.py <-- Sample query script
│
+---src <-- Production source code
│ +---rag_answering_api <-- Main Python package
│ │ +---api <-- FastAPI routes and dependencies
│ │ +---cache <-- SQLite caching layer
│ │ +---citations <-- Citation builder
│ │ +---config <-- Settings and configuration
│ │ +---context <-- Context construction
│ │ +---data <-- Data loaders and preparation
│ │ +---evaluation <-- Answer evaluation
│ │ +---generation <-- Answer generation strategies
│ │ +---indexing <-- Embedding and Qdrant indexing
│ │ +---observability <-- Phoenix tracing and metrics
│ │ +---retrieval <-- Dense, BM25, hybrid retrieval
│ │ +---schemas <-- Pydantic request/response models
│ │ +---validation <-- Answer validation
│ │
│ README.md <-- Source folder documentation
│
+---tests <-- Test suite
│ README.md <-- Test folder documentation
│ conftest.py <-- Pytest fixtures
│ test_health.py <-- Health endpoint tests
│ test_query_endpoin… <-- Query endpoint tests
│
.env.example <-- Environment variables template
.gitignore <-- Git ignore rules
docker-compose.yml <-- Multi-service orchestration
Dockerfile <-- Container image definition
LICENSE <-- MIT License
Makefile <-- Build and run commands
pyproject.toml <-- Python project configuration
README.md <-- Project overview
- Docker Desktop (Windows / macOS / Linux) with WSL2 on Windows
- VS Code or Cursor with the Docker and Dev Containers extensions
- NVIDIA Container Toolkit and a CUDA-capable GPU (recommended)
- Git
- Make (optional but recommended — all common tasks are wrapped in the
Makefile)
-
Clone the repository
git clone git@github.com:RAG-Implementation/production-rag-answering-api.git cd production-rag-answering-api -
Configure environment variables
cp .env.example .env
-
Build and start all services
make build make up
Or without Make:
docker compose up -d --build
-
Open in VS Code / Cursor
- Open the project folder in VS Code or Cursor.
- After containers are running, use Dev Containers: Attach to Running Container… and select
rag-apiorrag-notebook. - Open
/appinside the container to access project files with all dependencies pre-installed.
-
Verify GPU access (recommended)
make gpu-check
-
Run & explore
- JupyterLab:
make notebook→ http://localhost:8888 — run notebooks00through10in order. - FastAPI docs: http://localhost:8000/docs
- Phoenix UI: http://localhost:6006
- Sample query:
make query
- JupyterLab:
-
Shutdown
make down # or: docker compose down
This project is Docker-first. All Python dependencies, GPU libraries, and services run inside containers — you do not need a local Python virtual environment on the host.
- Install Docker Desktop with WSL integration on Windows 11, or Docker Engine on Linux/macOS.
- Install VS Code or Cursor with these extensions: Docker, Dev Containers, Python, and Jupyter.
- Install the NVIDIA Container Toolkit if you have a CUDA-capable GPU (recommended for embedding and generation).
- Ensure Make is available (pre-installed on most Linux/macOS systems; on Windows use WSL).
Clone the repository and create your local environment file:
git clone git@github.com:RAG-Implementation/production-rag-answering-api.git
cd production-rag-answering-api
cp .env.example .envKey project files:
| File | Purpose |
|---|---|
Dockerfile |
CUDA 12.8 + Python 3.11 image for the app and notebook services |
docker-compose.yml |
Orchestrates app, Qdrant, Phoenix, and JupyterLab |
pyproject.toml |
Python dependencies and project metadata (replaces requirements.txt) |
Makefile |
Shortcuts for build, run, test, and data tasks |
.env.example |
Template for all environment variables |
The Dockerfile builds a multi-stage image:
- Base:
nvidia/cuda:12.8.0-runtime-ubuntu22.04with Python 3.11 - Dependencies: Installs PyTorch (CUDA 12.8) and the project via
pip install ".[dev]"frompyproject.toml - Application: Copies source code, installs the package in editable mode, and sets up the entrypoint at
docker/entrypoint.sh - Commands: The same image serves both the FastAPI app (
CMD ["api"]) and JupyterLab (CMD ["notebook"])
docker-compose.yml defines four services on a shared rag-network:
| Service | Container | Port | Role |
|---|---|---|---|
app |
rag-api |
8000 | FastAPI answering API (GPU-enabled) |
qdrant |
rag-qdrant |
6333 / 6334 | Vector database for dense retrieval |
phoenix |
rag-phoenix |
6006 / 4317 | Arize Phoenix observability UI |
notebook |
rag-notebook |
8888 | JupyterLab for phase notebooks (GPU-enabled) |
Persistent volumes: qdrant-storage, phoenix-data, and hf-cache (HuggingFace model cache).
On your host machine, in the project folder:
make build # Build Docker images
make up # Start all services in detached modeOr directly:
docker compose up -d --buildVerify containers are running:
docker compose psAll four services should show status Up with the expected port mappings.
Check GPU access inside the app container:
make gpu-checkConfirm the API is healthy:
curl http://localhost:8000/healthOpen service UIs:
- FastAPI Swagger: http://localhost:8000/docs
- JupyterLab: http://localhost:8888
- Phoenix: http://localhost:6006
- Open VS Code or Cursor and press
Ctrl+Shift+P. - Select Dev Containers: Attach to Running Container….
- Choose
rag-api(for API development) orrag-notebook(for notebooks). - Open the folder
/appinside the container. - Use the integrated terminal to run scripts, tests, and Make targets via
docker compose exec.
Start JupyterLab:
make notebookOpen http://localhost:8888 in your browser, or open .ipynb files directly in VS Code/Cursor after attaching to rag-notebook.
Run notebooks in order (00 → 10). Each notebook imports real modules from src/rag_answering_api/ — they are not standalone mock scripts.
Alternatively, run data and indexing tasks from the CLI:
make prepare-data # Download and prepare SciFact
make index # Build embeddings and index into Qdrantmake down # Stop all services
make clean # Stop, remove volumes, and remove local images-
Rebuild after Dockerfile or dependency changes:
docker compose up -d --build
-
After adding a Python dependency, update
pyproject.tomlunder[project.dependencies]or[project.optional-dependencies], then rebuild:make build
-
Pull latest base images:
docker compose build --pull
┌─────────────────────────────────────────────────────────┐
│ Client Request │
│ POST /query { question, ... } │
└──────────────────────┬──────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────┐
│ FastAPI App │
│ ┌──────────┐ ┌───────────┐ ┌──────────────────────┐ │
│ │ Cache │ │ Retrieval │ │ Generation │ │
│ │ (SQLite) │ │ Engine │ │ (Phi-3.5-mini) │ │
│ └──────────┘ └─────┬─────┘ └──────────┬───────────┘ │
│ │ │ │
│ ┌───────────────────┴────────────────────┘ │
│ │ Context Builder → Citation Builder → Validator │
│ └───────────────────────────────────────────────────┐ │
│ │ │
│ Phoenix Tracing (OpenTelemetry) ◄──────────────────┘ │
└──────────┬──────────────┬────────────────────────────────┘
│ │
┌─────▼─────┐ ┌────▼────┐
│ Qdrant │ │ Phoenix │
│ (Vectors) │ │ (UI) │
└───────────┘ └─────────┘
| Service | Port | Description |
|---|---|---|
| app | 8000 | FastAPI answering API |
| qdrant | 6333 | Qdrant vector database |
| phoenix | 6006 | Arize Phoenix observability UI |
| notebook | 8888 | JupyterLab interactive environment |
| Method | Endpoint | Description |
|---|---|---|
| GET | /health |
Service health check |
| POST | /query |
Run RAG answering pipeline |
| POST | /evaluate |
Run evaluation on SciFact subset |
| GET | /metrics |
Runtime metrics |
| GET | /cache/stats |
Cache statistics |
curl -X POST http://localhost:8000/query \
-H "Content-Type: application/json" \
-d '{
"question": "Does drinking green tea reduce the risk of cancer?",
"retrieval_mode": "hybrid_rrf",
"generation_mode": "grounded_json",
"top_k": 5
}'| # | Notebook | Purpose |
|---|---|---|
| 00 | 00_environment_check.ipynb |
Verify Docker, CUDA, GPU, services |
| 01 | 01_download_and_prepare_scifact.ipynb |
Download real SciFact data |
| 02 | 02_build_qdrant_index.ipynb |
Build embeddings and index into Qdrant |
| 03 | 03_retrieval_orchestration.ipynb |
Dense, BM25, and hybrid RRF retrieval |
| 04 | 04_context_construction.ipynb |
Build prompt-ready context |
| 05 | 05_generation_strategies.ipynb |
Stuff and grounded JSON generation |
| 06 | 06_citation_and_validation.ipynb |
Citations and rule-based validation |
| 07 | 07_sqlite_cache.ipynb |
SQLite caching layer |
| 08 | 08_observability_phoenix.ipynb |
Phoenix tracing and metrics |
| 09 | 09_api_query_flow.ipynb |
Full API query flow |
| 10 | 10_answering_evaluation.ipynb |
Answering evaluation on SciFact |
This project uses NVIDIA RTX 5070 Ti for:
- Embedding generation (
BAAI/bge-small-en-v1.5) - Answer generation (
microsoft/Phi-3.5-mini-instruct) - Batch processing
Requirements:
- NVIDIA Container Toolkit installed
- Docker Desktop with WSL2 GPU support
- CUDA 12.4+
Verify with make gpu-check after starting containers.
make build # Build Docker images
make up # Start all services
make down # Stop all services
make notebook # Open JupyterLab
make prepare-data # Download and prepare SciFact data
make index # Build embeddings and index into Qdrant
make query # Run a sample query
make evaluate # Run answering evaluation
make test # Run pytest
make lint # Run Ruff linting
make format # Format code with Ruff
make gpu-check # Verify GPU access inside Docker
make logs # View service logs
make clean # Remove containers, volumes, and imagesThis project is released under the MIT License.
For questions not addressed in the resources above, please connect with Max Ghadri on LinkedIn for personalized assistance.