An open, evaluated, domain-specialized RAG platform. The core engine is
domain-agnostic; each domain profile is a drop-in folder of
{curated corpus + domain config + eval set}. First profile: energy.
Think NotebookLM's notebooks — but curated, domain-tuned, and measured for trustworthiness (RAGAS), open, and multi-provider (LiteLLM). Naive LLMs invent numbers; grounded cites its figures or abstains.
Slice 0 — scaffold. The package structure, config, and the energy profile
skeleton are in place as import-light stubs. No pipeline logic yet — that
lands slice by slice: ingestion → retrieval + answer → CLI → evaluation → UI.
The full ML stack is declared in pyproject.toml but not needed yet. The
scaffold imports and runs on typer + pyyaml alone:
uv venv
uv pip install -e . --no-deps # the package, without the heavy stack
uv pip install typer pyyaml # the only runtime deps Slice 0 uses
uv run grounded --help
uv run grounded --profile energy ask "How has utility-scale solar PV cost changed?"From Slice 1 onward, install the whole stack with uv sync.
profiles/<domain>/ → download → ingest (parse + chunk + embed) → per-domain
Chroma + BM25 → hybrid retrieve + BGE rerank → grounded, cited answer (or
abstain) via LiteLLM → RAGAS evaluation. See SPEC.md and
CLAUDE.md.
| Path | What |
|---|---|
src/grounded/ |
the domain-agnostic engine (stubs in Slice 0) |
config/settings.yaml |
provider, models, chunk, top_k, rerank_k, profile |
profiles/energy/ |
sources.yaml, config.yaml, golden_qa.yaml |
eval/run_eval.py |
RAGAS runner |
data/ |
raw/, processed/ (gitignored) |
A thin FastAPI shell over the existing engine (no retrieval/answer/eval logic is
rewritten). fastapi/uvicorn live in a separate web extra, out of the core
and ml lanes.
uv sync --extra ml --extra web # engine + web deps
uv run uvicorn "grounded.api:create_app" --factory --host 127.0.0.1 --port 8000
# interactive docs at http://127.0.0.1:8000/docsThe retriever (embedder + reranker + Chroma) is built once at startup and
reused across requests. /api/ask needs Ollama running; the read-only endpoints
do not. CORS origin is configurable via GROUNDED_CORS_ORIGINS (default
http://localhost:3000).
| Endpoint | curl |
|---|---|
POST /api/ask |
curl -sN -X POST localhost:8000/api/ask -H 'content-type: application/json' -d '{"question":"How much has utility-scale solar PV LCOE fallen?","profile":"energy"}' |
POST /api/ask?stream=true (SSE) |
curl -sN -X POST 'localhost:8000/api/ask?stream=true' -H 'content-type: application/json' -d '{"question":"How much has solar PV LCOE fallen?"}' |
POST /api/search |
curl -s -X POST localhost:8000/api/search -H 'content-type: application/json' -d '{"query":"solar pv lcoe","profile":"energy"}' |
GET /api/sources |
curl -s 'localhost:8000/api/sources?profile=energy' |
GET /api/document/{id} |
curl -s 'localhost:8000/api/document/irena-rpgc-2024?profile=energy' -o out.pdf |
GET /api/eval/latest |
curl -s localhost:8000/api/eval/latest |
GET /api/settings |
curl -s localhost:8000/api/settings |
GET /api/profiles |
curl -s localhost:8000/api/profiles |
GET /api/health |
curl -s localhost:8000/api/health |
Errors map to precise codes: 409 no_index (run ingest), 503 ollama_down
(start Ollama), 502 llm_error, 404 not_found (missing PDF / no eval results).
PDF page numbers are physical file pages — correct for a viewer's #page=N.
The frontend/ folder is a Next.js (App Router + TypeScript) app wired to the
FastAPI backend. Run the two together:
# 1 — backend (terminal A). Port 8000 is often taken (e.g. by `operon`) → use 8077.
uv sync --extra ml --extra web
ollama serve & # needed for /api/ask
uv run uvicorn "grounded.api:create_app" --factory --host 127.0.0.1 --port 8077
# 2 — frontend (terminal B) → http://localhost:3000
cd frontend
npm install # also copies the PDF.js worker into public/
cp .env.local.example .env.local # NEXT_PUBLIC_API_BASE=http://localhost:8077/api
npm run devOpen http://localhost:3000. You should see the 3-region workspace: the left
rail lists the real 9 energy sources; asking a question streams a cited answer;
clicking a irena-rpgc-2024·p87 citation opens that PDF in the viewer at the
physical page and highlights the passage; "view detail" shows the corpus RAGAS
trust scores. Next dev runs on :3000, which the backend's GROUNDED_CORS_ORIGINS
default already allows. See docs/ for the API contract and architecture.
To be reported after Slice 4: a RAGAS baseline → improvement (e.g. "0.72 → 0.89 after reranking") and a benchmark vs. a deployed general tool. Slice 0 has no measurable pipeline yet.
Never commits copyrighted PDFs. The corpus is fetched at runtime from
profiles/energy/sources.yaml; prefer CC0 / gov / free-IGO sources.