Chat with your medical documents — locally, privately, with citations.
Upload a PDF (textbook chapter, clinical notes, lab report) and ask anything. chatMed reads every page, retrieves the most relevant passages, and streams a grounded answer with page-level citations back to you. Everything runs on your own machine — your documents never leave it.
MedAssist is the product name shown in the UI;
chatMedis the project.
Medical study material is dense and hard to query by hand. chatMed turns a pile of PDFs into a searchable companion: ask a question, get the exact answer plucked from your own documents, and verify it against the source page it came from. No internet, no cloud account, no data leaving your laptop.
- Evidence-based Q&A — answers are grounded in your documents with page
citations (
[p. 12]/[Title, p. 4]) you can click to open the source PDF. - Four task modes — route any query to the right job automatically: answer questions, summarize a document, extract exact values, or compare across sections/documents.
- Streaming answers — tokens appear as they generate (ChatGPT-style) with an animated "thinking" indicator.
- Document library — sidebar panel to rename, delete, restore, and share your documents; deleted chats stay shareable via restore.
- Private by default — anonymous per-session isolation out of the box, or Google OAuth for multi-device history.
- Medical safety rails — emergency queries short-circuit to a static crisis message; dosage questions attach a caution block; low-confidence answers abstain with an explicit "I don't know" instead of hallucinating.
- Opinionated CLI tooling — one command (
make dev) boots the whole stack.
| Component | Tool | Why |
|---|---|---|
| Chat UI | Chainlit | Python-native ChatGPT-style interface with sidebar, file upload, and custom front-end theming |
| PDF parsing | Docling | Layout-aware parsing + OCR of scanned pages, producing structured chunks |
| Embeddings | Ollama nomic-embed-text |
Local dense vector embeddings, zero cloud cost |
| Vector store | Qdrant | Hybrid (dense and sparse) vector search with payload filtering |
| Reranking | FlashRank (ms-marco-MiniLM-L-12-v2) |
Cross-encoder reranking of top candidates; drives the abstention gate |
| LLM | Ollama deepseek-r1:14b |
Streaming reasoning model for grounded answers, run locally |
| Router / chitchat | Ollama qwen2.5:3b |
Fast keyword + LLM query classifier and small talk |
| Persistence | SQLite | Users, sessions, documents, ACL grants, messages, threads |
| API framework | FastAPI (via Chainlit) + httpx | Streaming SSE/NDJSON transport, ASGI middleware |
All providers are interchangeable behind adapters — swap any of them for a cloud OpenAI-compatible endpoint (Azure OpenAI, Foundry, Groq, Gemini) without touching pipeline code.
┌──────────────────────────────────────────────────────────────────────┐
│ app.py (Chainlit callbacks) │
│ library UI · auth · history · citations · safety · orchestration │
└───────────────┬──────────────────────────────────────────┬───────────┘
│ │
▼ ▼
rag/RAGEngine.py ──────────────────► store.py (SQLite)
facade · composition root · users/messages
│ · documents/ACL
┌─────────┼───────────────────────┐
▼ ▼ ▼
Ingestor Retriever Generator
(Docling) (Qdrant+FlashRank) (Ollama streaming)
│ parse │ hybrid search │ task routing
│ chunk │ ACL/owner filter │ safety gate
│ PHI-redact │ soft-delete │ prompt security
│ content-add │ abstention │ circuit breaker
└─────┬───────┴──────┬─────────────┴──────────┬──────────┘
▼ ▼ ▼
documents Qdrant Ollama
(PDFs) (vector store) (LLMs + embeds)
- Resolve identity — map the requester to an owner id (
google:…oranon:<session>); every downstream retrieval is scoped to them. - Safety gate — emergency keywords short-circuit to a static crisis message before anything else runs.
- Route the task — a small LLM (with keyword fallback) classifies the
query into
qa / summarize / extract / compare / chitchat. - Retrieve — Qdrant hybrid (dense + sparse) search, filtered by owner + explicit ACL grants + thread scope + non-deleted; results are deduped.
- Rerank — FlashRank re-orders candidates; if the top score is below the abstention threshold, the model refuses rather than guesses.
- Generate — the primary LLM streams a citation-anchored answer, then a disclaimer is appended.
- Validate — citations are checked against the pages actually retrieved; unsupported page references and uncited numeric claims are logged.
# 1. Set up env (template → .env), edit as needed
cp .env.example .env
# 2. Install dependencies + dev tools
uv sync --group dev
# 3. Pull the models Ollama needs (idempotent)
make models
# 4. Boot the stack (Qdrant + app on http://localhost:8000)
make devNo
.envedits required for a bare-bones local run — sensible defaults are baked in. Start withmake dev.
| Model | Purpose |
|---|---|
deepseek-r1:14b |
Primary LLM (QA / summarize / extract / compare) |
qwen2.5:3b |
Fast router + chitchat model |
nomic-embed-text |
Embeddings |
Open http://localhost:8000 and:
- Attach a PDF with the paperclip (or drag & drop, or drop files into
documents/withPRELOAD_KB=true). - Ask a question — e.g. "What is the recommended fluid resuscitation rate for septic shock?"
- Verify — click the cited page chip to open the exact source.
You can also pre-index a whole folder without the UI:
make index # python index_pdfs.py --dir documentschatmed/
├── app.py Chainlit callbacks + turn orchestration
├── auth.py Google OAuth + anonymous session isolation
├── history.py Chainlit sidebar history + resume
├── library.py HTTP API for the document library (rename/delete/share)
├── store.py SQLite persistence (users, docs, ACL, messages, threads)
├── citations.py Citation + claim validation, action-advice heuristics
├── observability.py Logging, no-op metrics, circuit breaker
├── middleware.py Fetch-Metadata CSRF / same-site guard
├── index_pdfs.py Batch CLI pre-indexer (idempotent, incremental)
├── rag/ Retrieval pipeline
│ ├── config.py Pydantic env-driven configuration (RAGConfig)
│ ├── engine.py RAGEngine facade / composition root
│ ├── ingestor.py Docling parse → chunk → filter → redact → stamp
│ ├── retriever.py Qdrant hybrid search, rerank, ACL, cache, abstention
│ ├── generator.py Task routing, safety gate, streaming generation
│ ├── llm.py Ollama ⇄ OpenAI-compatible provider adapters
│ ├── embeddings.py Ollama / local / OpenAI embedding providers
│ ├── relevance.py Medical-relevance classifier (LLM + keyword)
│ ├── prompts.py Task prompts + prompt-injection defense
│ ├── redaction.py PHI redaction (Presidio NER + regex fallback)
│ └── constants.py Shared vocabulary, metadata keys, safety texts
├── evals/ Golden-set evaluation harness (recall@k, MRR, coverage)
├── public/ Custom UI assets (clinical.css, custom.js, avatar)
├── tests/ 29 file unit suite (297 tests)
└── Makefile Dev / test / lint / eval automation
Everything is environment-driven via .env (see .env.example
for the full template with comments). Highlights:
| Setting | Default | Purpose |
|---|---|---|
OLLAMA_LLM_MODEL |
deepseek-r1:14b |
Primary answer model |
QDRANT_COLLECTION_NAME |
enterprise_kb |
Vector collection |
INITIAL_K / FINAL_K |
20 / 5 |
Retrieve-then-rerank counts |
ABSTENTION_SCORE |
(off) | Rerank threshold; answers below abstain |
ROUTER_MODE |
hybrid |
rules (keywords) or hybrid (LLM fallback) |
SAFETY_ROUTER |
true |
Emergency/dosage short-circuits |
PHI_REDACTION |
false |
Regex PHI redaction; [phi] extra enables NER |
AUTH_REQUIRED |
false |
Fail closed without an authenticated identity |
PRELOAD_KB |
false |
Auto-index the documents/ directory |
LLM providers. Set LLM_PROVIDER=openai + LLM_BASE_URL to point at any
OpenAI-compatible endpoint (Azure OpenAI, Foundry). Embeddings can switch via
EMBEDDING_PROVIDER=openai|local|ollama. Storage/db/OCR are also provider-
swappable for a cloud path.
- Unit + integration tests —
uv run pytest -q(297 tests) with heavy dependency injection; CI runs them with ruff + a syntax battery. - Offline golden-set eval —
make evalchecks recall@k, MRR, keyword coverage, and citation precision against a curated medical corpus — fully offline with a mock retriever, so it gates CI without any services.
make lint # ruff check .
make check # syntax compilation battery
make test # pytest
make eval # offline golden-set eval (strict)
make eval-live-full # live Qdrant + embeddings + LLMEvery push / PR to main runs the full gate via GitHub Actions
(.github/workflows/ci.yml): ruff → syntax → pytest → offline eval.
- Local-first — documents, embeddings, and model inference stay on your machine; no telemetry is sent anywhere.
- Owner-scoped retrieval — every search is filtered by the requester's identity, ACL grants, and thread; isolation is enforced at the vector query, not in prompts.
- Prompt-injection defense — retrieved text is wrapped as untrusted context and explicitly barred from being treated as instructions.
- PHI redaction — optional SSN/phone/MRN/DOB redaction (Presidio NER or regex) during ingestion.
- CSRF guard — Fetch-Metadata middleware rejects cross-site requests.
- Fail-safe medical UX — emergency queries return a static crisis message before any generation; low-confidence answers abstain instead of guessing.
- Best-effort heuristics — safety and claim checks are review signals, not certified clinical advice. Always consult a qualified professional.
MIT © 2026 Ishan Arora