OPISbot

Research codebase for OPIS (IVF outcome prediction) plus a RAG assistant that answers questions using retrieved documents and the user’s prediction context.

Repository layout

Path	Purpose
apps/	Streamlit UIs — see apps/README.md for each app
src/	Streamlit entry shims (app*.py → apps/), logs, vignette JSON, package root for imports
src/opisbot/	Backend package — domain (prediction), RAG pipeline, config loaders, evaluation batch runner
configs/	YAML templates for RAG models, batch eval, judges, app hints
data/	Source documents for ingestion / indexing
embeddings_test_HG/	Default FAISS index (created or updated by ingestion)
evaluation/	Batch scripts (openai_run.py, ollama_run.py), sample datasets

Quick start

1. Environment

   uv sync
   source .venv/bin/activate   # optional
   export OPENAI_API_KEY=...

2. Run a Streamlit app (from repo root)

   streamlit run apps/opis_chat.py

   Other UIs: apps/README.md.

3. Batch evaluation (RAG + optional LLM/rule judges)

   PYTHONPATH=src python evaluation/openai_run.py --eval-config configs/eval/default_batch.yaml

   Skip extra judge API calls: --no-judges. Ollama single-turn: evaluation/ollama_run.py (see that file’s docstring).

4. Intent classification eval (lookup vs data/feature_category.json, separate from LLM judges)

   PYTHONPATH=src python evaluation/run_intent_eval.py

   RAG chat uses the same lookup for routing: OOD-led vs in-distribution-led retrieval over the pooled FAISS index (see configs/rag/baseline.yaml → intent).

Architecture notes

• Canonical backend lives in src/opisbot/ (pipeline, prompts, retriever, prediction). UIs import opisbot after adding src to sys.path.
• Migration: Duplicate modules (src/RAG.py, src/retriever.py, src/OPISpredict.py, old rag_pipeline*.py, doc_index.py, and copies under evaluation/multiturn_human/) have been removed. Use opisbot.core.rag.pipeline, opisbot.core.domain.prediction, and opisbot.core.rag.retriever instead.
• Paths: RAG resolves data/ and index dirs relative to the repository root (not the process working directory).

Configuration

• configs/rag/*.yaml — LLM provider/model, retrieval k, embedding id, optional paths overrides.
• configs/eval/default_batch.yaml — points at a RAG config, JSONL dataset, and judge file.
• configs/eval/judges.yaml — LLM rubric judges + rule-based checks.

Further reading

• App-specific behaviour and run commands: apps/README.md
• evaluation/multiturn_human/app.py (human-in-the-loop simulation) now uses the same opisbot stack; run it with src on PYTHONPATH or from a cwd where the app’s bootstrap finds the repo.

License / usage

Use and deployment are your responsibility; ensure compliance with clinical, data-protection, and API terms for your environment.