title	MizuBot
emoji	🌊
colorFrom	blue
colorTo	indigo
sdk	docker
pinned	false
app_port	7860

Mizu (水): Hyper-Kamiokande AI Hub

MizuBot is a RAG (Retrieval-Augmented Generation) chatbot for the Hyper-Kamiokande (HK) collaboration. It answers questions about collaboration personnel and structure, internal Technical Notes, journal papers, and operational documentation, by retrieving content from a local Chroma vector store and passing the retrieved chunks through a local LLM (Ollama / mistral-small).

✨ Highlights

• Local LLM by default — mistral-small via Ollama, kept resident (OLLAMA_KEEP_ALIVE=24h). Falls back to Groq Cloud only if no local model is configured.
• Streaming output — tokens render as the LLM emits them, so users see the answer flow within ~500 ms instead of staring at a spinner.
• Two-engine retrieval — ProfileHunter for personnel queries (uses HKDB role taxonomy + bare-role tokens), TechHunter for Technical-Note / paper / web content (Chroma MMR + cross-encoder rerank, source filtering via tn-mapping.csv).
• Persona switching — Mizu Doc (general), Mizu Tech (detector/ops), Mizu Phys (physics), Mizu Soft (software — under way 🚧).

🧠 Personas

Persona	Role	Status
Mizu Doc	General-purpose assistant (default)	Live
Mizu Tech	Detector hardware, PMTs, excavation	Live
Mizu Phys	Neutrino physics, sensitivities	Live
Mizu Soft	WCSim, fiTQun, data processing	Under way 🚧

Personas marked under way are tagged with wip: True in app.py:PERSONAS. The UI shows a warning banner so testers know the agent is still being built.

🏗️ Architecture

User question
   │
   ▼
ProfileHunter ─ first pass for "who is …?" queries
   │ (HKDB/roles/* — synonyms, leadership stems, bare roles,
   │  domain stopwords)
   │
   ├─ matches? ──► reranked person/glossary docs
   │
   └─ no match
        │
        ▼
TechHunter ─ Technical Notes, papers, web
   │ (Chroma MMR fetch_k=200, λ=0.5 → cross-encoder rerank top-15)
   │
   ▼
build_answer_stream(docs, question, persona, llm)
   │ (shared prompt assembled by _build_prompt_chain)
   │
   ▼
LLM stream → st.write_stream → user

📂 Project layout

File	Purpose
app.py	Streamlit UI, retrieval pipeline, prompt assembly
chatbot.py	LLM and embeddings setup (setup_llm, setup_embeddings)
add_all_content_smart.py	Incremental ingestion: scans new_documents/, smart-updates Chroma
ingest.py	Full rebuild (nuclear option) — deletes and re-creates Chroma
ingestion_HRInfo.py	Person / institution ingestion from HR Excel sheets
convert_excel_to_text.py	Excel → plain-text helper used during ingestion prep
create_stopwords.py	Generates HKDB/roles/domain_stopwords.txt from current corpus
evaluate.py	RAGAS evaluation runner against benchmark_data.csv
profile_query.py	End-to-end pipeline timing (cold start + per-query)
run_mizu.sh	Activate env + run Streamlit
startup.sh	One-shot env setup (deps, Ollama install + pull, model env vars)
HKDB/roles/*.txt	Role taxonomy (synonyms, stems, suffixes, stopwords)
documents/wiki/technicalnotes/	Per-TN PDF tree + tn-mapping.csv (filename → HK-TN-NNNN + title)
new_documents/bibliography.csv	Journal-paper metadata (filename, title, DOI, URL)
chroma_db/	Persistent vector store

🚀 Quick start (local)

# 1. One-off environment setup (installs deps, brews Ollama, pulls models)
source startup.sh

# 2. Run the app
streamlit run app.py

The app opens at http://localhost:8501. First launch pulls the embeddings model (all-MiniLM-L6-v2) and warms Ollama.

LLM configuration

Variable	Effect
LOCAL_LLM_MODEL	Ollama model name (defaults to mistral-small)
LOCAL_LLM_PATH	Path to a local GGUF file (alternative to the named model)
OLLAMA_KEEP_ALIVE	Keep model resident; startup.sh sets it to 24h
GROQ_API_KEY	Cloud fallback if no local LLM is reachable
APP_PASSWORD	Optional gate for EncryptedCookieManager (24 h cookie TTL)

Secrets can live in .streamlit/secrets.toml (gitignored) or as environment variables — both are checked by chatbot._get_secret.

📥 Ingesting documents

Daily driver

Drop new files into new_documents/ (recurses through subfolders) and optionally update new_documents/bibliography.csv and documents/wiki/technicalnotes/tn-mapping.csv. Then:

python add_all_content_smart.py

The script:

1. Parses tn-mapping.csv (filename → HK-TN-NNNN + title) and stamps the headers FILENAME / DOCUMENT ID / TITLE onto each chunk so the LLM sees the TN identity inline.
2. Parses bibliography.csv and attaches citation_display, citation_doi, citation_url metadata to journal-paper chunks.
3. Smart-updates: deletes prior chunks for any filename it is about to re-add, so re-ingesting the same file replaces (not duplicates).

Full rebuild

python ingest.py

Wipes chroma_db/ and rebuilds from scratch. Use after large re-organisations of the source tree.

tn-mapping.csv format

Pipe-delimited; the columns are Note | Date | File | Title | Problems. Avoid wrapping rows in double quotes — csv.reader(delimiter="|") and pandas.read_csv(delimiter="|") both interpret a leading " as a quoted field and will collapse the entire row into a single column. If a title contains a quote character, write it plainly (or replace with a single quote / dash); do not escape with "".

🧪 Evaluation

benchmark_data.csv holds the validation set. Columns:

Column	Purpose
question	The query sent to MizuBot.
ground_truth	The expected reference answer (used by RAGAS scoring).
expected_hunter	profile or tech — which retrieval engine the question targets. Used for the routing-accuracy summary.
persona	Optional. Which persona answers the question. See table below.

Persona routing strategy

Each row's persona column decides which MizuBot persona answers it:

persona cell	Behaviour
(empty)	Defaults to Mizu Doc (preserves pre-schema behaviour — the existing 41 rows leave this blank).
Mizu Doc / Mizu Phys / Mizu Tech / Mizu Soft	Runs that single persona.
all or *	Fans out across every persona; one report row per (question, persona) pair. Useful for parity testing.

The eval mirrors the chat handler in app.py exactly: ProfileHunter → TechHunter fallback → physics-paper injection for physics queries → persona-driven source bias → build_answer(). So eval scores should track what real users see in Streamlit.

Running

The thin wrapper activates ./env for you and forwards every flag through to evaluate.py:

./run_eval.sh                                 # every row, default report path
./run_eval.sh --persona "Mizu Phys"           # only Mizu Phys rows
./run_eval.sh --persona-override "Mizu Phys"  # force every row to Mizu Phys
./run_eval.sh --limit 5                       # smoke test (first 5 tasks)
./run_eval.sh -h                              # full CLI reference

Equivalent without the wrapper:

source ./env/bin/activate
python evaluate.py --persona "Mizu Phys"

CLI flags

Flag	What it does
--persona NAME	Keep only rows whose persona column resolves to NAME. Rows tagged all are kept (they include every persona). Mutually exclusive with --persona-override.
--persona-override NAME	Ignore the column; force every row to NAME. Comparison runs like "answer the whole benchmark as Mizu Phys" use this.
--limit N	Only run the first N tasks after filtering. Smoke-test pattern.
--report-file PATH	Custom output CSV path. Default mizubot_evaluation_report.csv. Use this when running back-to-back overrides so they don't overwrite each other.
--benchmark-file PATH	Custom benchmark CSV path. Default benchmark_data.csv.

The output report gains a persona column. The stdout summary shows a per-metric overall plus a per-persona breakdown when more than one persona appears in the run.

Profiling (timing only)

For timing without scoring, use profile_query.py:

source ./env/bin/activate
python profile_query.py

Reports cold-start cost (embeddings load, Chroma warmup, Ollama warmup) and per-stage timing for one ProfileHunter and one TechHunter query.

🚢 Deployment

🚧 Hetzner Cloud deploy under way. The plan is a single VM (CPX41 CPU box or GX44 GPU box) running Ollama + Streamlit as systemd services behind an HTTPS reverse proxy, with the existing EncryptedCookieManager providing the password gate. Instructions will land here once the box is provisioned.

For the Hugging Face Space build the frontmatter at the top of this file is honoured (sdk: docker, app_port: 7860).

Production secrets

For deploy, prefer environment variables over .streamlit/secrets.toml. The local file is convenient on your laptop, but a server-side secrets.toml is gitignored, has no clear provisioning step, and gets wiped on a fresh checkout. Environment variables survive re-deploys and are the standard pattern. get_secret() already reads os.environ first and only falls back to st.secrets, so the same code works for both targets.

Target	Where APP_PASSWORD lives
Local dev	.streamlit/secrets.toml (gitignored)
Hetzner systemd	/etc/mizubot/env (chmod 0600, owned by service user) referenced via EnvironmentFile= in the unit
Docker / docker-compose	docker run -e APP_PASSWORD=… or environment: in docker-compose.yml
HF Space	Repository → Settings → Secrets → APP_PASSWORD

Pick a non-trivial shared password (≥16 chars, mixed). The cookie encryption key is derived from it, so a weak password is also a weak tamper-resistance key.

Cookie & rotation behaviour

Once APP_PASSWORD is set the gate writes an encrypted mizu_auth_* cookie on successful login and re-validates it on every page load against a 24 h TTL. After 24 h the cookie auto-clears and the user re-types the same password. No server-side session store, no DB, nothing to clean up between launches.

To rotate (e.g. a tester leaks the password): change the env value, systemctl restart mizubot (or docker compose up -d). Existing cookies fail to decrypt under the new key, so every browser is forced to re-login with the new value on next request.

Sunset

When SSO lands (Keycloak / Auth0 / KCL identity), drop APP_PASSWORD from the environment and remove the if app_password: block from main(). The rest of the app is unaware of the gate.

🐞 Reporting issues

For collaboration-internal issues, contact Francesca Di Lodovico (f.r.di.lodovico@gmail.com, KCL). When filing a bug, please attach:

• the exact question asked,
• the persona that was active,
• the returned answer + sources (if any),
• the expected answer.

For benchmark regressions, add a row to benchmark_data.csv and run python evaluate.py to capture before/after numbers.