Currently in progress

Working on creating debate model between models to generate more accurate aanswers.

BioExpert(In collaboration with Carnegie Mellon School of Computer Science)

Multi-expert ensemble agent for RNA analysis with RAG-grounded verification.

User Prompt
     │
┌────┼────┐
▼    ▼    ▼
BE1  BE2  BE3     ← 3 BioExperts (Gemma 3, Llama 3.1, Qwen 2.5 via Ollama)
└────┼────┘
     ▼
Evaluation Stage  ← RAG-based claim validation (FAISS + open-access literature)
     │
     ▼
Verified Output   ← Hallucination-filtered, evidence-grounded answer

Features

• RNA analysis tools — GC content, motif detection, secondary structure prediction (ViennaRNA / Nussinov fallback)
• 3-model ensemble — Gemma 3 4B, Llama 3.1 8B, Qwen 2.5 7B via Ollama (all free, local)
• RAG validation — FAISS vector store over open-source biology textbooks; claim-level verification
• Benchmarks — PubMedQA, MMLU-Bio, GPQA-Bio, WMDP-Bio, LAB-Bench adapters
• SafeScientist metrics — hallucination rate, faithfulness, consistency scoring

Setup

1. Install Ollama

# macOS
brew install ollama

# or download from https://ollama.com

Start the Ollama server:

ollama serve

2. Pull the models

ollama pull gemma3:4b
ollama pull llama3.1:8b
ollama pull qwen2.5:7b

3. Install BioExpert

cd bioexpert_model
pip install -e ".[dev]"

4. (Optional) Install ViennaRNA for accurate folding

# macOS
brew install viennarna

# or conda
conda install -c bioconda viennarna

Without ViennaRNA, a Nussinov-based fallback is used automatically.

Quick Start

Analyze an RNA sequence

bioexpert analyze --seq "GGGCCCUUAGCUCAGCUGGGAGAGCGCCUGCUUUGCACGCAGGAGGUCUGCGGUUCGAUCCCGCUAAGGGCCA"

Build the RAG corpus

Download open-access biology references and index them:

bioexpert download-corpus
bioexpert index data/corpus

Ask a question

bioexpert ask "What type of RNA is this and what is its likely function?" \
  --seq "GGGCCCUUAGCUCAGCUGGGAGAGCGCCUGCUUUGCACGCAGGAGGUCUGCGGUUCGAUCCCGCUAAGGGCCA"

Run benchmarks

bioexpert benchmark pubmedqa --n 50
bioexpert benchmark mmlu_bio --n 100
bioexpert benchmark gpqa_bio --n 50

Project Structure

bioexpert_model/
├── bioexpert/
│   ├── __init__.py           # Package root
│   ├── orchestrator.py       # Main pipeline
│   ├── schemas.py            # Pydantic data models
│   ├── cli.py                # Typer CLI
│   ├── experts/
│   │   ├── expert.py         # Single BioExpert (Ollama wrapper)
│   │   ├── ensemble.py       # 3-model ensemble + majority voting
│   │   └── prompts.py        # System/task prompts
│   ├── rag/
│   │   ├── ingest.py         # PDF/text chunking
│   │   ├── retriever.py      # FAISS index + embedding
│   │   ├── validator.py      # Claim verification
│   │   └── corpus_downloader.py  # Open-access text downloader
│   ├── tools/
│   │   ├── rna_features.py   # GC content, motifs, sequence analysis
│   │   └── rnafold.py        # ViennaRNA / Nussinov wrapper
│   ├── eval/
│   │   └── metrics.py        # SafeScientist + BixBench metrics
│   └── benchmarks/
│       ├── loaders.py        # HuggingFace dataset loaders
│       └── runner.py         # Benchmark evaluation loop
├── data/
│   ├── corpus/               # Place PDFs / textbooks here
│   └── indexes/              # FAISS index output
├── examples/
│   ├── quickstart.py
│   └── run_benchmark.py
├── tests/
│   ├── test_rna_tools.py
│   └── test_schemas.py
├── pyproject.toml
└── README.md

Benchmarks

Benchmark	Source	Questions	Format
PubMedQA	HuggingFace qiaojin/PubMedQA	500	MC (3)
MMLU-Bio	HuggingFace cais/mmlu (7 bio subjects)	~1300	MC (4)
GPQA-Bio	HuggingFace Idavidrein/gpqa	~78	MC (4)
WMDP-Bio	HuggingFace cais/wmdp	~1273	MC (4)
LAB-Bench	HuggingFace futurehouse/lab-bench	~199	MC (2-10)

RAG Corpus Sources (open access)

• NCBI Bookshelf — Molecular Biology of the Cell chapters (open access)
• PubMed Central — open access RNA review papers
• Rfam documentation — RNA family descriptions
• Curated seed corpus — verified RNA biology reference facts

Place additional PDFs or text files in data/corpus/ and re-run bioexpert index.

License

MIT