OPERATIONS.md

Operations

First-Time Setup

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -e .
rag init

If rag is not on PATH, use .venv/bin/rag from the repo root:

.venv/bin/rag status

If the package is installed but the console script is unavailable:

python -m rag_pipeline.cli status

Demo Build

rag rebuild articles --yes
rag status
rag eval
rag context "Which sources discuss antibody orientation and ToF-SIMS?"

Production Build

rag rebuild corpus --yes
rag status
rag eval --file eval/your-production-questions.yml

To test a production rebuild without touching the default generated data/ directory:

rag --data-dir data-production-test rebuild corpus --yes
rag --data-dir data-production-test status
rag --data-dir data-production-test eval --file eval/your-production-questions.yml
rag --data-dir data-production-test context "your question"

When --data-dir is supplied, generated artifacts and the default SQLite database are both written under that directory.

The same isolation can be configured with environment variables:

RAG_DATA_DIR=data-production-test rag rebuild corpus --yes

RAG_DB_PATH can override only the SQLite path when needed.

Clean Reset

rag reset --yes

This deletes generated artifacts and the SQLite database. It does not delete files in articles/ or corpus/.

If corpus/ only contains README.md, rag rebuild corpus --yes warns and exits with an error because no real production files have been added yet. Use --allow-empty only for intentional empty-index testing.

Useful SQLite Queries

Inspect documents:

sqlite3 data/rag.sqlite "select source_type, title, source_uri from documents order by title limit 20;"

Inspect chunk counts:

sqlite3 data/rag.sqlite "select title, count(*) from chunks group by document_id order by count(*) desc;"

Inspect keyword payloads:

sqlite3 data/rag.sqlite "select title, keywords_json from chunks limit 10;"