feat(pipeline): stateful, resumable study store + runner (+ bibliography study)

[juniperbevensee]

Motivation

Long analyses (e.g. validating a paper's findings against an OpenNeuro dataset) currently run inline in a single agent turn. Two failure modes, observed live:

• OOM-killed — a script preloaded a 2.5 GB dataset into the agent container and was OOM-killed.
• State lost on container rebuild — a mid-run analysis was wiped when the agent image was rebuilt, with no resume and no memory it ever happened.

Both have the same remedy: durable, checkpointed analysis state so an analysis resumes from the last completed step after an OOM or a container rebuild, instead of vanishing. Spec: docs/stateful-study-pipeline.md.

What's included

• matilde_plugin/engine/store.py — StudyStore (stdlib sqlite3, WAL, autocommit/atomic). Tables: studies / steps / artifacts / findings. API: create_study (idempotent on slug), get_study, list_studies, add_steps, set_step_status, record_step_result, add_artifact, add_finding, get_findings, study_summary. Artifacts reference on-disk files, never blobs. Injected db path (MATILDE_STUDY_DB → $HERMES_HOME/engagements → repo engagements/), so state lives on the mounted data volume.
• matilde_plugin/engine/pipeline.py — resumable runner. Step / StepContext / StepResult; run(store, study_id, steps, resume=True) skips done steps, marks running→done, persists result+artifacts+findings after each step, and on exception marks the step failed, the study blocked, and stops (resumable). resume() continues from the first non-done step. Pure/injected — testable with in-memory fake steps.
• matilde_plugin/engine/bibliography_study.py — first concrete study (parse_refs → verify_each → summarize) reusing the existing citations/parsing engine, with the per-reference verifier injected. verify_each is mid-step resumable (skips refs that already have a persisted finding).
• Agent tools in matilde_plugin/tools.py + wiring in __init__.py: matilde_study_create, matilde_study_run, matilde_study_status, matilde_study_list — matching the file's existing tool-dict + thin-handler conventions. Handlers stay thin; logic lives in store.py/pipeline.py.

Resumability test evidence

• tests/test_store.py::test_state_survives_fresh_store_instance — writes state, drops the connection, opens a new StudyStore on the same file, and reads everything back (the container-rebuild guarantee).
• tests/test_pipeline.py::test_resume_continues_from_first_non_done_without_rerunning — first pass fails at step b; a working b is swapped in; resume re-runs only b+c (a side-effect counter proves a is not re-run).
• tests/test_bibliography_study.py::test_fail_then_resume_does_not_reverify_done_refs — verifier fails on ref OpenNeuro/BIDS dataset client + agent tools (discovery, metadata, files) #3 → study blocked, refs 1–2 recorded as findings → swap in a working verifier → resume → study done, and only ref OpenNeuro/BIDS dataset client + agent tools (discovery, metadata, files) #3 is re-verified (refs 1–2 not re-checked).

Verification

• .venv/bin/python -m pytest -q → 123 passed, 6 skipped (baseline was 93+6; +30 new tests).
• Sanitization gate (scripts/check_sanitization.py) on changed content files: clean (deterministic layer; semantic layer skips locally without ANTHROPIC_API_KEY, as designed — CI runs it with the key).
• No ruff config in the repo.
• Engine stays stdlib-only — no mne/numpy/heavy deps added (git diff imports: json, os, sqlite3, time, dataclasses, typing + intra-package relative imports).

Follow-up (out of scope)

The heavy MEG data-analysis study (fetch_dataset → preprocess → epoch → evoked → validate_finding, memory-bounded, intermediates checkpointed as artifacts) is a documented follow-up — it needs the agent container memory budget decided and where mne is installed. This PR is the framework + a lightweight first study that proves it.

Please do not merge — left open for independent audit.

🤖 Generated with Claude Code