Skip to content

feat(study): MEG data-sample validation study (M100, memory-bounded, resumable)#12

Merged
juniperbevensee merged 1 commit into
mainfrom
dev/juniperbevensee/meg-validation-study
Jun 22, 2026
Merged

feat(study): MEG data-sample validation study (M100, memory-bounded, resumable)#12
juniperbevensee merged 1 commit into
mainfrom
dev/juniperbevensee/meg-validation-study

Conversation

@juniperbevensee

Copy link
Copy Markdown
Collaborator

What it does

Adds a meg_validation study type — the first heavy / real-data study built on the stateful study pipeline. It validates the auditory M100 evoked-response claim (a magnetic field peak ~80–120 ms after an auditory stimulus) against a bounded sample of an open MEG dataset (bst_auditory), as five resumable, checkpointed steps:

fetch_sample → preprocess → epoch → evoked → validate_finding

validate_finding emits a finding: supported if the measured peak latency is within the expected window (default 80–120 ms), refuted if clearly outside, inconclusive if the sample is degenerate (no measurable peak). Evidence = measured latency + amplitude.

Lazy-mne + injected-I/O testing approach

  • Lazy heavy deps. mne/numpy are imported only inside the real backend's methods (_MneIO), never at module top. matilde_plugin.engine.meg_study — and the whole plugin — imports cleanly with neither installed (verified in a venv with no mne/numpy). grep confirms no top-level heavy imports anywhere in engine/.
  • Injected I/O. Every step calls through an injected MegIO handle. Production passes none and gets the real backend; tests pass fakes — no mne, no numpy, no network, no data files, fully deterministic and stdlib-only.
  • The tools layer uses a small _TEST_IO hook so the offline dispatch test stays stdlib-only; production leaves it unset (real lazy-mne).

Memory-bounded by design

The real backend never full-preloads a recording: preload=False, one run, a gradiometer channel subset, crop to a stimulus-locked window, downsample — then persists small intermediates. Each step reloads the prior step's on-disk artifact rather than holding live objects, so a memory kill or container rebuild resumes from the last completed step instead of re-fetching the whole chain.

Resumability evidence

A unit test forces a failure at the evoked step: the study goes blocked with fetch_sample/preprocess/epoch done and no finding emitted. On resume with a working backend, call counters prove the earlier steps are not re-run (fetch/preprocess/epoch = 0 calls), only the failed step and the remaining one execute, and the study completes with the expected finding.

Tests

  • New: tests/test_meg_study.py (step transitions, verdict boundaries 100ms→supported / 300ms→refuted / empty→inconclusive, custom window, resumability) and tests/test_meg_study_tools.py (create+run dispatch through the tools).
  • Opt-in live test (MATILDE_LIVE=1 + mne importable, via pytest.importorskip): runs the real bounded pipeline on bst_auditory and asserts a peak in a plausible window. Skipped without the flag/mne — that's expected.
  • Full suite: 131 passed, 7 skipped (the live test is one of the skips).

Do not merge — leaving open for independent audit.

…resumable)

Adds a meg_validation study type — the first heavy / real-data study built on
the stateful study pipeline. Validates the auditory M100 evoked-response claim
(magnetic peak ~80-120 ms) against a bounded sample of an open MEG dataset, as
five resumable checkpointed steps: fetch_sample -> preprocess -> epoch ->
evoked -> validate_finding.

- engine/meg_study.py: the five steps + a memory-bounded real mne backend
  (_MneIO) and an injected-I/O contract (MegIO). Heavy deps (mne/numpy) are
  imported ONLY inside backend methods — the module and the whole plugin import
  fine with neither installed. The real backend never full-preloads: preload=
  False, single run, gradiometer subset, crop, downsample; each step reloads the
  prior step's on-disk artifact, so an OOM/restart resumes from the last
  completed step instead of re-fetching. validate_finding emits supported (peak
  in window) / refuted (clearly outside) / inconclusive (degenerate sample),
  with measured latency + amplitude as evidence.
- tools.py: wire meg_validation into matilde_study_create (default plan + params:
  dataset_id, expected_window_ms, bounds) and matilde_study_run dispatch,
  mirroring the bibliography dispatch; handlers stay thin.
- tests: stdlib + fakes (no mne/numpy/network/data) — step state transitions,
  verdict boundaries (100ms supported / 300ms refuted / empty inconclusive),
  custom window, and resumability proven with call counters (fail at evoked,
  resume, earlier steps not re-run). Plus an opt-in live test (MATILDE_LIVE=1 +
  mne) on bst_auditory, skipped otherwise.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@juniperbevensee
juniperbevensee merged commit 046088a into main Jun 22, 2026
1 check passed
@juniperbevensee
juniperbevensee deleted the dev/juniperbevensee/meg-validation-study branch June 22, 2026 04:43
juniperbevensee added a commit to NimbleCoAI/hermes-swarm-map that referenced this pull request Jun 22, 2026
…ation study) (#109)

v0.4.0 adds the memory-bounded resumable MEG data-sample validation study
(NimbleCoOrg/Matilde#12). Reapply to deployed Matilde to expose it.

Co-authored-by: Juniper Bevensee <juniperbevensee@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant