Skip to content

Fix microstate training and segmentation issues - #1199

Open
hatute wants to merge 5 commits into
neuropsychology:devfrom
hatute:master
Open

Fix microstate training and segmentation issues#1199
hatute wants to merge 5 commits into
neuropsychology:devfrom
hatute:master

Conversation

@hatute

@hatute hatute commented Aug 5, 2026

Copy link
Copy Markdown

Description

First, I would like to express my sincere appreciation for the work that has gone into NeuroKit2. It is an extremely valuable open-source project, and its broad collection of neurophysiological processing tools provides substantial value to the scientific community.

While reviewing the microstate implementation, I identified several issues that could materially alter clustering, segmentation, and derived microstate statistics. Since these outputs may be used directly in scientific analyses and publications, the affected behavior had a high likelihood of introducing substantial
bias into academic results without necessarily raising an explicit error.

The main issues addressed by this PR are:

  • ICA was silently routed to the PCA implementation.
  • The documented aahc method did not reach the AAHC backend.
  • train="all" generated indices from the number of channels instead of the number of timepoints.
  • Precomputed GFP values supplied to microstates_peaks() were ignored.
  • Non-GFP training modes unnecessarily required a sampling rate.
  • Clustering keyword arguments leaked into EEG preprocessing functions.
  • EEG standardization was performed across channels rather than across time.
  • MNE Epochs inputs failed despite being documented as supported.
  • Microstate labels were incorrectly remapped after map reordering.
  • GEV_per_microstate was not reordered together with the maps.
  • The criterion="cv" path failed with a TypeError.
  • Backfitting was affected by arbitrary differences in microstate-map magnitude.
  • The first microstate lifetime was overcounted by one sample.
  • The ICA implementation used a FastICA argument that is invalid in current scikit-learn versions.

Proposed Changes

Clustering backend routing

The ICA aliases now call _cluster_ica() rather than _cluster_pca().

The documented aahc method is now recognized as an alias for _cluster_aahc(), alongside the existing aahc_frederic and aahc_eegmicrostates aliases.

FastICA now uses whiten="unit-variance" for compatibility with current scikit-learn versions.

Regression tests inspect the selected clustering function to ensure that ICA and AAHC reach their intended implementations.

Training timepoint selection

For EEG data shaped (channels, timepoints), train="all" now returns:

np.arange(eeg.shape[1])

The previous implementation used len(eeg), which represents the number of channels and therefore selected only the first n_channels timepoints.

Numeric and "all" training modes no longer require sampling_rate, because no time-based GFP peak distance is calculated in these modes.

Numeric training selections are validated to ensure that the requested number of samples is valid and does not exceed the available timepoints.

GFP handling

A precomputed GFP vector supplied to microstates_peaks() is now used directly instead of being silently discarded and recomputed.

Precomputed GFP input is validated to ensure that it is one-dimensional and contains exactly one value per EEG timepoint.

The GFP calculated by microstates_clean() is also reused for peak detection, ensuring consistency between the returned GFP and the GFP used to select training samples.

Input preprocessing

DataFrame input is converted into a consistent NumPy representation.

MNE Epochs data, originally shaped (epochs, channels, timepoints), is converted into (channels, concatenated_timepoints). Epochs are concatenated in trial order while preserving the channel dimension and MNE channel information.

The same Epochs conversion is applied in microstates_findnumber().

EEG standardization is now performed independently for each channel across time, matching the documented behavior.

Clustering-specific keyword arguments such as n_init are no longer passed to standardize(), eeg_gfp(), or microstates_peaks().

Classification and output consistency

microstates_classify() now uses the inverse permutation when relabeling the segmentation sequence.

This ensures that, after reordering, every value in Sequence still refers to the correct row in Microstates.

An optional return_order argument exposes the applied map permutation for internal consumers while preserving the existing default two-value return signature.

GEV_per_microstate is reordered using the same permutation, keeping Sequence, Microstates, and per-state GEV values aligned.

Segmentation and backfitting

The criterion="cv" path now stores current_info correctly. The previous info -= current_info statement attempted to subtract a dictionary from None and failed during the first selected run.

The criterion value and n_runs are now explicitly validated.

Whole-recording backfitting now centers each topographic map and normalizes it to unit norm before calculating activation.

This makes assignments invariant to arbitrary scaling of individual maps. This is particularly important for KMeans, PCA, and ICA maps, whose raw magnitudes are not inherently comparable.

Polarity-independent assignment is preserved by selecting the largest absolute activation.

Lifetime statistics

The lifetime calculation now starts iteration from the second sample because the first state is already represented by the initial tau = 1.

For example, [0, 0, 1] now correctly reports a two-sample lifetime for state 0 rather than three samples.

Tests

Regression coverage was added for:

  • ICA and AAHC backend routing.
  • All-timepoint and numeric training selection.
  • Precomputed GFP handling.
  • DataFrame input.
  • MNE Epochs preprocessing and end-to-end segmentation.
  • Per-channel temporal standardization.
  • Isolation of clustering keyword arguments.
  • Non-self-inverse classification permutations.
  • Sequence-to-map correspondence.
  • Per-state GEV ordering.
  • CV-based run selection.
  • Map-scale-invariant backfitting.
  • Documented PCA, ICA, and AAHC segmentation methods.
  • First-run lifetime counting.

The targeted test results are:

tests/tests_microstates.py: 9 passed
tests/tests_stats.py: 7 passed
Total: 16 passed

All modified files pass Ruff lint and Ruff format checks.

Checklist

Here are some things to check before creating the PR. If you encounter any issues, do let us know :)

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.81818% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.77%. Comparing base (ff419d9) to head (600fd6d).
⚠️ Report is 20 commits behind head on dev.

Files with missing lines Patch % Lines
neurokit2/microstates/microstates_peaks.py 75.75% 8 Missing ⚠️
neurokit2/microstates/microstates_segment.py 87.50% 2 Missing ⚠️
neurokit2/microstates/microstates_clean.py 87.50% 1 Missing ⚠️
neurokit2/microstates/microstates_findnumber.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1199      +/-   ##
==========================================
+ Coverage   57.78%   59.77%   +1.99%     
==========================================
  Files         310      310              
  Lines       15680    15725      +45     
==========================================
+ Hits         9060     9399     +339     
+ Misses       6620     6326     -294     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants