Fix microstate training and segmentation issues - #1199
Open
hatute wants to merge 5 commits into
Open
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev #1199 +/- ##
==========================================
+ Coverage 57.78% 59.77% +1.99%
==========================================
Files 310 310
Lines 15680 15725 +45
==========================================
+ Hits 9060 9399 +339
+ Misses 6620 6326 -294 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
First, I would like to express my sincere appreciation for the work that has gone into NeuroKit2. It is an extremely valuable open-source project, and its broad collection of neurophysiological processing tools provides substantial value to the scientific community.
While reviewing the microstate implementation, I identified several issues that could materially alter clustering, segmentation, and derived microstate statistics. Since these outputs may be used directly in scientific analyses and publications, the affected behavior had a high likelihood of introducing substantial
bias into academic results without necessarily raising an explicit error.
The main issues addressed by this PR are:
aahcmethod did not reach the AAHC backend.train="all"generated indices from the number of channels instead of the number of timepoints.microstates_peaks()were ignored.GEV_per_microstatewas not reordered together with the maps.criterion="cv"path failed with aTypeError.Proposed Changes
Clustering backend routing
The ICA aliases now call
_cluster_ica()rather than_cluster_pca().The documented
aahcmethod is now recognized as an alias for_cluster_aahc(), alongside the existingaahc_fredericandaahc_eegmicrostatesaliases.FastICA now uses
whiten="unit-variance"for compatibility with current scikit-learn versions.Regression tests inspect the selected clustering function to ensure that ICA and AAHC reach their intended implementations.
Training timepoint selection
For EEG data shaped
(channels, timepoints),train="all"now returns:The previous implementation used len(eeg), which represents the number of channels and therefore selected only the first n_channels timepoints.
Numeric and "all" training modes no longer require sampling_rate, because no time-based GFP peak distance is calculated in these modes.
Numeric training selections are validated to ensure that the requested number of samples is valid and does not exceed the available timepoints.
GFP handling
A precomputed GFP vector supplied to microstates_peaks() is now used directly instead of being silently discarded and recomputed.
Precomputed GFP input is validated to ensure that it is one-dimensional and contains exactly one value per EEG timepoint.
The GFP calculated by microstates_clean() is also reused for peak detection, ensuring consistency between the returned GFP and the GFP used to select training samples.
Input preprocessing
DataFrame input is converted into a consistent NumPy representation.
MNE Epochs data, originally shaped (epochs, channels, timepoints), is converted into (channels, concatenated_timepoints). Epochs are concatenated in trial order while preserving the channel dimension and MNE channel information.
The same Epochs conversion is applied in microstates_findnumber().
EEG standardization is now performed independently for each channel across time, matching the documented behavior.
Clustering-specific keyword arguments such as n_init are no longer passed to standardize(), eeg_gfp(), or microstates_peaks().
Classification and output consistency
microstates_classify() now uses the inverse permutation when relabeling the segmentation sequence.
This ensures that, after reordering, every value in Sequence still refers to the correct row in Microstates.
An optional return_order argument exposes the applied map permutation for internal consumers while preserving the existing default two-value return signature.
GEV_per_microstate is reordered using the same permutation, keeping Sequence, Microstates, and per-state GEV values aligned.
Segmentation and backfitting
The criterion="cv" path now stores current_info correctly. The previous info -= current_info statement attempted to subtract a dictionary from None and failed during the first selected run.
The criterion value and n_runs are now explicitly validated.
Whole-recording backfitting now centers each topographic map and normalizes it to unit norm before calculating activation.
This makes assignments invariant to arbitrary scaling of individual maps. This is particularly important for KMeans, PCA, and ICA maps, whose raw magnitudes are not inherently comparable.
Polarity-independent assignment is preserved by selecting the largest absolute activation.
Lifetime statistics
The lifetime calculation now starts iteration from the second sample because the first state is already represented by the initial tau = 1.
For example, [0, 0, 1] now correctly reports a two-sample lifetime for state 0 rather than three samples.
Tests
Regression coverage was added for:
The targeted test results are:
tests/tests_microstates.py: 9 passed
tests/tests_stats.py: 7 passed
Total: 16 passed
All modified files pass Ruff lint and Ruff format checks.
Checklist
Here are some things to check before creating the PR. If you encounter any issues, do let us know :)