implement channel specific epoch rejection

[CarinaFo]

Implements channel-specific epoch rejection as suggested in PR #11705.

Use case: In ECoG recordings, an epoch may be corrupted for a single channel only, while remaining valid for other channels. The current MNE implementation rejects entire epochs across all channels, which is appropriate for EEG/MEG but overly restrictive for ECoG data.

This change enables channel-wise epoch rejection.

Important consideration: Channels may end up with substantially different numbers of retained epochs. This can introduce issues during source localization, where nave (number of averaged epochs, integer) is used to scale the noise covariance. Downstream handling of nave must therefore account for channel-specific epoch counts.

Current state of this PR:

• implemented drop_bad_epochs_per_channel method that takes a boolean mask with shape n_epochs, n_channels and sets epochs to NaN for a given channel
• Evoked object contains a nave attribute (number of averaged epochs)
• Nave is an integer; channel-specific epoch rejection allows for nave to be different for each channel
• This will impact source localisation as noise is no longer equal among channels

ToDo:

• Simulate the effect of nave_per_channel on the source localisation error using Elekta phantom recordings

Issues:

• mne.compute_covariance(Epochs) does not allow for NaNs in the input data

Alternatives:

• restrict drop_bad_epochs_per_channel to data type "ECOG" and "ignore" source localisation

Key hypothesis to test:

Channel-specific epoch rejection preserves more usable data without increasing localisation error.

ToDo:
-pairwise covariance instead of sample covariance if NaNs detected
https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.cov.html

Update:

• NaN-aware covariance abandoned after simulations showed that the source localisation error is huge
• channel-wise epoch rejection only useful for channel-wise analysis:

2 Options:

• restrict to ECoG or LFP data
• restrict averaging to non-nan epochs
• add a warning to the function

[dominikwelke]

apart from the main functionality and some test (tbd), we would probably also want some kind of record that this was done.
drop_log comes to mind, but i think as is it wouldnt be suitable because no epoch is dropped.

any good idea?

[dominikwelke]

btw, i dont know about the state of the PR but for me this code doesnt work so far..
i think it's the overwriting with nan values using the mask.. but behavior was not really consistent, i have to track this down :)

[CarinaFo]

Update: the code works after implementing a preload_check. TBD: do we want channel specific epoch rejection, meaning we kick out epoch outliers based on the channel specific standard deviation OR do we calculate the standard deviation over all channels to define outlier epochs?

[dominikwelke]

related to one of my previous comments:
currently data has to be preloaded, so that chosen epochs can be overwritten with nan. as mentioned, currently this change is not logged anywhere which we should probably do :).
if we were to extend drop_log in a way that it can handle individual channel/epoch combos, this would enable us to process without preloading, as i think is the case for all other rejection methods.

but this would be a deeper change, and basically a design decision.
any thoughts @drammock @larsoner ?

[dominikwelke]

TBD: do we want channel specific epoch rejection, meaning we kick out epoch outliers based on the channel specific standard deviation OR do we calculate the standard deviation over all channels to define outlier epochs?

+1 for this question!
you are right @carina - the requested functionality in #11705 was std within channels, and not across all data..

[drammock]

Sorry it took me so long to look at this PR

TBD: do we want channel specific epoch rejection, meaning we kick out epoch outliers based on the channel specific standard deviation OR do we calculate the standard deviation over all channels to define outlier epochs?

This implementation is actually a bit more specific than I was expecting ("specific" in the sense that it bakes in the idea of using standard deviation to decide what to do). I was expecting @larsoner's idea (2) from here, as I tried to indicate in my subsequent comment:

marking as NaN and using nanmean / interpolation seems like the right approach to me.

In other words, I was thinking the implementation should be something like:

• make it easy to replace channel data with nan on a per-epoch basis. Currently if a channel is bad it is bad for all epochs in the Epochs object, so epochs.interpolate_bads can't do nan-replacement on a per-trial basis. The solution might be to modify interpolate_bads or it might be a new function, not sure.
• modify epochs.average to use np.nanmean
• deal with the "effective n_averaged" issue somehow

That way, the user's job is to somehow create the inputs to that new function (or the modified interpolate_bads function) --- I'm imagining a dict of {epoch_idx: list_of_bad_chs} as one possible way --- and they can decide how to populate that (std deviation or somethign else, within channel vs across-channel, etc).

@larsoner was this more or less what you were expecting, or am I way off base here?

[larsoner]

Yes that sounds reasonable to me!