Skip to content

chore: Replace metadata dict data_loss_ratio with measure-based equivalents - #140

Open
cbueth wants to merge 2 commits into
mainfrom
chore/metadata-data-loss-migration
Open

chore: Replace metadata dict data_loss_ratio with measure-based equivalents#140
cbueth wants to merge 2 commits into
mainfrom
chore/metadata-data-loss-migration

Conversation

@cbueth

@cbueth cbueth commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

This PR replaces pre_processed_metadata['data_loss_ratio'] and ['data_loss_ratio_blinks'] (EyeLink-only, computed during ASC parsing) with format-independent equivalents:

  • Total data loss: pymovements.measure.data_loss('pixel', sampling_rate=sr, unit='ratio'). Per-trial via group_by, simple mean across trials
  • Blink data loss: gaze.measure_events_ratio('blink_eyelink', sampling_rate=sr). Per-trial via group_by, simple mean across trials
    The old metadata values remain in the overview alongside the new ones for comparison during transition.

Instead of just changing this, I first had both be calculated in parallel (1351619), to see if the results are the same, or differ:

Comparison data

Here compare the results of the metadata numbers of before and the measure numbers of after.

Session meta_total meas_total (\Delta) meta_blink meas_blink (\Delta)
[redacted] 0.04633 0.06233 0.01600 0.04632 0.05794 0.01161
[redacted] 0.03367 0.03807 0.00440 0.03367 0.03553 0.00186
[redacted] 0.02036 0.02329 0.00293 0.02035 0.02182 0.00146
[redacted] 0.05463 0.06996 0.01533 0.05463 0.06547 0.01084
[redacted] 0.01961 0.02206 0.00245 0.01961 0.02068 0.00107

The ~1-2% difference comes from two sources:

  1. Simple mean vs. weighted aggregate: the new code uses .mean() (trials equally weighted), the old metadata uses Σ(expected − valid) / Σ(expected) (weighted by trial duration per recording block).
  2. Trial columns ≠ recording blocks: the measure groups by ['trial', 'stimulus', 'page'] while metadata uses EyeLink START/STOP recording messages. Boundaries differ.
    See my comment Remove data_loss_ratio and data_loss_ratio_blinks from parsed metadata pymovements/pymovements#1584 (comment) for more details on this.

Open question

Should we use a weighted aggregate instead of simple mean? The current .mean() treats each trial equally. To replicate the old metadata-style weighted aggregate, we would need to weight by trial duration. Replacing

.select(pl.col('data_loss_ratio').mean())

with

# Sum per-trial loss counts, divide by total expected
.agg(dl_count).sum() / .agg(dl_expected).sum()

Please add opinions on this when reviewing! Is the information saved where expcted?

References

- use pymovements 0.27.1 for this to work, before these metadata fields are removed in later versions.

Signed-off-by: Carlson Büth <commit@cbueth.de>
@cbueth
cbueth requested a review from theDebbister July 15, 2026 12:47
@cbueth cbueth self-assigned this Jul 15, 2026
@cbueth
cbueth requested a review from dkrako July 15, 2026 12:48
- Directly compute trial-based data loss using `pymovements` without temporary metadata
- clear parsed events for stability

Signed-off-by: Carlson Büth <commit@cbueth.de>

@theDebbister theDebbister left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, so in consultation with the other MultiplEYE team mebers we decided that it is best to report the session level measures (in general) not as simple averages over the trials, but taking the length into account (i.e. just calculate the data loss at the session level directly).

As the other values can simply be calcualted based on values we already have at the trial level, we can later still add these, but for now, we just use this one value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants