Speed up small reads on the Raw.get_data hot path - #14212
Conversation
|
|
||
| return sel, start, stop | ||
|
|
||
| def _get_windows(self, starts, width, *, out=None, sel=None): |
There was a problem hiding this comment.
This doesn't appear to be used anywhere or tested, so probably shouldn't be in this PR
larsoner
left a comment
larsoner
left a comment
There was a problem hiding this comment.
Other than two comments, seems reasonable to me. Can you give speedup numbers for this PR?
|
Oh never mind the speedups are above 🤦 |
|
Addressed both comments:
This PR alone, random 2 s windows, public API, best of 4 interleaved runs vs |
larsoner
left a comment
larsoner
left a comment
There was a problem hiding this comment.
Please also take out of draft mode if ready to go from your end!
| Sped up raw data access for workloads making many small reads: | ||
| ``Raw.get_data`` no longer materializes the full time axis when ``tmin`` and | ||
| ``tmax`` are unset, resolves ``picks=None`` without channel-name machinery, | ||
| and ``_mult_cal_one`` applies gather/type-cast/calibration in a single pass. |
There was a problem hiding this comment.
Needs name at the end, and should be simpler. This is also not a bug but an enhancement so should be 14212.newfeature.rst. Something like
Sped up reading by optimizing small read paths, by `<name>`_
Please also modify AGENTS.md in a way that would have made this stuff clear
0763b69 to
c19d5dc
Compare
get_data resolves picks=None to arange directly instead of going through string-based channel-name machinery on every call; _picks_to_idx gets an early return for integer arrays already unique and in range (duplicate picks keep taking the validating path); _mult_cal_one fuses gather, type-cast, and calibration into a single elementwise pass.
c19d5dc to
8321243
Compare
|
All set:
|
|
I don't see the 
|
|
sorry, it was my rule configuration of not pushing agents' things, i am pushing at the moment. |
Adds the performance-changelog entry and captures in AGENTS.md the two conventions this effort leaned on: how to write changelog fragments (type by intent — performance work is newfeature; one-sentence entries ending with a contributor name link anchored in names.inc) and the interleaved-A/B benchmarking discipline used to validate these speedups.
46acd9a to
52a9a25
Compare
|
AGENTS.md is updated on the branch with the fragment rules and the A/B benchmarking discipline. |
larsoner
left a comment
larsoner
left a comment
There was a problem hiding this comment.
Marking for merge-when-greene, thanks in advance @bruAristimunha !
github-project-automation
Bot
moved this to Ideas
in 2026 MNE-Python Maintainers Sprint
github-project-automation
Bot
moved this from In Progress
to Done
in 2026 MNE-Python Maintainers Sprint
2 participants
Reference issue (if any)
None. Performance work; overview of the planned series in #14209.
What does this implement/fix?
Speeds up small reads on the
Raw.get_datahot path:get_dataresolvespicks=Nonetoarange(nchan)directly instead of goingthrough string-based channel-name machinery on every call
_picks_to_idxreturns early for integer arrays that are unique and in range(duplicates keep the validating path, per review)
_mult_cal_onefuses gather, type-cast and calibration into one passOutput is bit-identical; all suites pass.
Random 2 s windows via public API, best of 4 interleaved runs vs
main(128 ch × 1800 s @ 512 Hz fixtures): EDF 1543 → 774 µs, BDF 2255 → 1498 µs,
BrainVision 973 → 245 µs, FIF 1087 → 305 µs per read.
Additional information
AI disclosure: I directed the work and reviewed/tested everything; Claude Code
(Opus 4.1) wrote most of the code edits and the benchmark harness under my
direction (scripts in
benchmarks/io_dl/on the branch).This is part 1 of a stacked series (#14213–#14216 depend on it).