BUG: ewm(adjust=False) NaN gaps disagree for times vs no-times - #66754
Open
hasancankeles wants to merge 3 commits into
Open
BUG: ewm(adjust=False) NaN gaps disagree for times vs no-times#66754hasancankeles wants to merge 3 commits into
hasancankeles wants to merge 3 commits into
Conversation
Contributor
|
Thanks for the pull request, @hasancankeles! It's linked to #66523, but that issue isn't assigned to you yet. To make sure two people don't unknowingly work on the same thing, we ask contributors to claim an issue first. Just comment |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Needs Discussion//take; xref until the label is removed)new_wt = 1 - old_wtused accumulated decay)doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.AGENTS.md.The
adjust=Falseloop special-casedcom == 1(GH-59142) asnew_wt = 1 - old_wt. That is the right convex weight whenold_wtis just this step's decay, which is true for irregulartimeswith no missing values. After anignore_na=FalseNaN row,old_wthas already been multiplied once, so the same assignment gives the new observation the entire forgotten mass. That is:timespath forcom=1(ewm NaN handling is wrong : simple example #31178:4.0instead of11/3);timespath, which always forces_com = 1.0.Gating the special case on
use_deltas(GH-66521) would fix only the first and split the two spellings — the inconsistency filed as #66523.The intended split, from the documented
ignore_na=Falserecursion, GH-59142's irregular-interval tests, andtest_ewma_with_times_equal_spacing, is:new_wt = 1 - (1-α)^δ(δ = 1withouttimes);old_wtand renormalize.That is option 1 from the issue, without switching the irregular no-NaN
timespath to the normalized form.test_ewma_with_times_variable_spacingis unchanged (0.2376251864…). Equally spacedtimesagain match no-timesfor everyadjust×ignore_nacombination, including on NaN data.Verified:
pandas/tests/window/test_ewm.py,test_numba.py,test_online.py, and the other ewm-related window tests.