Skip to content

fix: IntervalEncoder average-method window predicate (OR should be AND) - #819

Merged
FBruzzesi merged 1 commit into
koaning:mainfrom
AlejandroCoronadoN:fix-intervalencoder-average-window
Aug 29, 2026
Merged

fix: IntervalEncoder average-method window predicate (OR should be AND)#819
FBruzzesi merged 1 commit into
koaning:mainfrom
AlejandroCoronadoN:fix-intervalencoder-average-window

Conversation

@AlejandroCoronadoN

Copy link
Copy Markdown
Contributor

IntervalEncoder(method="average") never applies the lower bound of its averaging window. In _mk_average the predicate is:

predicate = (xs < (interval + span)) | (xs < (interval - span))

Since interval - span < interval + span always, the second condition is a subset of the first, so the OR collapses to just xs < (interval + span): there is no lower bound. Every point below the interval (however far) leaks into the weighted average, which biases the smoothed height low.

The docstring describes span as "the span around the interval" (a symmetric, two-sided window), and the sibling method="normal" uses all points weighted by a Gaussian, so average is meant to be the hard-windowed version of the same idea. The fix uses AND:

predicate = (xs < (interval + span)) & (xs > (interval - span))

Repro with a linear target y = 2x (the smoothed height at each quantile should track 2 * quantile):

import numpy as np
from sklego.preprocessing import IntervalEncoder

np.random.seed(0)
x = np.random.uniform(0, 10, 2000)
ie = IntervalEncoder(n_chunks=5, span=0.1, method="average").fit(x.reshape(-1, 1), 2 * x)
print(ie.heights_)   # before: biased low (~17.5 at the top quantile near 10); after: ~20

Added a regression test. The existing method="average" test only used a constant target, which cannot detect the bias (averaging any subset of a constant array still gives that constant).

@FBruzzesi FBruzzesi added the fix label Aug 29, 2026
@FBruzzesi FBruzzesi changed the title IntervalEncoder: fix the average-method window predicate (OR should be AND) fix: IntervalEncoder average-method window predicate (OR should be AND) Aug 29, 2026

@FBruzzesi FBruzzesi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AlejandroCoronadoN that indeed seems like a copy-pasting gone wrong + wrong logical operator! Thanks for spotting and fixing it

@FBruzzesi
FBruzzesi merged commit 83f9985 into koaning:main Aug 29, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants