Skip to content

Compare the signs of the two rho z scores - #286

Open
arpitjain099 wants to merge 1 commit into
CDCgov:mainfrom
arpitjain099:fix/rho-outlier-sign-comparison
Open

Compare the signs of the two rho z scores#286
arpitjain099 wants to merge 1 commit into
CDCgov:mainfrom
arpitjain099:fix/rho-outlier-sign-comparison

Conversation

@arpitjain099

Copy link
Copy Markdown

Closes #279.

The rho rule in flag_ww_outliers() reads

sign(.data$z_score_rho != sign(.data$z_score_rho_t_plus_1))

The close parenthesis is one term too far to the right. Instead of comparing the two signs, it compares a z score against the sign of the next one and then takes the sign of that logical. sign(TRUE) is 1, so the term is true whenever z_score_rho is anything other than exactly 1 or -1, which is almost always.

That makes the sign test a no-op in practice, and the rule reduces to "two consecutive steep moves", regardless of direction. A sustained rise or fall then matches, and gets flagged as an outlier, which indicate_ww_exclusions() can go on to exclude from the model. Only the spike case, a steep move followed by a steep move the other way, was meant to match.

Running the same data through both versions, with log_conc_threshold set high so only the rho rule can fire:

sustained rise (same sign)       rho_threshold=0.8  current=1  fixed=0
sustained rise (same sign)       rho_threshold=1.0  current=1  fixed=0
sustained rise (same sign)       rho_threshold=1.2  current=1  fixed=0
spike up then down (opposite)    rho_threshold=0.8  current=1  fixed=1
spike up then down (opposite)    rho_threshold=1.0  current=1  fixed=1
spike up then down (opposite)    rho_threshold=1.2  current=1  fixed=1
sustained fall (same sign)       rho_threshold=0.8  current=1  fixed=0
sustained fall (same sign)       rho_threshold=1.0  current=1  fixed=0
sustained fall (same sign)       rho_threshold=1.2  current=1  fixed=0

So the change removes the false positives and keeps the true one.

Two tests added, at three thresholds each. One asserts a monotone rise and a monotone fall produce no outlier, and the other asserts the spike still produces exactly one, so the fix cannot pass by simply flagging less. The first fails on main with six failures; the second passes on both, which is the point of including it.

All the pre-existing tests in test_flag_as_ww_outliers.R pass against both versions, so nothing that was flagged before and should still be flagged has changed. I ran them by sourcing R/preprocessing.R directly, since the suite's setup.R loads the compiled package and I do not have the Stan toolchain here. CI covers the rest.

The rho outlier rule is written as

  sign(.data$z_score_rho != sign(.data$z_score_rho_t_plus_1))

The close parenthesis is one term too far right, so instead of comparing
the two signs it compares a z score against the sign of the next one and
takes the sign of that logical. sign(TRUE) is 1, so the term is true
whenever z_score_rho is not exactly 1 or -1, which makes the sign test
pass for almost any pair.

The effect is that two consecutive steep moves in the SAME direction
satisfy the rule, so a sustained rise or fall gets flagged as an outlier
and can then be excluded from the model. Only the spike case, a steep
move followed by a steep move the other way, was meant to match.

Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix incorrect close parenthesis in preprocessing

1 participant