Updated Multisession3ItMemoryThr flaky test checks - #1758
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1758 +/- ##
==========================================
- Coverage 61.37% 61.36% -0.01%
==========================================
Files 291 291
Lines 51231 51231
==========================================
- Hits 31441 31436 -5
- Misses 19790 19795 +5 🚀 New features to boost your workflow:
|
fallenmi
left a comment
There was a problem hiding this comment.
The new adjacent-variant margin addresses the observed cross-run reversal without removing the regression signal. At exact head cf2c3a1, the old assertion rejects the documented local=0.0956 / none=0.0946 pair, while the new strict +0.015 bound accepts that 0.001 reversal and still rejects a reversal at or beyond the margin. The two independent both-retrieval < ... assertions are retained, so a collapse of the configured retrieval benefit remains a failure.
I compiled and ran boundary cases against the exact base/head assertion logic. I also inspected the exact-head macOS Sequoia run: 18/18 CTest targets passed, and Multisession3ItMemoryThr ran (not skipped) and passed in 175.981 seconds, with mean value differences of 0.0461745 (both retrieval), 0.075264 (local only), and 0.0932199 (none).
AI disclosure: I used OpenAI Codex to assist this review; I verified the source and evidence against exact head cf2c3a156659426e1fd713e78c3f7ce9022f82ff.
No description provided.