fix(alerts): a flapping phone still buzzed every cycle — cap pushes per worker - #349
Conversation
|
Warning Review limit reached
Next review available in: 24 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches 💡 1⚔️ Resolve merge conflicts 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…a rhythmic phone The sustained-recovery damping (v1.36.1) was defeated within hours by the same phone: its NORMAL duty cycle turned out to be 7-10 minutes awake, 2-3 minutes dozing — every wake window passed the sustained bar, re-armed the alert, and the next nap pushed again. Eight pushes in two hours, live, with the damping active. A worker whose ordinary operation is rhythmic flapping can never be rate-limited by episode detection. So the cap moves to the notification itself: after an offline push for a worker, further offline pushes for THAT worker wait out a 6-hour floor, independent of episodes. The first push for a genuinely dead server is still immediate; the dashboard and bell keep tracking every episode in real time; only the phone-buzz is capped (worst case 4/day/worker, down from ~100). Recovery deliberately never resets the floor — resetting on recovery is exactly the re-arm loop this ends. A total delivery failure releases the floor so the un-dedupe retry contract stays true. Floor state is in-memory: a restart forgets it, costing at most one early push. Pinned in TestOfflinePushFloor (records-but-does-not-push inside the floor, expiry re-pushes, recovery never resets, failed delivery releases); mutation-verified — removing the gate turns the suite red.
d138d9d to
2b6aaad
Compare
|
@coderabbitai review |
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #349 +/- ##
==========================================
+ Coverage 95.56% 95.58% +0.01%
==========================================
Files 51 51
Lines 7259 7268 +9
==========================================
+ Hits 6937 6947 +10
+ Misses 322 321 -1
🚀 New features to boost your workflow:
|
|
@coderabbitai review |
|
|
@coderabbitai review |
|
1 participant
The sustained-recovery damping merged this morning (#348) was defeated within hours, by the same phone, in the opposite direction: its normal duty cycle turned out to be 7–10 minutes awake, 2–3 minutes dozing. Every wake window passed the ~4-minute sustained bar, re-armed the alert, and the next nap pushed again — eight Telegram messages in two hours with the damping active. A worker whose ordinary operation is rhythmic flapping can never be rate-limited by episode detection, whatever the threshold.
The cap now sits on the notification itself. After an offline push for a worker, further offline pushes for that worker wait out a 6-hour floor, independent of episodes. The first push for a genuinely dead server is still immediate. The dashboard and bell keep tracking every episode in real time — only the phone-buzz is capped (worst case 4/day/worker, down from ~100/day observed). Recovery deliberately never resets the floor, because resetting on recovery is exactly the re-arm loop this ends. A total delivery failure releases the floor, so the existing un-dedupe retry contract ("will retry on the next cycle") stays true.
Floor state is in-memory: a restart forgets it, which at worst costs one early push.
TestOfflinePushFloorpins all four properties (suppressed-but-recorded inside the floor, expiry re-pushes, recovery never resets, failed delivery releases), mutation-verified — removing the gate turns the suite red.