fix(catchup): self-heal a future-parked mention-catchup cursor (#396) - #397
Open
DipeshRajoria007 wants to merge 1 commit into
Open
fix(catchup): self-heal a future-parked mention-catchup cursor (#396)#397DipeshRajoria007 wants to merge 1 commit into
DipeshRajoria007 wants to merge 1 commit into
Conversation
… skew) The mention-catchup recovery scanner advanced its cursor with `Math.max(nowTs, maxSeenTs)` and the cursor only ever moves forward. A single system-clock jump into the future (this environment's clock bounces) — or a future-dated message — parked `mention_catchup_cursor_ts` days ahead of real time. Since each scan starts at `oldest = cursor - 5`, a future cursor makes the scan skip every real message until the wall clock catches up, silently blinding catch-up. Combined with the live Socket Mode connection dying on the same skew, miniOG dropped every mention for the gap — including a "please review these prs" request that never created a job (thread 1782110778.388229). - Add `effectiveOldestTs()`: when the stored cursor is ahead of `nowTs` (or non-finite/unset), fall back to the lookback window so the scanner self-heals once the clock normalizes, instead of staying blind. - Advance the cursor to `nowTs` only (drop maxSeenTs) so a future-dated message can't ratchet it past real time. - Tests for effectiveOldestTs (sane resume, unset fallback, future-cursor self-heal, non-finite). Follow-up (separate issue): catch-up scans conversations.history, which does not return thread replies, so a missed mention posted as a thread reply still can't be recovered even with a healthy cursor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Fixes the cursor half of #396. miniOG's missed-mention catch-up scanner parked its cursor in the future on a system-clock skew and never recovered, so it silently stopped recovering mentions. In the incident (thread
1782110778.388229) a@miniOG please review these prsrequest created no job and got no reply.Root cause
mentionCatchup.tsadvanced the cursor withMath.max(nowTs, maxSeenTs), and the cursor only ever moves forward. A clock jump into the future (this box's clock bounces) — or a future-dated message viamaxSeenTs— latchedmention_catchup_cursor_ts~5.5 days ahead (observed value1782634754→ 2026-06-28, written 06-22 20:43). Each scan starts atoldest = cursor - 5, so a future cursor makesconversations.historyskip every real message until wall-clock time catches up. Nothing reset it → multi-day blackout.Change (
sidecar/src/slack/mentionCatchup.ts)effectiveOldestTs(storedCursor, nowTs, lookbackSeconds)— when the stored cursor is ahead ofnowTs(or non-finite/≤0), fall back to the lookback window so the scanner self-heals once the clock normalizes; otherwise resume atstoredCursor - 5. Logs a WARN when a future cursor is detected.nowTsonly (droppedmaxSeenTs) so a stray future-dated message can't ratchet it past real time.Tests
sidecar/tests/mentionCatchupHistory.test.ts(+4) covereffectiveOldestTs: sane resume (cursor-5), unset → lookback, future-parked cursor → lookback (self-heal), non-finite → lookback.All 843 sidecar tests pass; build, eslint (zero-warning), prettier green.
Out of scope (tracked in #396)
conversations.history, which doesn't return non-broadcast thread replies — so a missed mention posted as a thread reply (this incident) still can't be recovered even with a healthy cursor. Needsconversations.repliesover recently-active threads.Closes part of #396.
🤖 Generated with Claude Code