fix(ddocs-indexer): worker fired 13-27x/day but was structurally unable to index anything (hardcoded date prefix) - #2
Open
rwnq8 wants to merge 1 commit into
Conversation
The scheduled handler called /index?limit=30 with no prefix param, so handleIndex resolved prefix to (env.DDOC_PREFIX || "2026-09-04") + "/". The worker therefore only ever listed objects under the literal key prefix "2026-09-04/" and skipped every later object as wrong_prefix. Effect: the worker fired 13-27x/day but produced no output after 2026-09-04. Evidence: ddocs_index_state holds exactly 1 row, indexed_at 2026-09-04T09:57:49, while worker_activity_daily shows req24 10-27/day through 2026-09-13. Fix: derive prefixes from a rolling UTC date window (last 7 days) when DDOC_PREFIX is unset, and derive `rel` from the key's leading YYYY-MM-DD segment instead of assuming one pinned prefix. An explicit ?prefix= param still overrides. Content-hash dedup in ddocs_index_state is unchanged, so repeat scans stay cheap. Co-authored-by: Chatbox <chatbox@chatboxai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Defect
qnfo-ddocs-indexerfires 13–27×/day but has produced one row of output ever.SELECT COUNT(*), MAX(indexed_at) FROM ddocs_index_staten=1,indexed_at=2026-09-04T09:57:49worker_activity_dailyreq24 (09-10..09-13)Root cause
The scheduled handler invokes
/index?limit=30with noprefixparam, so:With
DDOC_PREFIXunset, the prefix is the literal string2026-09-04/. Every run lists only that one key range, and every object uploaded after 2026-09-04 is rejected aswrong_prefix. The worker can never see a newer document — it is structurally incapable of indexing anything, permanently.Fix
DDOC_PREFIXis unset.relis derived from the key's own leadingYYYY-MM-DD/segment instead of assuming one pinned prefix.?prefix=still overrides. Comma-separatedDDOC_PREFIXis supported.scannedPrefixesfor diagnosis;/statsreportsnewest.ddocs_index_stateis unchanged, so repeat scans of the same window stay cheap.Deploy note
This is a source fix. It only takes effect on redeploy, and the deploy transport reads a single R2 key (
r2:qnfo-canonical/qnfo-ddocs-indexer.js) — the canonical object must be refreshed from this repo before the fix is live. Same transport limitation documented inqnfo-observability/worker.jsv1.1.6-single-module.Verification after deploy
ddocs_index_state.indexed_atshould move past 2026-09-04 on the next cron.Co-authored-by: Chatbox chatbox@chatboxai.com