Skip to content

fix(ddocs-indexer): worker fired 13-27x/day but was structurally unable to index anything (hardcoded date prefix) - #2

Open
rwnq8 wants to merge 1 commit into
mainfrom
chatbox/fix-ddocs-indexer-rolling-prefix
Open

rwnq8 wants to merge 1 commit into
mainfrom
chatbox/fix-ddocs-indexer-rolling-prefix

Conversation

@rwnq8

@rwnq8 rwnq8 commented Sep 13, 2026

Copy link
Copy Markdown
Member

Defect

qnfo-ddocs-indexer fires 13–27×/day but has produced one row of output ever.

Evidence Value
SELECT COUNT(*), MAX(indexed_at) FROM ddocs_index_state n=1, indexed_at=2026-09-04T09:57:49
worker_activity_daily req24 (09-10..09-13) 10, 27, 13, 13 — firing continuously
Worker created 2026-09-04

Root cause

The scheduled handler invokes /index?limit=30 with no prefix param, so:

// handleIndex (v1.0)
const prefix = url.searchParams.get("prefix") || (env.DDOC_PREFIX || "2026-09-04") + "/";
// processObject (v1.0)
const prefix = env.DDOC_PREFIX || "2026-09-04";
if (!key.startsWith(prefix + "/")) return { key, skipped: true, reason: "wrong_prefix" };

With DDOC_PREFIX unset, the prefix is the literal string 2026-09-04/. Every run lists only that one key range, and every object uploaded after 2026-09-04 is rejected as wrong_prefix. The worker can never see a newer document — it is structurally incapable of indexing anything, permanently.

Fix

  • Prefixes now come from a rolling UTC window (last 7 days) when DDOC_PREFIX is unset.
  • rel is derived from the key's own leading YYYY-MM-DD/ segment instead of assuming one pinned prefix.
  • An explicit ?prefix= still overrides. Comma-separated DDOC_PREFIX is supported.
  • Response now reports scannedPrefixes for diagnosis; /stats reports newest.
  • Content-hash dedup in ddocs_index_state is unchanged, so repeat scans of the same window stay cheap.

Deploy note

This is a source fix. It only takes effect on redeploy, and the deploy transport reads a single R2 key (r2:qnfo-canonical/qnfo-ddocs-indexer.js) — the canonical object must be refreshed from this repo before the fix is live. Same transport limitation documented in qnfo-observability/worker.js v1.1.6-single-module.

Verification after deploy

GET /stats   -> indexed > 1 and newest advancing

ddocs_index_state.indexed_at should move past 2026-09-04 on the next cron.

Co-authored-by: Chatbox chatbox@chatboxai.com

The scheduled handler called /index?limit=30 with no prefix param, so
handleIndex resolved prefix to (env.DDOC_PREFIX || "2026-09-04") + "/".
The worker therefore only ever listed objects under the literal key
prefix "2026-09-04/" and skipped every later object as wrong_prefix.

Effect: the worker fired 13-27x/day but produced no output after
2026-09-04. Evidence: ddocs_index_state holds exactly 1 row, indexed_at
2026-09-04T09:57:49, while worker_activity_daily shows req24 10-27/day
through 2026-09-13.

Fix: derive prefixes from a rolling UTC date window (last 7 days) when
DDOC_PREFIX is unset, and derive `rel` from the key's leading
YYYY-MM-DD segment instead of assuming one pinned prefix. An explicit
?prefix= param still overrides. Content-hash dedup in ddocs_index_state
is unchanged, so repeat scans stay cheap.

Co-authored-by: Chatbox <chatbox@chatboxai.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant