Skip to content

fix(cron): ack on self-continuation send failure instead of retrying (CF 10250) - #362

Merged
important-new merged 2 commits into
InspectorHub:mainfrom
important-new:fix/cron-queue-overload-10250
Sep 24, 2026
Merged

important-new merged 2 commits into
InspectorHub:mainfrom
important-new:fix/cron-queue-overload-10250

Conversation

@important-new

Copy link
Copy Markdown
Contributor

Problem

The cron queue consumer re-enqueues itself after each sweep batch (self-continuation hop). When the queue is under load, \CRON_QUEUE.send()\ throws CF error 10250 (\Queue is overloaded. Please back off.). Because the send call was inside the outer \ ry\ block, the error propagated to the catch, which called \message.retry()\ with no delay — immediately hammering the same overloaded queue up to \max_retries\ (3) times before dropping the message.

Observed in production logs on \openinspection-cron-saas\ for the \orphan-media\ job (and potentially any other paginating job).

Fix

*\server/cron/consumer.ts* — wrap the self-continuation \send()\ in its own inner \ ry/catch. On failure, log a warning and fall through to \message.ack(). This is safe because \writeCursor\ already ran before the send, so the next */5\ tick will re-probe, find the stored cursor, and re-enqueue the sweep from the same point.

*\wrangler.jsonc* — add
etry_delay: 30\ to the cron consumer entry. This does not affect 10250 (handled in code above) but ensures any other retriable failure on the outer \job.run()\ path waits 30 s before re-delivery rather than bouncing back instantly.

(\wrangler.saas.jsonc\ carries the same
etry_delay: 30\ addition but is gitignored — applied separately to the SaaS deploy config.)

When the queue returns 10250 (overloaded) the self-continuation send()
now catches and logs rather than propagating to the outer catch and
calling message.retry().  The cursor is already written before the send,
so the next */5 tick re-probes, finds the stored cursor, and resumes the
sweep from the same point.  Retrying was the wrong response: with no
retry_delay the message bounced back immediately, hammering the same
overloaded queue up to max_retries (3) times before being dropped.

Also add retry_delay: 30 to both cron consumer configs (wrangler.jsonc
and wrangler.saas.jsonc) so any other retriable failure on the outer
job.run() path waits 30 s before re-delivery.
@important-new
important-new merged commit 97aabf6 into InspectorHub:main Sep 24, 2026
20 checks passed
@important-new
important-new deleted the fix/cron-queue-overload-10250 branch September 24, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant