fix(meerkat-dbm): stop idle shutdown from terminating the worker mid-query#290
Merged
Merged
Conversation
c4b37c6 to
db71bc6
Compare
…query The shutdownInactiveTime timer is armed only when the queue drains (_stopQueryQueue) but was never cleared when the next query started, so a timer armed on the previous drain kept counting down through the start of the next query. When it fired, it only checked `queriesQueue.length > 0` — but a query that has been shifted off the queue executes with queriesQueue.length === 0 (currentQueryItem set, queue running) — so it treated the in-flight query as idle and called terminateDB() mid-query, killing the duckdb-wasm worker while a RUN_QUERY / SET-TimeZone postTask was still in flight. duckdb-wasm does not throw here — postTask on a detached worker only console.errors "cannot send a message since the worker is not set!" and resolves — so the earlier catch-based self-heal (#287) never fired and the error kept surfacing to users on vista/list views. Fix (two layers): - Cancel the pending recycle/shutdown timers when a query starts (_startQueryQueue). They re-arm on the next queue drain. This is the primary fix: no idle timer is pending while a query runs. - Guard the shutdown timer callback on _isBusy() (queue length OR queue running OR currentQueryItem), matching the recycle timer. Covers the event-loop edge where the timer callback was already queued before the new query cleared it. Also removed the dead _isStaleWorkerError retry from #287 (it matched a thrown message that is only ever console.error'd, never thrown). Bumps meerkat-dbm 0.1.45 -> 0.1.46. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
db71bc6 to
2bb1e5b
Compare
zaidjan-devrev
approved these changes
Jul 1, 2026
shriram-devrev
added a commit
that referenced
this pull request
Jul 1, 2026
…'t kill mid-registration (#291) Follow-up to #290. The idle recycle/shutdown timers judged "idle" only by the query queue (queriesQueue / currentQueryItem). But file-buffer registration (consumers' fetchAndRegisterChunksWithIndexedDb) holds a table lock across its multi-second download and registers buffers on the worker OUTSIDE the query queue and the teardownInProgress barrier. So the timers saw the engine as idle during registration and terminated the worker mid-flight — the next registerFileBuffer/postTask then hit a dead worker ("cannot send a message since the worker is not set!"), which is what surfaced on vista/list views. Fix: - TableLockManager.hasActiveLocks(): reports whether any reader/writer lock is currently held. - DBM._isBusy() now also returns true when hasActiveLocks() — a held lock (i.e. an in-flight registration) blocks the idle recycle/shutdown. - The shutdown timer re-arms itself when it defers on a busy state, so the engine still idles down once a lock-only operation (no trailing query) finishes — no leaked warm engine. - setShutdownLock(false) re-arms the idle timer (fixes a latent leak: a timer that fired while locked returned early and was never rescheduled). Regression test added: a held table lock across the idle-shutdown window must not terminate the worker, and shutdown still fires once the lock releases. Bumps meerkat-dbm 0.1.46 -> 0.1.47. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two layered changes so an idle timer can't terminate the duckdb-wasm worker while a query is running:
_startQueryQueue). They re-arm on the next queue drain. Primary fix — no idle timer is left pending during a query._isBusy()(queue length OR queue running ORcurrentQueryItem), matching the recycle timer. Covers the event-loop edge where the timer callback was already queued before the new query cleared it.Also removes the dead
_isStaleWorkerErrorretry from #287.Why (root cause, verified against prod)
The
shutdownInactiveTimetimer is armed only when the queue drains (_stopQueryQueue), and was never cleared when the next query started. So a timer armed on the previous drain kept counting down through the start of the next query._startQueryExecutionshifts the query off the queue before running it, so during executionqueriesQueue.length === 0whilecurrentQueryItemis set andqueryQueueRunningis true. The timer only checkedqueriesQueue.length > 0, so it treated the in-flight query as idle and called_shutdown()→terminateDB()mid-query.duckdb-wasm does not throw for this —
AsyncDuckDB.postTaskon a detached worker onlyconsole.errors"cannot send a message since the worker is not set!"and resolves. Confirmed in prod RUM: every event issource: console, handling-stackconsole error → postTask. That's also why #287's catch-based self-heal never fired — the 0.1.45 build was empirically a no-op (fixed vs unfixed builds showed identical error rates).Tests
does not terminate the worker when the shutdown timer elapses mid-query— query held in flight (slowpreQuery) pastshutdownInactiveTimemust not triggerterminateDB, then shuts down normally once idle.cancels the pending shutdown timer when a new query starts— a second query started before a prior armed timer elapses must clear it so it never fires.Full
meerkat-dbmsuite green (20 dbm.spec + all others);nx build+nx lintclean.Notes
@devrev/meerkat-dbm0.1.45 → 0.1.46. devrev-web bumps to pick it up.work-item: ISS-334477