fix(meerkat-dbm): treat held table locks as busy so idle shutdown can't kill mid-registration#291
Merged
Merged
Conversation
…'t kill mid-registration Follow-up to #290. The idle recycle/shutdown timers judged "idle" only by the query queue (queriesQueue / currentQueryItem). But file-buffer registration (consumers' fetchAndRegisterChunksWithIndexedDb) holds a table lock across its multi-second download and registers buffers on the worker OUTSIDE the query queue and the teardownInProgress barrier. So the timers saw the engine as idle during registration and terminated the worker mid-flight — the next registerFileBuffer/postTask then hit a dead worker ("cannot send a message since the worker is not set!"), which is what surfaced on vista/list views. Fix: - TableLockManager.hasActiveLocks(): reports whether any reader/writer lock is currently held. - DBM._isBusy() now also returns true when hasActiveLocks() — a held lock (i.e. an in-flight registration) blocks the idle recycle/shutdown. - The shutdown timer re-arms itself when it defers on a busy state, so the engine still idles down once a lock-only operation (no trailing query) finishes — no leaked warm engine. - setShutdownLock(false) re-arms the idle timer (fixes a latent leak: a timer that fired while locked returned early and was never rescheduled). Regression test added: a held table lock across the idle-shutdown window must not terminate the worker, and shutdown still fires once the lock releases. Bumps meerkat-dbm 0.1.46 -> 0.1.47. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
zaidjan-devrev
approved these changes
Jul 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Follow-up to #290.
DBM._isBusy()now also treats a held table lock as busy (via newTableLockManager.hasActiveLocks()), so the idle recycle/shutdown timers cannot terminate the worker while a file-buffer registration is in flight. Also: the shutdown timer re-arms when it defers on a busy state, andsetShutdownLock(false)re-arms the idle timer.Why (root cause, audited)
#290 stopped the idle timers from firing while a query is in the queue. But list-view data loads register parquet buffers via
fetchAndRegisterChunksWithIndexedDb, which:dbm.lockTables()write lock across its multi-second download, andqueryWithTables, so outside the query queue and theteardownInProgressbarrier._isBusy()only looked at the queue, so during registration the engine looked idle. The recycle (30s) / shutdown timers then calledterminateDB()mid-registration, and the nextregisterFileBuffer/postTaskhit a dead worker →"cannot send a message since the worker is not set!"— the vista/list-view error (ISS-334477).An audit of every worker-touching call site confirmed this is the gap for the lock-holding (INDEXDB) registration path, which is the confirmed failing path in production.
Changes
TableLockManager.hasActiveLocks()— any reader/writer lock currently held.DBM._isBusy()— also true whenhasActiveLocks().setShutdownLock(false)re-arms the idle timer (fixes a latent leak where a timer that fired while locked was never rescheduled).Tests
Added to
dbm.spec.ts: a held table lock across the idle-shutdown window must not terminate the worker; shutdown still fires once the lock releases. Fullmeerkat-dbmsuite green;nx build+nx lintclean.Notes
@devrev/meerkat-dbm0.1.46 → 0.1.47. devrev-web will bump to pick this up (with a regression test that reproduces the gap against the shipped package).downloadAndRegisterFilesInDuckDB) registers without a lock, so it isn't covered byhasActiveLocks. Tracked for a follow-up; Razorpay is on the non-parallel INDEXDB path, which this PR covers.work-item: ISS-334477