pools: async Pool surface — ModalFlashPool rides Modal's native .aio(); readiness discovers off-loop#68
Open
jvmncs wants to merge 1 commit into
Open
pools: async Pool surface — ModalFlashPool rides Modal's native .aio(); readiness discovers off-loop#68jvmncs wants to merge 1 commit into
jvmncs wants to merge 1 commit into
Conversation
…); readiness discovers off-loop Base Pool grows gateway_url_async/discover_replicas_async/wake_async with to_thread fallbacks over the sync impls, so any sync-only Pool stays usable from async callers; ModalFlashPool overrides them with the SDK's .aio() and async-httpx wake. service.readiness switches to the async discovery, keeping blocking pool-client I/O off the caller's event loop. Confirmed by probe (stitch-dev, read-only): sync and async surfaces return identical gateway/replica results against a live Flash pool; NotFoundError parity on an app without the cls; no event-loop stall (<25ms heartbeat gap) during the async calls. Modal's own AsyncUsageWarning fires on the sync surface in an async context and suggests exactly the .aio rewrite adopted here. uv run pytest: 52 passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Poolgrows an async surface —gateway_url_async/discover_replicas_async/wake_async— withasyncio.to_threadfallbacks over the sync implementations, so any sync-only pool subclass remains usable from async callers.ModalFlashPooloverrides all three natively: the Modal SDK's.aio()interface for gateway resolution and container discovery, and async httpx for the wake fan-out (replacing sync httpx + a ThreadPoolExecutor).service.readinessdiscovers replicas through the async surface instead of calling blockingpool.discover_replicas()on the caller's event loop.Why
readiness()is awaited from serving processes' event loops (e.g. a deployment's readiness-poll handler); the inline blocking discovery stalled that loop for the duration of the pool-client round-trips. More generally, async callers previously had to choose between blocking the loop or parking a worker thread just to wait on Modal's own background loop —.aio()is the SDK's intended interface here.Verification
uv run pytest: 52 passed. Newpools/base_test.pyproves the base fallbacks run sync impls off the loop thread;_replica_urlscovered inpools/modal_flash_test.py.NotFoundErrorparity on an app without the cls; max 21ms event-loop heartbeat gap through real API round-trips. No/wakesent (live run).AsyncUsageWarningfires when the sync surface is called from a loop, and its suggested rewrite is exactly the.aio()form adopted here.