fix(schedule): bound preload startup wait - #213
Merged
Conversation
smiggleworth
marked this pull request as ready for review
August 24, 2026 18:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FITZ_SCHEDULE_PRELOAD_TIMEOUT_SECSWhy
Cloud recovery and the Midge primary lease can complete successfully while Schedule preload is still queued behind actor work. Fitz previously treated a one-second actor reply delay as a preload failure and exited, causing an ECS restart loop even though the exclusive-writer lease had already been acquired.
An unbounded reply wait would remove that false failure but introduce a worse liveness regression when storage or the actor remains alive but stalls. This change gives S3-backed recovery a realistic default budget while retaining a finite, observable startup failure.
Schedule is the only domain with this exact fatal startup reply wait. Other one-second domain waits remain bounded runtime delivery, cleanup, admin, or observability operations. Queue and Stream perform boot storage preparation directly and do not add this aggregate actor-reply deadline.
Midge still has unbounded internal runtime response waits that Fitz cannot interrupt once entered. That follow-up is tracked in cntryl/midge#254.
Validation
cargo fmt --all -- --checkcargo clippy --locked --workspace --all-targets --all-features -- -D warnings -D clippy::pedanticcargo test --locked --workspacecargo test --locked --lib domains::schedule::sink::tests(25 passed)actionlint .github/workflows/ci-backend.ymlAll local validation is green.