feat(router): install the container runtime and complete scale-to-zero - #66
Merged
Conversation
Closes #19. The router now installs a ContainerLifecycle, so everything the previous chunks built actually runs: a service with --sleep-after stops its containers when idle and starts them on the next request. Deploy refuses --sleep-after when no runtime is configured, and proves every container reference resolves before installing anything. A reference that names nothing now fails on the operator's terminal instead of at the first idle timeout an hour later, with an error that says to use --sleep-container. The one exception is a socket that answers but denies inspect, which is what a hardened socket proxy does -- that warns and proceeds, so the operators doing the right thing are not locked out. Two defects in already-merged code are fixed here rather than left: statePersister was added in #65 with a call site but nothing ever set it, so sleep and wake edges were never written and a restart forgot everything. The router now hands every service -- deployed or restored -- a persister. Configure treats a changed container set as a redeploy and forces the state back to active. A restored service builds a brand-new controller whose refs always look changed, so Configure silently undid RestoreSleeping and a sleeping service came back awake with a pool pointing at stopped containers. Configure now runs before the restore. The idle gate also moved below the response cache, into sendRequestToTarget. A cache hit never reaches the target, so it must not spend a container start -- serving stored responses while the containers stay stopped is the whole reason to run both features on one service. The gate is still below every auth, allow-list, rate-limit and redirect check, and still above target selection, so a held request is handed on with its body unread. Refs #19
4 tasks
mhenrixon
added a commit
that referenced
this pull request
Jul 29, 2026
Cuts the scale-to-zero cold wake from 1154ms to 278ms, measured end to end against a real Docker daemon on the same machine. #19 asked for a cold-wake number before optimising anything, and the number turned out to indict our own code rather than the container runtime. Waking a container whose app was ready almost immediately took 1154ms, of which docker start was only ~215ms. The proxy log showed where the rest went: 15:52:44.733 Healthcheck failed ... connection refused 15:52:45.742 Target health updated ... healthy 15:52:45.743 Service awake `docker start` returns when the container process is created, not when the application is accepting connections, so the immediate probe is routinely refused. The next probe then came a full check interval later. The wake was quantised to the health check interval -- roughly a second of pure waiting, for nothing. The health check now probes immediately, retries from 50ms doubling up to the configured interval until the target first answers, and settles to the configured interval after that. A running target is probed no harder than before, and a container that never comes up backs off rather than spinning. Deploys pay the same tax and get the same benefit: waiting for a new target to become healthy is the same wait. Also fixes a -race flake in TestService_CacheMissWakesASleepingService, which landed with #66. A wake persists state from its own goroutine, and Dispose does not wait for a write already in flight, so the test could race t.TempDir cleanup and fail with "directory not empty". The cleanup now takes saveLock, which returns only once any in-progress save has finished. Closes #59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #19. The router installs a
ContainerLifecycle, so everything the previous four chunks built actually runs. A service with--sleep-afternow stops its containers when idle and starts them on the next request.The full surface, for the gem side:
run--docker-socket/DOCKER_SOCKETdeploy--sleep-after0(never)deploy--wake-timeout30sdeploy--sleep-container(repeatable)Not
--idle-timeout— that name is already taken onrunfor HTTP keep-alive.Deploy-time preflight
--sleep-afterwith no runtime configured is refused outright, and every container reference is checked against the runtime before anything is installed. A reference naming nothing fails on the operator's terminal, with an error that says to use--sleep-container— rather than being accepted and failing at the first idle timeout an hour later.The one exception: a socket that answers but denies inspect. That is what a hardened socket proxy does, so it warns and proceeds — refusing would lock out exactly the operators doing the right thing.
Two defects in already-merged code, fixed here
1. Sleep state was never persisted.
statePersistershipped in #65 with a call site but nothing ever set it, so every sleep and wake edge was silently dropped and a restart forgot the service was asleep. The router now hands every service — deployed or restored — a persister.TestRouter_SleepEdgeIsPersistedWithoutAnExplicitSavepins it.2. A restored sleeping service came back awake.
Configuretreats a changed container set as a redeploy and forces the state to active. A restored service builds a brand-new controller whose refs always look changed, soConfiguresilently undidRestoreSleeping— leaving a service that believed it was awake with a pool pointing at stopped containers. Configure now runs before the restore. Caught byTestRouter_SleepingStateSurvivesARestart, which asserts both the state and the empty pool.The cache interaction, now fixed
Flagged in #65's merge and resolved here: the idle gate moved from
serviceRequestWithTargetintosendRequestToTarget, i.e. below the response cache.A cache hit never reaches the target, so it must not spend a container start. Serving stored responses while the containers stay stopped is the entire reason to run both features on one service — and it was the one combination that did not work.
The gate is still below every auth, allow-list, rate-limit and redirect check (so none of those can start a container), and still above target selection (so a held request is handed on with its body unread).
Test plan
DeployRejectsSleepAfterWithoutAContainerRuntime,DeployRejectsAnUnknownContainer(error names--sleep-container),DeployWarnsButProceedsWhenInspectIsForbidden,DeployReportsALifecycleFailureDeployWithoutSleepNeedsNoContainerRuntime— the feature costs nothing when unusedSleepingStateSurvivesARestart— state and suspended poolSleepEdgeIsPersistedWithoutAnExplicitSaveListShowsSleepingAndPrefersPaused— a pause is a human decision and outranks anything traffic-drivenCacheHitDoesNotWakeASleepingService/CacheMissWakesASleepingServicegofmt,go vetclean;go test -race ./internal/server ./internal/cmdgreenDeviations & judgment calls
Configure-undoes-restore bug is the one to review closely. It is invisible in isolation — the controller reportsactive, the pool reports healthy, and only a real request against a stopped container reveals it. I found it because the restart test asserted the pool was empty as well as the state; asserting state alone would have passed while the bug shipped.persistStatelogs its error rather than returning it. By the time it runs the controller has already moved, so a failed write costs a wrong state on the next boot, not a broken proxy now. Returning it would mean plumbing an error out of a background goroutine with nobody to hand it to.describeServiceStateranks paused/stopped above sleeping/waking. A human decision outranks anything traffic-driven; a paused service that is also asleep readspaused.--sleep-containeronly. The selector — which also closes thekamal rollbackhole — remains the follow-up, additive: oneDockerClientmethod and one flag.