Skip to content

feat(router): install the container runtime and complete scale-to-zero - #66

Merged
mhenrixon merged 2 commits into
dashfrom
feature/scale-to-zero-router
Jul 29, 2026
Merged

feat(router): install the container runtime and complete scale-to-zero#66
mhenrixon merged 2 commits into
dashfrom
feature/scale-to-zero-router

Conversation

@mhenrixon

Copy link
Copy Markdown
Collaborator

Summary

Closes #19. The router installs a ContainerLifecycle, so everything the previous four chunks built actually runs. A service with --sleep-after now stops its containers when idle and starts them on the next request.

kamal-proxy run --docker-socket /var/run/docker.sock
kamal-proxy deploy service1 --target web-1:3000 --host app.example.com --sleep-after 30m

The full surface, for the gem side:

Command Flag Default
run --docker-socket / DOCKER_SOCKET empty (disabled)
deploy --sleep-after 0 (never)
deploy --wake-timeout 30s
deploy --sleep-container (repeatable) infer from target

Not --idle-timeout — that name is already taken on run for HTTP keep-alive.

Deploy-time preflight

--sleep-after with no runtime configured is refused outright, and every container reference is checked against the runtime before anything is installed. A reference naming nothing fails on the operator's terminal, with an error that says to use --sleep-container — rather than being accepted and failing at the first idle timeout an hour later.

The one exception: a socket that answers but denies inspect. That is what a hardened socket proxy does, so it warns and proceeds — refusing would lock out exactly the operators doing the right thing.

Two defects in already-merged code, fixed here

1. Sleep state was never persisted. statePersister shipped in #65 with a call site but nothing ever set it, so every sleep and wake edge was silently dropped and a restart forgot the service was asleep. The router now hands every service — deployed or restored — a persister. TestRouter_SleepEdgeIsPersistedWithoutAnExplicitSave pins it.

2. A restored sleeping service came back awake. Configure treats a changed container set as a redeploy and forces the state to active. A restored service builds a brand-new controller whose refs always look changed, so Configure silently undid RestoreSleeping — leaving a service that believed it was awake with a pool pointing at stopped containers. Configure now runs before the restore. Caught by TestRouter_SleepingStateSurvivesARestart, which asserts both the state and the empty pool.

The cache interaction, now fixed

Flagged in #65's merge and resolved here: the idle gate moved from serviceRequestWithTarget into sendRequestToTarget, i.e. below the response cache.

A cache hit never reaches the target, so it must not spend a container start. Serving stored responses while the containers stay stopped is the entire reason to run both features on one service — and it was the one combination that did not work.

The gate is still below every auth, allow-list, rate-limit and redirect check (so none of those can start a container), and still above target selection (so a held request is handed on with its body unread).

Test plan

  • DeployRejectsSleepAfterWithoutAContainerRuntime, DeployRejectsAnUnknownContainer (error names --sleep-container), DeployWarnsButProceedsWhenInspectIsForbidden, DeployReportsALifecycleFailure
  • DeployWithoutSleepNeedsNoContainerRuntime — the feature costs nothing when unused
  • SleepingStateSurvivesARestart — state and suspended pool
  • SleepEdgeIsPersistedWithoutAnExplicitSave
  • ListShowsSleepingAndPrefersPaused — a pause is a human decision and outranks anything traffic-driven
  • CacheHitDoesNotWakeASleepingService / CacheMissWakesASleepingService
  • gofmt, go vet clean; go test -race ./internal/server ./internal/cmd green

Deviations & judgment calls

  • The Configure-undoes-restore bug is the one to review closely. It is invisible in isolation — the controller reports active, the pool reports healthy, and only a real request against a stopped container reveals it. I found it because the restart test asserted the pool was empty as well as the state; asserting state alone would have passed while the bug shipped.
  • persistState logs its error rather than returning it. By the time it runs the controller has already moved, so a failed write costs a wrong state on the next boot, not a broken proxy now. Returning it would mean plumbing an error out of a background goroutine with nobody to hand it to.
  • Preflight timeout is 10s — it runs on the RPC path, where a hung socket holds the operator's terminal open.
  • describeServiceState ranks paused/stopped above sleeping/waking. A human decision outranks anything traffic-driven; a paused service that is also asleep reads paused.
  • README documents the socket as root-equivalent, in a callout rather than a footnote, along with the prune limitation for Kamal versions predating the per-role fix (fix(prune): scope container pruning to a role and destination kamal#68).
  • No label-selector support. Derived refs plus --sleep-container only. The selector — which also closes the kamal rollback hole — remains the follow-up, additive: one DockerClient method and one flag.
  • Not measured. No cold-wake number for this implementation. Add opt-in scale-to-zero for idle services basecamp/kamal-proxy#228's 3.15s is theirs, and R5: Reduce scale-to-zero cold-wake latency (tiered pause/stop) #59 should not be scoped against it until we have our own. That is also why R5: Reduce scale-to-zero cold-wake latency (tiered pause/stop) #59 is not in this PR.

Closes #19. The router now installs a ContainerLifecycle, so everything the
previous chunks built actually runs: a service with --sleep-after stops its
containers when idle and starts them on the next request.

Deploy refuses --sleep-after when no runtime is configured, and proves every
container reference resolves before installing anything. A reference that names
nothing now fails on the operator's terminal instead of at the first idle
timeout an hour later, with an error that says to use --sleep-container. The one
exception is a socket that answers but denies inspect, which is what a hardened
socket proxy does -- that warns and proceeds, so the operators doing the right
thing are not locked out.

Two defects in already-merged code are fixed here rather than left:

statePersister was added in #65 with a call site but nothing ever set it, so
sleep and wake edges were never written and a restart forgot everything. The
router now hands every service -- deployed or restored -- a persister.

Configure treats a changed container set as a redeploy and forces the state back
to active. A restored service builds a brand-new controller whose refs always
look changed, so Configure silently undid RestoreSleeping and a sleeping service
came back awake with a pool pointing at stopped containers. Configure now runs
before the restore.

The idle gate also moved below the response cache, into sendRequestToTarget. A
cache hit never reaches the target, so it must not spend a container start --
serving stored responses while the containers stay stopped is the whole reason
to run both features on one service. The gate is still below every auth,
allow-list, rate-limit and redirect check, and still above target selection, so
a held request is handed on with its body unread.

Refs #19
@mhenrixon mhenrixon self-assigned this Jul 29, 2026
@mhenrixon mhenrixon added the enhancement New feature or request label Jul 29, 2026
@mhenrixon
mhenrixon merged commit 07c3c5a into dash Jul 29, 2026
5 checks passed
@mhenrixon
mhenrixon deleted the feature/scale-to-zero-router branch July 29, 2026 13:59
mhenrixon added a commit that referenced this pull request Jul 29, 2026
Cuts the scale-to-zero cold wake from 1154ms to 278ms, measured end to end
against a real Docker daemon on the same machine.

#19 asked for a cold-wake number before optimising anything, and the number
turned out to indict our own code rather than the container runtime. Waking a
container whose app was ready almost immediately took 1154ms, of which docker
start was only ~215ms. The proxy log showed where the rest went:

  15:52:44.733  Healthcheck failed ... connection refused
  15:52:45.742  Target health updated ... healthy
  15:52:45.743  Service awake

`docker start` returns when the container process is created, not when the
application is accepting connections, so the immediate probe is routinely
refused. The next probe then came a full check interval later. The wake was
quantised to the health check interval -- roughly a second of pure waiting, for
nothing.

The health check now probes immediately, retries from 50ms doubling up to the
configured interval until the target first answers, and settles to the
configured interval after that. A running target is probed no harder than
before, and a container that never comes up backs off rather than spinning.

Deploys pay the same tax and get the same benefit: waiting for a new target to
become healthy is the same wait.

Also fixes a -race flake in TestService_CacheMissWakesASleepingService, which
landed with #66. A wake persists state from its own goroutine, and Dispose does
not wait for a write already in flight, so the test could race t.TempDir cleanup
and fail with "directory not empty". The cleanup now takes saveLock, which
returns only once any in-progress save has finished.

Closes #59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

R5: Scale-to-zero idle services

1 participant