Add opt-in scale-to-zero for idle services

[komagata]

Summary

Add opt-in scale-to-zero for low-traffic services. When configured, kamal-proxy stops write containers after an idle timeout, starts them on the next application request, waits for health readiness, and then forwards the held request.

This continues the work started by @martijnenco in #197. Thank you for providing the implementation that made this possible.

Behavior

• Disabled by default; enabled per service with --idle-timeout
• Coalesces concurrent wake requests into one container start
• Holds requests before reading their bodies, preserving POST and chunked bodies
• Prevents idle shutdown while requests, WebSockets, or streams are in flight
• Persists sleeping state across proxy restarts
• Handles deploy, pause, stop, and resume transitions without waking stale targets
• Restarts health checks after waking and returns 503 on timeout or lifecycle failure
• Negotiates a Docker API version compatible with the daemon

Container operations are isolated behind the small ContainerLifecycle interface. The initial implementation connects directly to the Docker socket because it is the smallest opt-in approach. This grants powerful host-level Docker access and is documented explicitly. A restricted host-side start/stop service can be added later without changing the idle controller.

The design and Docker socket tradeoff were discussed in #222. The corresponding Kamal PR, basecamp/kamal#1916, configures idle settings and mounts the Docker socket only when the feature is enabled.

Validation

• go test ./...
• go test -race ./internal/server ./internal/cmd
• go vet ./...
• git diff --check origin/main...HEAD

The implementation was also tested with real Docker and a Rails application on a 2 GB Ubuntu VPS:

• Rails used roughly 200–315 MiB while running and 0 B while stopped
• Cold wake completed in about 3.15 seconds
• A 65,536-byte chunked POST body was preserved
• 20 concurrent mixed requests caused exactly one Docker start
• Proxy restart, wake timeout/retry, Docker failure/retry, and deploy while sleeping passed
• Three Rails services slept and woke independently

Detailed results and reproduction instructions: https://docs.komagata.org/6456

[Copilot]

Pull request overview

Adds an opt-in “scale-to-zero” capability to kamal-proxy services by introducing an idle state machine that can stop write containers after inactivity and transparently wake them (including waiting for health readiness) on the next non-health-check request. This integrates Docker lifecycle management behind a small interface, persists sleep state, and exposes configuration via deploy/run flags plus documentation.

Changes:

• Introduces IdleController and a Unix-socket DockerClient to stop/start containers and coordinate wake coalescing + readiness waiting.
• Wires idle behavior into Service/Router (state persistence, listing state, health-check behavior while sleeping, wake health-check restart).
• Adds CLI/config/docs support for --idle-timeout, --idle-wake-timeout, and --docker-socket.

Tip

If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
README.md	Documents the opt-in scale-to-zero behavior, Docker socket implications, and version negotiation.
internal/server/testing.go	Updates test server construction for the new NewRouter signature.
internal/server/target.go	Adds container-name access and health-check restart hooks used by wake behavior.
internal/server/service.go	Integrates IdleController into request flow, state persistence, and wake readiness waiting.
internal/server/service_test.go	Updates service construction and adds coverage for idle lifecycle initialization.
internal/server/router.go	Adds Docker client wiring, restores idle state correctly, and serializes state snapshots.
internal/server/router_test.go	Updates router construction for the new NewRouter signature.
internal/server/load_balancer.go	Adds PrepareForWake to restart health checks and re-arm healthy-wait state.
internal/server/load_balancer_test.go	Adds coverage for wake health-check restart behavior.
internal/server/idle_controller.go	New idle state machine with sleep/wake transitions, coalescing, and persistence hooks.
internal/server/idle_controller_test.go	New unit tests for idle sleep/wake/coalescing/restore behavior.
internal/server/health_check.go	Adds explicit Start() and refactors construction to support restart semantics.
internal/server/docker_client.go	New minimal Docker HTTP client over Unix socket with API version negotiation.
internal/server/docker_client_test.go	New tests for negotiation, fallback, concurrency, and error-body bounding.
internal/server/config.go	Adds DefaultDockerSocketPath and a config field for socket path.
internal/cmd/util.go	Adds getEnvString for string-valued flag defaults from env.
internal/cmd/run.go	Adds --docker-socket and passes it into NewRouter.
internal/cmd/deploy.go	Adds --idle-timeout and --idle-wake-timeout service options.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

internal/server/service.go:252

• Dispose assumes s.active is always non-nil and will panic if a Service is constructed before an active load balancer is installed (e.g., tests already cover this scenario). Guarding the call avoids a nil dereference during teardown/error paths.

	s.active.Dispose()
	if s.rollout != nil {
		s.rollout.Dispose()
	}

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

internal/server/router.go:394

• When os.Create fails, the error is returned without logging. Many call sites defer saveStateSnapshot() and ignore its return value, so state persistence failures can become silent. Logging the create error here avoids losing that signal.

	f, err := os.Create(r.statePath)
	if err != nil {
		return err
	}

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

internal/server/idle_controller.go:139

• In BeginRequest, the timer is stopped but its channel is not drained. If the timer fires concurrently with the wake completion / ctx cancellation, leaving the value in timer.C can cause unnecessary retention and subtle races. Use the standard Stop+drain pattern in both the <-done and <-ctx.Done() branches.

		timer := time.NewTimer(timeout)
		select {
		case <-done:
			timer.Stop()
			if state == IdleStateStopping {

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

internal/server/idle_controller.go:139

• time.NewTimer is stopped without draining its channel. If the timer fires concurrently with selecting <-done, this can leave a value in timer.C, which is a common source of subtle leaks and flaky behavior in loops. Drain timer.C when Stop() reports it already fired.

		timer := time.NewTimer(timeout)
		select {
		case <-done:
			timer.Stop()
			if state == IdleStateStopping {

internal/server/idle_controller.go:150

• In the ctx.Done() branch, the timer is stopped but not drained. As with the <-done branch, a concurrent timer fire can leave a value buffered on timer.C. Drain when Stop() returns false to avoid accumulating unread timer events.

			return ErrIdleWakeTimeout
		case <-ctx.Done():
			timer.Stop()
			return ctx.Err()

[komagata]

Hi @kevinmcconnell, would you be able to review this PR when you have a chance?

All feedback so far has been addressed, and the implementation has been validated with real Docker and a Rails application. The corresponding Kamal configuration is in basecamp/kamal#1916.

I’d especially appreciate your feedback on whether the overall opt-in, Docker-socket-based approach fits the direction of kamal-proxy. I’m happy to rework or split the implementation if that would make it easier to review.

[komagata]

Hi @kevinmcconnell, just following up on this review request. Please let me know if there’s anything I can do to help move the review forward.