feat(load-balancer): suspend and resume a pool around a sleeping container

[mhenrixon]

Summary

The pool half of scale-to-zero (#19): two LoadBalancer methods that take targets out of rotation while their container sleeps and put them back when it wakes.

Still inert. Nothing calls them yet — the idle controller gets wired up when the request gate lands. Proxy behavior is unchanged.

ResumeFromSleep is the whole point of this PR

SuspendForSleep is the obvious half: empty the pool, stop probing, so a deliberately-stopped container is neither routed to nor dialled once a second for the entire nap.

The resume is where the real defect lives, and it only shows up in the shape scale-to-zero actually targets — a single-target pool:

updateHealthyTargets stops health checking once a lone target first goes healthy (load_balancer.go:313). That means markHealthy() has already fired and waitForHealthyContext is already cancelled. So on wake, without re-arming:

WaitUntilHealthy(30s) -> nil, in 0.00s, for a container that has not started

The wake reports ready and forwards its held request straight into a connection refused. Verified by mutation — drop the re-arm and TestLoadBalancer_ResumeFromSleepRearmsWaitUntilHealthy fails in 0.00s with "readiness was reported for a target that never answered".

Replaced, not cancelled — and what that costs

WaitUntilHealthy treats any non-deadline cancellation as success. Cancelling the old context on resume would therefore tell a waiter that parked before the resume "healthy" at the exact moment every target was marked unverified.

So the context is replaced. The cost, which I want visible rather than buried: a waiter spanning a resume is never released and times out instead. That is a timeout it can retry rather than a false ready, and it is unreachable in the real flow — the controller serializes wakes behind its generation counter, so each wake waits on the context its own resume installed. TestLoadBalancer_ResumeFromSleepDoesNotReleaseAPreviousWaiter pins that behavior explicitly rather than leaving it to be discovered.

Targets re-enter as Adding, not Healthy

A successful probe promotes Adding → Healthy; a failed one only ever demotes Healthy → Unhealthy. So a container that never comes up stays out of the pool instead of flapping into it, and a woken target is held to exactly the standard a freshly deployed one is.

The third race, now fixed where it belongs

WaitUntilHealthy now reads waitForHealthyContext under the lock. This is the third finding from the review that landed in #58, which I deliberately did not fix then: the field was written once in NewLoadBalancer before publication, so it was not a race — there was no second writer. ResumeFromSleep is that second writer, so the lock lands in the commit that creates the defect rather than ahead of it.

Simplification the plan didn't anticipate

docs/plans/2026-07-29-scale-to-zero.md calls for a new RestartHealthChecks method, because BeginHealthChecks assigned stateConsumer outside the inflight lock and was only safe before a target served anything. #58 moved that assignment inside the lock, so there is nothing left to work around — this reuses BeginHealthChecks directly and adds no method.

Test plan

Six tests, all against a single-target pool on purpose:

• SuspendForSleepEmptiesThePoolAndStopsProbing — pool empty, and probe count flat across three check intervals
• ResumeFromSleepRearmsWaitUntilHealthy — the headline; readiness withheld while the backend 503s, granted once it answers. Mutation-verified.
• ResumeFromSleepDoesNotReleaseAPreviousWaiter — a resume never hands a parked waiter a success
• ResumeFromSleepReentersUnverified — suspended targets leave the pool; woken ones rejoin only after a probe
• SuspendAndResumeAreSafeUnderConcurrentRouting — 20 sleep/wake cycles against 200 concurrent pool reads
• gofmt, go vet clean; go test -race — 1251 pass

Deviations & judgment calls

• I had to correct my own test rather than the code. ResumeFromSleepDoesNotReleaseAPreviousWaiter originally asserted the parked waiter would eventually succeed once the backend answered. It does not, and should not — the resume replaced the markHealthy that would have closed its context. The design is right; my assertion was wrong. It now pins the actual guarantee (no false success) and documents why the timeout is the acceptable outcome.
• Two tests needed the backend answering during setup, since the helper waits for initial health before returning. Obvious in hindsight, but it produced a confusing 5s failure in the helper rather than in the test.
• No benchmark. SuspendForSleep/ResumeFromSleep run once per sleep and once per wake, not per request. WaitUntilHealthy gained one uncontended mutex acquire; it is called once per deploy and once per wake, not on the request path.
• SuspendForSleep clears readers as well as writers. Read targets are replicas whose lifecycle the proxy does not own, so they are never stopped — but leaving them in a pool whose writers are gone would route reads at a service that cannot serve writes. Worth a second opinion if you disagree.

Refs #19