bug: A config reload stops the old pipeline while requests are still using it #1624

Problem

A config reload stops the old pipelines on a fixed timer: 30 seconds after the swap, whether or not anything is still using them (core/reloader/reloader.go, the drain goroutine after pipelines swapped). Stop calls each plugin's Shutdown.

Since #1561, a request keeps the pipeline it started on: Holder.Run pins it on the request context, and every later phase (RunResponse, RunResponseFrame, RunFinish, Resend) runs on that pin. That fixed the crash #1561 reported. But it means any request still open 30 seconds after a reload finishes on a pipeline whose plugins have shut down. Long requests are ordinary here:

• a long SSE inference stream;
• a CONNECT tunnel, whose Finishers run when it closes;
• a Codex Responses WebSocket turn (Feat: Carry Codex's Responses WebSocket and parse every turn on it #1467), which keeps the pipeline it started on.

#1561 handled the one plugin where this skipped policy: cpex now follows fail_open when it is not ready. It did not change when the reloader stops a pipeline.

What happens today

Among plugins that tear something down in Shutdown, these are the ones a request reaches after Stop, read from the code (not reproduced):

• lineage (OnFinish): the request's response span is dropped, since the tracer provider is shut down and the OTel SDK ignores spans that end after it.
• session-budget (last frame): the tokens reach the old instance's cache, and the Redis write fails with a WARN. The new instance never sees them.
• script (frames, last frame): the worker is shared with the new build when the script did not change. A changed script's old worker has stopped, so the hook fails, and under fail_closed that is a late, recorded refusal.
• cpex: denies or allows per fail_open (bug: A config reload can crash the proxy mid-request, skip CPEX policy, or report changes it never applied #1561).

Nothing crashes, and nothing the client receives changes. The laptop's default pipeline (inference-parser, mcp-parser, a2a-parser, tool-prune) has none of these.

Possible approach

Stop an old pipeline when nothing holds it, not after a fixed delay:

• Count holders: increment when Holder.Run pins the pipeline, and decrement after the request's RunFinish. Paths that pin a pipeline outside Holder.Run (the WebSocket turns) would hold it the same way.
• The reloader waits for the count to reach zero, with a ceiling (minutes, not seconds), so a tunnel left open for hours cannot keep a pipeline's resources forever. Past the ceiling it stops the pipeline as it does today, and logs how many requests were still on it.

Raised in

CodeRabbit's review of #1467, about WebSocket turns. The same exposure exists on main for every long-lived request, so it is tracked here rather than fixed in that PR.