Fix: Stop the live and draining pipelines on shutdown, not the boot ones #1662

Problem

On shutdown, cmd/cortex and cmd/cortex-envoy stop the pipelines they built at boot, not the ones their Holders serve. After a config reload, that means:

• The boot pipelines are stopped twice: once by the reload's drain, once at shutdown. Pipeline.Stop is not idempotent, so every plugin's Shutdown runs a second time.
• The pipelines serving traffic are never stopped, so their plugins never flush. Lineage drops its last spans, session-budget its last Redis writes, and script workers are left to die with the process.
• Old pipelines still draining after a reload are also never stopped. The drain window is now 10 minutes, and an open CONNECT tunnel or WebSocket makes the full window the usual case, so a shutdown soon after a reload usually lands in it.

Possible approach

• Stop inboundH.Load() and outboundH.Load() at shutdown instead of the boot variables.
• Give the reloader a Close(ctx) that stops every pipeline it is still draining. Call it from both binaries' shutdown, before the cost ledger flushes.
• Optionally, make Pipeline.Stop run once, so a pipeline reachable both ways is not stopped twice.

Related

Raised in the review of #1661.

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com