fix(telemetry): adapter_function span completion can strand _in_flight_spans entries

[planetf1]

Introduced by: PR #1558 (adapter_function lifecycle spans — Epic #929 Phase 2, not yet released). One mechanism pre-dates it (see §2).

Problem

The process-lifetime _in_flight_spans registry (mellea/telemetry/tracing.py:242) is not guaranteed to drain in two distinct ways, both triggered only by an exception during span completion — so this is a robustness gap, not a functional bug.

1. Unguarded dangling-child cleanup in finish_adapter_function_span (new in #1558)

finish_adapter_function_span (mellea/telemetry/tracing.py:1157-1202) first closes any adapter_function.<phase> child spans still open when the invocation failed (a phase that raised fires phase-start but never its success-only phase-complete). For each remaining child it:

1. pops the registry entry (:1183), then
2. calls record_exception / set_status / set_attribute / end on the child span (:1188-1191)

with no isolation in between. If any span operation raises (a misbehaving custom SpanProcessor.on_end, a debug/export pipeline, or a non-OTel span implementation), the exception aborts the loop, so:

• the remaining sibling children are never popped, and
• the parent's _in_flight_spans.pop(invocation_id) (:1193) and span.end() (:1202) never run.

Every other finisher in the module follows a pop-first invariant (finish_backend_span_success/_error :406-471, _finish_application_span_success/_error :502-560: pop, then try/finally around the span operations), so reaching the finisher always releases the registry entry — even when the span operations themselves raise. This loop is the one site in the module that breaks that invariant.

2. Handler-dispatch routing (shared by every plugin-closed span family, pre-existing)

Every span family is closed from inside a plugin hook handler — e.g. BackendTracingPlugin.on_post_call/on_error (mellea/telemetry/tracing_plugins.py:123-226) and AdapterFunctionTracingPlugin.on_invocation_complete. Plugin handler exceptions propagate (that is exactly why the adapter firing sites carry dispatch guards), so if a plugin registered ahead of the tracing plugin raises on the same hook, the closing handler never runs and that family's registry entries are stranded. For adapter_function invocations the failure is silent to the caller: the dispatch guard absorbs it with a debug warning, the adapter call succeeds, and its span tree leaks.

Impact

• Slow memory creep in long-lived processes: one parent entry plus up to N child entries per failed completion.
• The stranded parent span is never ended, so it is not exported as a complete span (it stays in the SDK batch-processor state until process exit).
• No cross-invocation functional impact: registry keys are per-invocation UUIDs, so a leaked entry cannot corrupt a later invocation.

Severity / frequency

Latent, low–medium: it requires a raising span operation or a raising earlier-registered plugin handler — i.e. broken or misconfigured telemetry. Not a blocker for #1558, which is why it is tracked here instead of being folded into that PR.

Proposed direction

• §1: restore the module invariant — isolate the per-child span operations so the loop always drains (wrap each child's close in try/except with a logged warning, or drain-then-re-raise), and consider moving the parent pop ahead of the parent's attribute/end block.
• §2: if hardened, decide at module scope — e.g. per-handler exception isolation in invoke_hook (logged) so one misbehaving plugin cannot strand another plugin's spans.

Verification status

Code-read verified at PR #1558 head (2026-08-20); not reproduced. #1558's registry-drain acceptance tests cover success/failure with well-behaved spans, which is why they pass.