Reliability: Media Streams WS lifecycle — reconnection, graceful degradation, 'abandoned' outcome

[teetangh]

Context

A mid-call Twilio Media Streams WebSocket drop currently just kills the pipeline — no fallback, no resume, no recorded outcome. This is a documented real failure mode: engineering-log #10 (cloudflared disconnect, Twilio unreachable ~30 s) and #15 (silent inbound STT). streaming_ws() (apps/voice/streaming.py:415) has an idle timeout (60 s) but no abnormal-close handling, and CollectionsResult.final_outcome="abandoned" is defined in the schema but never produced.

Implementation plan

1. Twilio-side fallback: the outbound call is placed with record=True and status callbacks (apps/api/voice_twiml.py:52). Add a TwiML fallback: when <Connect><Stream> ends unexpectedly (stream stop without an end_call tool outcome), the next TwiML instruction runs — append a <Say> apology + callback promise, or a <Redirect> to a recovery endpoint that re-issues <Connect><Stream> to a fresh WS (one retry max).
2. Server-side cleanup: in streaming_ws(), distinguish normal end (EndFrame from end_call) from abnormal close (WebSocketDisconnect, handshake failure, pipeline exception). On abnormal close: cancel the PipelineTask, finalize metrics, mark the session outcome abandoned, and persist partial transcript.
3. Status-callback teardown: handle Twilio completed/failed call-status callbacks to reconcile session state (call ended by borrower hangup vs stream drop) and stamp the final outcome.
4. Wire abandoned: produce CollectionsResult.final_outcome="abandoned" for unrecovered drops so the Temporal workflow and eval data see it.
5. Tests: extend scripts/simulate_twilio_stream.py with a --kill-after N mode that severs the WS mid-conversation; integration smoke asserting clean teardown + abandoned outcome; unit tests for the close-classification logic.

Acceptance criteria

• Killing the WS mid-call in the simulator: no unhandled exceptions, pipeline task cancelled, session outcome recorded as abandoned, metrics summary still written.
• One automatic reconnect attempt on stream drop while the call is still up; second drop → apology + graceful hangup.
• Borrower hangup (call-status completed without tool outcome) recorded distinctly from stream failure.

Dependencies

• Reliability: Redis-backed session store (replace in-memory _SESSIONS/_PENDING) #4 Redis-backed session store (cross-worker session lookup for the recovery endpoint).
• P0 fixes PR (metrics finalize-on-close).