fix(models): close the LiteLLM provider stream on exit

[SawhneySatvik]

Summary

LitellmModel.stream_response() iterates the litellm.acompletion(..., stream=True) result with no try/finally, so CustomStreamWrapper.aclose() is never called on any path that finalizes the generator, including normal completion.

LiteLLM is the only Chat Completions adapter with this gap:

adapter	closes the provider stream?
models/openai_chatcompletions.py:370-398	yes, added by #3689
extensions/models/any_llm_model.py:603-604	yes, finally: await self._maybe_aclose(stream)
extensions/models/litellm_model.py:400-404	no

#3689 touched two files and does not mention LiteLLM. Generator finalization does not cover it either: ChatCmplStreamHandler.handle_stream contains no cleanup, which is why #3689 closed the stream in the adapter itself.

Impact: after a cancelled or completed streamed run the provider connection stays checked out of the pool until the next gen-2 collection, while the upstream provider keeps generating and transmitting billed tokens.

The fix mirrors the cleanup block openai_chatcompletions.py:370-398 already uses, rather than inventing a narrower variant:

• close the provider stream in a finally;
• on CancelledError, schedule the close in the background and re-raise, so a provider close that waits on transport I/O cannot delay cancellation (.agents/references/runner-lifecycle.md);
• once response.completed has been yielded, log and ignore a cleanup failure; before any terminal event, still raise it. This keeps equivalent streaming paths aligned on errors (.agents/references/model-provider-boundaries.md).

_maybe_aclose matches any_llm_model.py:1091-1101 so the adapters share one idiom. Scoped to LiteLLM: any_llm_model.py closes its stream but has neither guard, which is a separate gap and I did not widen this PR to cover it. A direct await stream.aclose() is not available anyway: the declared type openai.AsyncStream exposes close, not aclose, so it fails mypy --strict.

Test plan

Six tests in tests/models/test_litellm_chatcompletions_stream.py. Three pin the close itself, scoped to explicit aclose(), normal exhaustion, and cancellation; one pins that a slow provider close does not delay cancellation; two pin the terminal-error semantics. The cancellation test uses a test-owned asyncio.Event barrier with a bounded wait_for, not sleep(0).

All six fail on main:

$ uv run pytest tests/models/test_litellm_chatcompletions_stream.py -q
6 failed, 7 passed in 1.17s

$ uv run pytest tests/models/test_litellm_chatcompletions_stream.py -q -k "closes_provider_stream"
3 failed, 10 deselected in 1.52s       (assert 0 == 1)

Each guard is mutation-tested:

drop the terminal guard       -> ignores_close_error_after_terminal_event    FAILED
always swallow (if True)      -> surfaces_close_error_before_terminal_event  FAILED
drop the CancelledError branch-> does_not_block_cancellation_on_slow_close   FAILED (TimeoutError)
restored                      -> all pass

Replacing the finally body with pass fails all six, so the cleanup itself is load-bearing too.

Through the public Runner.run_streamed(...) plus result.cancel() API, with only the network boundary faked: LITELLM aclose_calls=0, OPENAI-CC aclose_calls=1.

$ ./.agents/skills/code-change-verification/scripts/run.sh
code-change-verification: all commands passed.

uv run pytest tests/models -q      604 passed
make tests                         5929 passed, 4 skipped  |  38 passed, 5 skipped (serial)

Codex /review on this head reports no actionable issues, focused regressions 62/62. Its two earlier [P1]s on this block are what produced the cancellation branch and the terminal-event guard above.

Checks

• I've added new tests, if relevant
• I've run .agents/skills/code-change-verification/scripts/run.sh
• I've confirmed all verification steps pass
• If using Codex, I've run /review before submitting this PR

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 71d8c75040

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[seratch]

Thanks for following up. The new code preserves the original stream exception correctly, but this is not a supported or reachable LiteLLM case today. The SDK requires LiteLLM >=1.83.0, acompletion(..., stream=True) returns CustomStreamWrapper, and its aclose() catches BaseException around the underlying close. The new regression test reaches the case only by replacing the private _fetch_response result with a custom stream shape.

Please simplify this PR back to the demonstrated requirement: close the LiteLLM provider stream on normal exhaustion, explicit generator close, and cancellation without delaying cancellation. Remove the stream_error and terminal cleanup-error policy plus their synthetic raising-close tests, while keeping the close and cancellation regression coverage.

[SawhneySatvik]

Sure, make sense.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a22cd076d5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".