Problem
robust.ChatCompleter.ChatComplete uses iter.Pull2 in commitOnFirstPart to peek the first streamed part before committing. That spawns a goroutine running the underlying provider's push iterator; stop() releases it.
The wrapped response's defer stop() only fires if the caller actually invokes Parts(). If the caller receives the response and drops it (handles an error, times out, takes a branch that never iterates), the goroutine is parked forever at the provider's yield call, holding whatever that partsFunc closed over (HTTP response body, TCP socket, SDK state).
For now PR #209 documents the contract ("caller must invoke Parts()"). This issue tracks a real fix.
Directions to consider
- Buffer the first part and close the Pull2 iterator synchronously before returning, trading a bit of memory for a self-contained response. Requires materialising the rest of the stream too, which defeats streaming.
- Spawn a watchdog goroutine with a short deadline; if the wrapped iterator hasn't been started, call
stop() on its behalf. Introduces its own leak vector (the watchdog itself) and arbitrary timeout policy.
- Expose an explicit
Close() method on the returned response and require callers to call it on all paths; requires changing gai.ChatCompleteResponse to support closing, which is a top-level API change.
- Finalizer as last-resort cleanup. Indeterminate timing and can mask leaks in tests.
- Widen
gai.ChatCompleteResponse to carry a Close() / context-like lifecycle, so this pattern works uniformly for all wrappers.
Acceptance
- The goroutine is released regardless of whether the caller iterates
Parts().
- Documented contract is clear and consistent with what the provider clients themselves do.
Reference
Problem
robust.ChatCompleter.ChatCompleteusesiter.Pull2incommitOnFirstPartto peek the first streamed part before committing. That spawns a goroutine running the underlying provider's push iterator;stop()releases it.The wrapped response's
defer stop()only fires if the caller actually invokesParts(). If the caller receives the response and drops it (handles an error, times out, takes a branch that never iterates), the goroutine is parked forever at the provider'syieldcall, holding whatever thatpartsFuncclosed over (HTTP response body, TCP socket, SDK state).For now PR #209 documents the contract ("caller must invoke
Parts()"). This issue tracks a real fix.Directions to consider
stop()on its behalf. Introduces its own leak vector (the watchdog itself) and arbitrary timeout policy.Close()method on the returned response and require callers to call it on all paths; requires changinggai.ChatCompleteResponseto support closing, which is a top-level API change.gai.ChatCompleteResponseto carry aClose()/context-like lifecycle, so this pattern works uniformly for all wrappers.Acceptance
Parts().Reference
robust/chat_completer.go:206-223(commitOnFirstPart)