[Feature]: Streaming requests continue upstream generation and full billing after client-side cancellation

[yyhhyyyyyy]

Category

Streaming (SSE, chunked responses)

API protocol used

OpenAI Responses (/api/v1)

Describe the bug

When using gpt-5.4-pro or gemini-3.1-pro-preview in streaming mode, I was told that ZenMux continues waiting for the upstream model to finish even if the client cancels early.

This means the request may still be billed normally in all of these cases:

1. full response received normally
2. no response content received, then client cancels
3. partial response received, then client cancels

This is confusing because some providers support early cancellation semantics, where billing stops after the client aborts and only input + partial output tokens are charged.

Steps to reproduce

all of these cases:

1. full response received normally
2. no response content received, then client cancels
3. partial response received, then client cancels

Expected behavior

Billing is based on the output tokens at termination, not on the full usage.

Actual behavior

Currently, full charges will apply.

Severity

High - Major feature is broken, no workaround

Environment

Generation ID (if available)

No response

Integration tool (if applicable)

No response

Screenshots / Logs

No response

Additional context

No response

Pre-submission checklist

• I have searched existing issues to avoid duplicates

[lvqier]

This is currently by design.

[thinkthinking]

reopen and turn into enhancement