ai client: add streaming so output can exceed the non-streaming max_tokens ceiling

[lsimons]

Summary

The AI client issues non-streaming chat completions. That caps usable output at
roughly 21k tokens, because the Anthropic API rejects non-streaming requests
whose max_tokens climbs past about that. The rewrite path therefore sends a
flat max_tokens of 16384 and splits long bodies into chunks instead (#89).

Chunking covers everything in the corpora seen so far, so this is not urgent.
It is the blocker for two things:

• Single-call rewrites of long pages. A 55k-character body needs roughly
  23k output tokens. It works chunked, but each chunk is rewritten without
  sight of the rest of the page, so cross-section repetition is not smoothed
  out. A single call would not have that limitation.
• Progress feedback on slow calls. A large single request currently returns
  nothing for 40 to 80 seconds. Streaming would let the client report progress
  instead of appearing hung.

Proposal

Add a streaming path to the client's chat call.

• Accumulate content deltas into the same text the non-streaming path returns,
  so callers are unaffected.
• Capture finish_reason from the terminal event. The truncation guards depend
  on it, and a streamed response that ends early must be treated exactly like a
  non-streamed one: refused, and never cached.
• Token accounting needs care. Usage arrives on the final event and may require
  opting in explicitly at the gateway; without it the run summary would
  under-report. Verify against the gateway rather than assuming.
• Cache semantics stay as they are: an entry is written only for a complete
  response, and the cached value is the fully accumulated text.
• Decide whether streaming is unconditional or a config knob. Unconditional is
  simpler and has no downside for small calls if usage reporting works; a knob
  is another branch to test.

Once it lands

Revisit whether the rewrite path should raise its flat cap and stop chunking
pages that fit in one streamed call, or keep chunking for the reliability it
buys. That is a separate decision with its own trade-off and should not be
bundled in.

Acceptance criteria

• A response longer than the current non-streaming ceiling completes and is
  written correctly.
• finish_reason is captured; a truncated stream is refused and not cached.
• Prompt and completion token counts and cost in the run summary match what the
  non-streaming path reports for an equivalent call.
• Tests do not depend on a live gateway.