Skip to content

fix: allow include_thoughts + thinking_level together for Gemini 3.x - #2433

Open
hustxiayang wants to merge 2 commits into
envoyproxy:mainfrom
hustxiayang:include-thoughts-fix
Open

fix: allow include_thoughts + thinking_level together for Gemini 3.x#2433
hustxiayang wants to merge 2 commits into
envoyproxy:mainfrom
hustxiayang:include-thoughts-fix

Conversation

@hustxiayang

@hustxiayang hustxiayang commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Description
For GCP Vertex Gemini models, the gateway could not send thinking_level and include_thoughts in the same request. As a result, callers using reasoning_effort (which maps to thinking_level) could never receive thought summaries (reasoning_content), even though Gemini 3.x returns them when both fields are set.

This pr is to allow a separate bool value: include_thoughts to determine whether it returns the thought summary.

client.chat.completions.create(
    model="gemini-3.5-flash",
    messages=[...],
    reasoning_effort="high",                 # depth  -> ThinkingLevel  (unchanged for all models)
    extra_body={"include_thoughts": True},   # opt-in summary -> IncludeThoughts  (new, Gemini-only)
)

This would not break existing interface. There is a bit duplication with the budget setup, but it's fine as the budget setup is deprecating.

@hustxiayang
hustxiayang requested a review from a team as a code owner July 27, 2026 17:56
@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jul 27, 2026
@hustxiayang
hustxiayang force-pushed the include-thoughts-fix branch from 6ad226b to 633f3c2 Compare July 27, 2026 17:57
@hustxiayang
hustxiayang marked this pull request as draft July 27, 2026 18:01
Signed-off-by: yxia216 <yxia216@bloomberg.net>
@hustxiayang
hustxiayang force-pushed the include-thoughts-fix branch from 633f3c2 to 18c5ed1 Compare July 27, 2026 18:50
@hustxiayang
hustxiayang marked this pull request as ready for review July 27, 2026 18:51
@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Jul 27, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.85%. Comparing base (e2c59d3) to head (500a7e3).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2433      +/-   ##
==========================================
- Coverage   84.86%   84.85%   -0.02%     
==========================================
  Files         154      154              
  Lines       22419    22423       +4     
==========================================
+ Hits        19026    19027       +1     
- Misses       2237     2239       +2     
- Partials     1156     1157       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants