Repository navigation
fix(officeace):顶格 max_tokens 加推理预算后越界,按模型族夹上限 - #259
Open
shadyrispy wants to merge 1 commit into
Open
shadyrispy wants to merge 1 commit into
shadyrispy wants to merge 1 commit into
Conversation
1. 症状(参考实现 officeace2api 的 d8ee830 实测同款,错误原文一致):客户端给的 `max_tokens` 正好是上游上限,我们自己的「追加 1024 推理预算」(`apply_reasoning_allowance`, 64→1088 那条规则)把它加成 `394240`,上游回 400 `ModelArts.81001`,**确定性失败** (请求不变重打一百遍一样被拒)。我们与参考实现都做这个加法 ⇒ 这个坑两家都会踩。 2. 修法照抄参考实现:**先加后夹**。新增 `max_tokens_ceiling(wire_model)` + `clamp_max_tokens(body)`,调用点放在追加**之后**(顺序反了夹的是客户端原值,加完照样越界)。 只夹上限、不设下限;客户端没给 / 模型族未知 / 本就没超时都不动(单轮请求一个字节都不改)。 3. 上限只收「错误原文写明是 max_tokens **参数范围**」的两族: deepseek `[1, 393216]`、glm `[1, 131072]`(含实测错误原文,见函数头)。⚠️ qwen **故意不收**:它的实测错误是 prompt+completion **合计**的上下文限制 (`…9 in the messages, 131072 in the completion`),静态上限挡不住它,夹了只会制造 「我们以为防住了」的假象 —— 不夹、让上游 400 原样透传(fail-open,与未知族同一取向)。 4. 用例 2 条:顶格 + 追加 → 夹回(并先断言「追加之后确实越界」钉住故障形状); 未知族 / qwen / 缺 model / 缺 max_tokens 全部 fail-open。全量 632 passed(基线 630,净增 2)。 5. 出处:officeace2api `d8ee830`(带三族实测错误原文)与我们的 `apply_reasoning_allowance` 加法规则互为印证;笔记 cpa-deploy/notes/agent2api-codearts-officeace-upstream-sweep-20261014.md 第五节。
aimod-cc
force-pushed
the
beta
branch
3 times, most recently
from
October 23, 2026 17:08
e9b0c3a to
e6e4434
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
现象
客户端把 max_tokens 给到上游上限(比如 deepseek 的 393216)时,这一发必 400,重发也没用:
根因
上游把 max_tokens 当「推理 + 正文」的总预算,我们为了防「HTTP 200 但正文空串」
(
officeace/chat.rs模块头的实测坑 1)会追加 1024。追加是加法,apply_reasoning_allowance把 393216 加成 394240,正好越过上游的参数范围。「客户端顶格」和「我们追加」两件事叠在一起,就是确定性失败。
上游按模型族的参数范围(实测错误原文):
max_tokens range is [1, 393216]max_tokens参数非法:限制数值范围[1,131072]maximum context length is 131072 tokens. However, you requested 131081 tokens (9 in the messages, 131072 in the completion)第三条要单独留意:它是 prompt+completion 合计的上下文限制,不是 max_tokens 的参数范围。
修法
新增
clamp_max_tokens,调用点放在追加之后——顺序反了夹的是客户端原值,加完照样越界。常规请求的出站体与改动前逐字节一致。
夹了只会让人以为防住了。没测过的族同样不夹,让上游的 400 原样透传。
社区件 officeace2api 的
d8ee830独立测到了同一组数(错误原文逐字一致),两家实现互为印证。验证
qwen / 未知族 / 缺 model / 缺 max_tokens 全部 fail-open。
cargo test -p agent2api-server:基点(16b8855)581 条 → 583 条,0 failed,删 0 条。影响面
只动
officeace/chat.rs一个文件(+85)。max_tokens 在上限内的请求出站体逐字节不变;只有「顶格或超限」的请求从必 400 变成按上游能给的上限发出。