graft build --deep fails with GPT-5.6 / o-series reasoning models (max_tokens, temperature, tool+reasoning_effort)

[nguyenduyhung1989]

Summary

graft build --deep fails outright when pointed at OpenAI's current reasoning-family models (tested with gpt-5.6-luna via GRAFT_PROVIDER=openai). Three separate 400s come back from src/ai/llm/openai.ts, each because the adapter sends a Chat Completions parameter that this model family rejects. All three block the whole --deep pass (summarize, crux, and synthesize all use the same adapter).

Environment

• graft 0.12.0 (built from source, this repo)
• GRAFT_PROVIDER=openai, GRAFT_MODEL=gpt-5.6-luna, default api.openai.com base URL
• Real repo, ~150 TS/TSX files, default --concurrency

Errors observed, in the order they surface

1. max_tokens

   400 Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.
   

   src/ai/llm/openai.ts unconditionally sets params.max_tokens = req.maxTokens (line ~111). This model family requires max_completion_tokens.

2. temperature (surfaces once docs: rewrite README around Graft #1 is worked around)

   400 Unsupported value: 'temperature' does not support 0 with this model. Only the default (1) value is supported.
   

   summarize.ts / crux.ts / synthesize.ts all hardcode temperature: 0 for determinism. This model only accepts the default (no temperature field at all).

3. Tool calls + reasoning effort (surfaces once docs: rewrite README around Graft #1 and graft viz: interactive context-graph visualizer #2 are worked around)

   400 Function tools with reasoning_effort are not supported for gpt-5.6-luna in /v1/chat/completions. To use function tools, use /v1/responses or set reasoning_effort to 'none'.
   

   Both crux.ts and synthesize.ts force tool-calling (responseFormat: { kind: "tool" }). On /v1/chat/completions, this model's default reasoning effort conflicts with function tools; the API's own error message names the fix.

Suggested fix

createChatCompletion already has a narrow, error-message-driven fallback for one OpenAI-compatible quirk (isRejectedObjectToolChoice → retry with tool_choice: "required"). The same pattern extends cleanly to all three errors above — detect the specific 400 by message, retry once with the corrected param, never guess from the model name (so older endpoints/models that still expect max_tokens/custom temperature are untouched).

I have a working patch against src/ai/llm/openai.ts doing exactly this (bounded retry loop, one fix applied per detected error, falls through to the original error if nothing matches):

--- a/src/ai/llm/openai.ts
+++ b/src/ai/llm/openai.ts
@@ -85,6 +85,43 @@ function isRejectedObjectToolChoice(err: unknown): boolean {
   );
 }
 
+/**
+ * Newer reasoning-family models (o1/o3/o4, the gpt-5.x line, …) reject the
+ * classic `max_tokens` param outright and require `max_completion_tokens`
+ * instead, even though both are still in wide use across OpenAI-compatible
+ * servers. Detect the specific 400 rather than guessing from the model name,
+ * so older endpoints that still expect `max_tokens` are untouched.
+ */
+function isRejectedMaxTokens(err: unknown): boolean {
+  return (
+    err instanceof OpenAI.APIError &&
+    err.status === 400 &&
+    /max_tokens.*not supported.*max_completion_tokens/i.test(String((err as { message?: string }).message ?? ""))
+  );
+}
+
+/** Same reasoning-family models: temperature is fixed at 1, not caller-settable. */
+function isRejectedTemperature(err: unknown): boolean {
+  return (
+    err instanceof OpenAI.APIError &&
+    err.status === 400 &&
+    /temperature.*does not support/i.test(String((err as { message?: string }).message ?? ""))
+  );
+}
+
+/**
+ * Same models again: function tools are rejected on /v1/chat/completions while
+ * the model's default reasoning effort is active. The API's own error message
+ * names the fix — `reasoning_effort: "none"` — so apply exactly that.
+ */
+function isRejectedToolsWithReasoning(err: unknown): boolean {
+  return (
+    err instanceof OpenAI.APIError &&
+    err.status === 400 &&
+    /function tools with reasoning_effort/i.test(String((err as { message?: string }).message ?? ""))
+  );
+}
+
 export class OpenAIChatModel implements ChatModel {
   readonly label: string;
   private client: OpenAI;
@@ -141,14 +178,34 @@ export class OpenAIChatModel implements ChatModel {
    * automatically.
    */
   private async createChatCompletion(params: ChatParams): Promise<OpenAI.Chat.Completions.ChatCompletion> {
-    try {
-      return await this.client.chat.completions.create(params);
-    } catch (err) {
-      if (isRejectedObjectToolChoice(err) && typeof params.tool_choice === "object" && params.tools?.length === 1) {
-        return this.client.chat.completions.create({ ...params, tool_choice: "required" });
+    let attempt = params;
+    // Bounded: one retry per known incompatibility below, never an open loop.
+    for (let i = 0; i < 4; i++) {
+      try {
+        return await this.client.chat.completions.create(attempt);
+      } catch (err) {
+        if (isRejectedObjectToolChoice(err) && typeof attempt.tool_choice === "object" && attempt.tools?.length === 1) {
+          attempt = { ...attempt, tool_choice: "required" };
+          continue;
+        }
+        if (isRejectedMaxTokens(err) && attempt.max_tokens !== undefined) {
+          const { max_tokens, ...rest } = attempt;
+          attempt = { ...rest, max_completion_tokens: max_tokens } as ChatParams;
+          continue;
+        }
+        if (isRejectedTemperature(err) && attempt.temperature !== undefined) {
+          const { temperature, ...rest } = attempt;
+          attempt = rest as ChatParams;
+          continue;
+        }
+        if (isRejectedToolsWithReasoning(err)) {
+          attempt = { ...attempt, reasoning_effort: "none" } as ChatParams;
+          continue;
+        }
+        throw err;
       }
-      throw err;
     }
+    return this.client.chat.completions.create(attempt);
   }
 
   private fromResponse(

Happy to open this as a PR instead if that's more useful than a patch pasted into an issue — let me know.

Unrelated but adjacent: rate limits on lower tiers

Not a graft bug, just a heads-up: gpt-5.6-luna on a tier-1/2 org caps at 200k TPM, separate from any daily token allowance. The default --concurrency 5 for the deep pass burst past that within the first few seconds on a ~150-file repo, and the existing "5 consecutive failures → abort" circuit breaker (ai/failure.ts) then killed the whole run rather than backing off. Dropping to --concurrency 2 worked around it. Might be worth a line in the README/--deep docs about lowering --concurrency for lower-tier accounts, since the abort-on-429 behavior otherwise looks like a hard failure rather than a pacing issue.

Result once patched

With the patch above, --deep completed cleanly end-to-end and produced genuinely useful concept nodes from a real repo — e.g. one node's generated summary: "A snapshot download plan is a constrained projection of repository state: every selected blob must be declared by the manifest, reside under wiki/public, and be a supported regular file; private paths, traversal, symlinks, unsupported files, missing entries, and incomplete trees must never enter the plan." So the fix is worth landing — the model family works well once these three params are handled.

[nguyenduyhung1989]

Follow-up: the same class of failure reproduces on a second provider, and one of the three fixes above turns out to be in the wrong order. Everything below supersedes the patch in the original report.

A fourth 400, from DeepSeek

Running graft build --deep against DeepSeek's v4 line (GRAFT_PROVIDER=openai, GRAFT_BASE_URL=https://api.deepseek.com/v1, GRAFT_MODEL=deepseek-v4-flash) gets through the file-summary pass and then dies on synthesis:

reading concepts 1/2: src/queue.ts
reading concepts 2/2: src/rate.ts
writing concepts 1/1: batch 1
400 Thinking mode does not support this tool_choice

DeepSeek v4 reasons by default, and in that mode it accepts only tool_choice: "auto". Probing the API directly with a single tool offered:

request	result
tool_choice: {type: "function", function: {name: …}}	400 Thinking mode does not support this tool_choice
tool_choice: "required"	400 Thinking mode does not support this tool_choice
tool_choice: "auto"	200, tool called
tool_choice: "required" + reasoning_effort: "none"	200, tool called
tool_choice: "required" + thinking: {type: "disabled"}	200, tool called

So this is the same underlying incompatibility as the third item in the original report — function tools refused while reasoning is on — and it accepts the identical reasoning_effort: "none" remedy that OpenAI's own error message names. It just says so in different words.

Why the existing fallback makes it worse

isRejectedObjectToolChoice matches on /tool_choice/i, and DeepSeek's message contains tool_choice. So the object-form fallback matches first and rewrites the request to tool_choice: "required" — which thinking mode also rejects — and the build fails anyway, one wasted round-trip later.

Ordering matters beyond just this bug: degrading to "auto" would "work" in the sense of returning 200, but "auto" lets the model decline to call the tool at all. Since fmt.kind === "json" uses a forced synthetic tool as graft's structured-output mechanism, a model that opts out returns prose and the node write fails downstream. Turning reasoning off preserves the forced call; loosening tool_choice does not.

The fix is to test the reasoning refusal before the object-tool_choice fallback, and to match both providers' phrasings in the same predicate.

Complete patch

This replaces the diff in the issue body — it is the whole change, all four negotiations, against 65a76e5:

diff --git a/src/ai/llm/openai.ts b/src/ai/llm/openai.ts
--- a/src/ai/llm/openai.ts
+++ b/src/ai/llm/openai.ts
@@ -85,6 +85,51 @@ function isRejectedObjectToolChoice(err: unknown): boolean {
   );
 }
 
+/**
+ * Newer reasoning-family models (o1/o3/o4, the gpt-5.x line, …) reject the
+ * classic `max_tokens` param outright and require `max_completion_tokens`
+ * instead, even though both are still in wide use across OpenAI-compatible
+ * servers. Detect the specific 400 rather than guessing from the model name,
+ * so older endpoints that still expect `max_tokens` are untouched.
+ */
+function isRejectedMaxTokens(err: unknown): boolean {
+  return (
+    err instanceof OpenAI.APIError &&
+    err.status === 400 &&
+    /max_tokens.*not supported.*max_completion_tokens/i.test(String((err as { message?: string }).message ?? ""))
+  );
+}
+
+/** Same reasoning-family models: temperature is fixed at 1, not caller-settable. */
+function isRejectedTemperature(err: unknown): boolean {
+  return (
+    err instanceof OpenAI.APIError &&
+    err.status === 400 &&
+    /temperature.*does not support/i.test(String((err as { message?: string }).message ?? ""))
+  );
+}
+
+/**
+ * Same models again: function tools are rejected on /v1/chat/completions while
+ * the model's default reasoning effort is active. The API's own error message
+ * names the fix — `reasoning_effort: "none"` — so apply exactly that.
+ *
+ * DeepSeek's v4 line refuses the same combination ("Thinking mode does not
+ * support this tool_choice") for every tool_choice except "auto", and accepts
+ * the identical `reasoning_effort: "none"` remedy. Matching both phrasings here
+ * keeps a forced tool_choice working rather than degrading it to "auto", which
+ * would leave the model free not to call the tool the caller asked for.
+ */
+function isRejectedToolsWithReasoning(err: unknown): boolean {
+  const message = String((err as { message?: string }).message ?? "");
+  return (
+    err instanceof OpenAI.APIError &&
+    err.status === 400 &&
+    (/function tools with reasoning_effort/i.test(message) ||
+      /thinking mode does not support this tool_choice/i.test(message))
+  );
+}
+
 export class OpenAIChatModel implements ChatModel {
   readonly label: string;
   private client: OpenAI;
@@ -141,14 +186,37 @@ export class OpenAIChatModel implements ChatModel {
    * automatically.
    */
   private async createChatCompletion(params: ChatParams): Promise<OpenAI.Chat.Completions.ChatCompletion> {
-    try {
-      return await this.client.chat.completions.create(params);
-    } catch (err) {
-      if (isRejectedObjectToolChoice(err) && typeof params.tool_choice === "object" && params.tools?.length === 1) {
-        return this.client.chat.completions.create({ ...params, tool_choice: "required" });
+    let attempt = params;
+    // Bounded: one retry per known incompatibility below, never an open loop.
+    for (let i = 0; i < 4; i++) {
+      try {
+        return await this.client.chat.completions.create(attempt);
+      } catch (err) {
+        // Checked before the tool_choice fallback below: a reasoning refusal
+        // also names tool_choice, and turning reasoning off keeps the caller's
+        // chosen tool instead of loosening the choice to work around it.
+        if (isRejectedToolsWithReasoning(err) && attempt.reasoning_effort === undefined) {
+          attempt = { ...attempt, reasoning_effort: "none" } as ChatParams;
+          continue;
+        }
+        if (isRejectedObjectToolChoice(err) && typeof attempt.tool_choice === "object" && attempt.tools?.length === 1) {
+          attempt = { ...attempt, tool_choice: "required" };
+          continue;
+        }
+        if (isRejectedMaxTokens(err) && attempt.max_tokens !== undefined) {
+          const { max_tokens, ...rest } = attempt;
+          attempt = { ...rest, max_completion_tokens: max_tokens } as ChatParams;
+          continue;
+        }
+        if (isRejectedTemperature(err) && attempt.temperature !== undefined) {
+          const { temperature, ...rest } = attempt;
+          attempt = rest as ChatParams;
+          continue;
+        }
+        throw err;
       }
-      throw err;
     }
+    return this.client.chat.completions.create(attempt);
   }
 
   private fromResponse(

Notes on the shape, in case you'd rather solve it differently:

• Every branch is gated on the parameter it removes still being present, and the reasoning branch is gated on reasoning_effort being unset, so no branch can fire twice. The loop bound is 4 — one per known incompatibility — and the trailing call outside the loop exists only so a genuine error still surfaces to the caller rather than being swallowed by loop exhaustion.
• thinking: {type: "disabled"} also works on DeepSeek, but reasoning_effort: "none" is the OpenAI-spec spelling and both providers accept it, so one remedy covers both.
• The detectors deliberately match error text rather than model ids. That's brittle if a provider rewords a message, but the alternative — a model-name allowlist — goes stale every time a provider ships a model, and silently sends the wrong shape to anything unrecognised. A wording change degrades to today's behaviour (a surfaced 400); a stale allowlist degrades to a broken build with no clue why.

Verified

Both providers, end to end, on a repo that previously failed at the first LLM call:

• gpt-5.6 — build --deep completes.
• deepseek-v4-flash — build --deep completes; concept nodes and links written, tsc --noEmit clean.

Happy to open this as a PR if the approach looks right to you — say the word and I'll send it with tests for the four detectors.

Unrelated, same run: concurrency vs. TPM limits

Also worth a separate issue if you agree it's one. The default -j 5 on a tier-1/2 OpenAI key (200k TPM) trips 429s within the first few seconds of a --deep build on a mid-size repo. Because graft stops after five consecutive provider failures, the whole build aborts rather than backing off, and the surfaced message is about consecutive failures rather than rate limiting — so the obvious next move (retry) fails the same way. Dropping to -j 2 fixed it here. Two things would have saved the time: treating 429 as backoff-and-retry rather than as a failure toward the circuit-breaker count, and naming the rate limit in the abort message.