[Bug]: qvac serve openai accepts `tools` but never returns `tool_calls`, so models fabricate answers instead of calling tools

[victorchimakanu]

What happens

qvac serve openai accepts a tools array on POST /v1/chat/completions but never returns tool_calls. The request succeeds with finish_reason: "stop", no tool_calls field, and the model answers in prose as if no tools were offered. Because the model is never told the tools exist, it fabricates an answer instead of calling them.

The OpenAPI spec the CLI emits does declare both tools and tool_calls, so the capability is advertised.

Reproduction

Serve any chat model, then:

curl -s http://127.0.0.1:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model":"<model>",
    "messages":[{"role":"user","content":"What is the weather in Lagos?"}],
    "tools":[{"type":"function","function":{
      "name":"get_weather",
      "description":"Get current weather for a city",
      "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}
    }}],
    "max_tokens":250
  }'

Expected: finish_reason: "tool_calls" with a tool_calls entry for get_weather.

Actual: finish_reason: "stop", no tool_calls, and a fabricated answer:

"The current weather in Lagos, Nigeria, is as follows: Temperature: Approximately 30°C (86°F), Conditions: Sunny, Humidity: High, around 80% ..."

Passing tool_choice: "auto" changes nothing (and tool_choice is absent from the emitted spec). Same result on Qwen3-4B-Q4_K_M and Qwen3-8B-Q4_K_M.

Spec says otherwise

qvac openai spec > spec.json
grep -c tool_calls spec.json   # present
grep -c '"tools"' spec.json    # present
grep -c tool_choice spec.json  # absent

Why it matters

Silent fabrication is the worst failure mode here. Nothing errors, so a caller cannot tell the difference between a tool result and an invention. Driving a real agent harness through it, I got a confident list of three KYC applicants who do not exist, in an account whose real applicants are entirely different people.

It also blocks agent harnesses generally. I was connecting OpenCode to a local QVAC model through a skill (the @qvac/opencode-plugin path). OpenCode sends its tools correctly, serve drops them, and the model narrates tool use instead of performing it. The same harness against a tool-calling endpoint works fine, so the harness is not the problem.

Practically this is the gap between "QVAC can replace your local model server" and "QVAC can replace it for agentic work". Everything conversational is unaffected; anything that needs the model to call a tool is blocked today.

Ask

Either implement tools / tool_calls (plus tool_choice) in the chat completions adapter, or, until then, reject requests containing tools with a clear error rather than silently ignoring them. A loud failure is far safer than a fabricated answer.

Environment

• qvac CLI 0.9.0
• macOS 26.5.1, arm64
• Models: Qwen3-4B-Q4_K_M, Qwen3-8B-Q4_K_M

[localhost41]

I can take a narrow reproduction and serve-layer triage for this. I found the related open PR #4120, which addresses native tool grammar and sampling, but its description leaves the SDK/serve adapter follow-up outside that PR. My proposed plan is: reproduce the request on the current qvac serve OpenAI path, trace whether tools are dropped before generation or tool-call results are lost during response serialization, and add the smallest regression test or an explicit unsupported-tools error. I will avoid duplicating #4120; please confirm whether the serve-layer portion is already being handled under QVAC-23460 or elsewhere.

[victorchimakanu]

The public record answers part of it. #4120 describes itself as the addon half of QVAC-23460 and says the TypeScript follow-ups stay under that task. All 32 files it touches are inside packages/llm-llamacpp, so the serve adapter is untouched there. That means the serve half is tracked internally under QVAC-23460, and I do not see a public PR for it. I will confirm with the team whether anyone has started that half and report back here before you sink time into a fix.

The narrow reproduction and the trace are useful no matter who writes the fix. My report shows the behavior from the outside only. The model answers in prose, which tells us the tools never reached the prompt or the call was lost on the way back, but not where. A trace that names the exact spot sharpens this issue either way. Of the outcomes you listed, the explicit unsupported tools error is the one I would want first, because today the failure is silent while the emitted spec advertises the capability.

Hold the regression test and any fix until I hear back on QVAC-23460, so your work and the internal task do not collide.

[victorchimakanu]

Coming back to this as I said I would, and the answer changes the issue.

Serve does not drop tools. In the published @qvac/cli 0.13.0, the chat route turns the request's tools into SDK tools and passes them to completion(). It also maps any tool calls that come back onto tool_calls with finish_reason: "tool_calls". That mapping was already there in 0.9.0.

The drop happens one layer down, behind a load flag. The llama.cpp completion path only puts the tool definitions into the prompt when the model was loaded with tools: true (completion-stream.ts#L402-L403, same code in the @qvac/inference 0.19.1 that ships with @qvac/sdk 0.19.1):

const toolsEnabled = (modelConfig as { tools?: boolean }).tools === true
const toolsActive = !!tools?.length && toolsEnabled

With the flag off, the request's tools are skipped and the model never sees them. That is exactly what I reported. My serve config set only ctx_size for the model, so the flag was off. Serve passes each model's config block straight through as the SDK modelConfig, so on the serve side the setting is:

"config": { "ctx_size": 16384, "tools": true }

I have not rerun the original curl with the flag on yet. I will post that result here, including whether Qwen3 4B and 8B come back with a clean tool_calls entry.

So what is left on this issue is narrower than what I filed. The failure is silent. A request that carries tools for a model loaded without the flag gets a 200 and a normal answer, with no sign the tools were ignored. The HTTP server docs show "tools": true in the example config but do not say tool calls depend on it.

#4120 merged today. It adds the grammar that keeps a tool call well formed, plus tool_choice, in the addon. It is not in a published @qvac/llm-llamacpp yet, and @qvac/sdk 0.19.1 takes ^0.49.1, so it reaches serve users in a later SDK release.

@localhost41, this answers most of the serve trace you offered, and it took me too long to come back with it. The open question is how serve should respond when tools arrive for a model loaded without the flag: reject the request, warn, or something else. That is the team's call, and I am taking this finding to them now. Please keep holding code until I report their answer here.

[localhost41]

Thanks for tracing this to the model-load flag and narrowing the issue. I’ll hold implementation until the tools: true rerun confirms the expected tool_calls response, then keep any follow-up scoped to the documented serve/model configuration behavior.

[victorchimakanu]

Opanin has opened #4430 for the silent failure here. It adds a 400 tools_not_enabled response when a request includes tools but the model was loaded without config.tools: true, and updates the docs.

The PR is still open. @localhost41, the team is handling this part, so there is no need to start a separate fix. I’ll keep this issue open while it goes through review and release. The rerun with tools enabled is still pending.