Skip to content

fix: model finishing with no tool calls gets misreported as a decode error - #1348

Open
zambalee wants to merge 1 commit into
ShishirPatil:mainfrom
zambalee:fix/fc-empty-tool-calls-misclassified-as-decode-error
Open

fix: model finishing with no tool calls gets misreported as a decode error#1348
zambalee wants to merge 1 commit into
ShishirPatil:mainfrom
zambalee:fix/fc-empty-tool-calls-misclassified-as-decode-error

Conversation

@zambalee

@zambalee zambalee commented Jul 18, 2026

Copy link
Copy Markdown

What

OpenAICompletionsHandler._parse_query_response_FC() — and the same
copy-pasted pattern in MistralHandler and FireworksHandler
wraps tool_call extraction in a bare except::

try:
    model_responses = [
        {func_call.function.name: func_call.function.arguments}
        for func_call in api_response.choices[0].message.tool_calls
    ]
    ...
except:
    model_responses = api_response.choices[0].message.content
    ...

When a model has legitimately finished a multi-turn task and replies
with plain text (no further tool calls needed — completely valid
behavior), tool_calls is None, the list comprehension raises
TypeError: 'NoneType' object is not iterable, and the bare except
falls back to treating the raw text content as model_responses.

That string is then passed to decode_execute()
convert_to_function_call(), which assumes a list[dict]. Python
happily iterates a string character-by-character, and calling
.items() on each single-character string raises:

AttributeError: 'str' object has no attribute 'items'

which gets caught by the outer handler and logged as
"Failed to decode the model response" — indistinguishable from an
actual model/serving failure, even though the model did nothing
wrong.

Impact

Found while running BFCL v4 multi_turn locally across three
different models on three different backends/hardware:

Model Model Card Serving Engine Hardware
Devstral-Small-2-24B-Instruct-2512-AWQ-4bit HF vLLM 0.24.0 (bare-metal / venv) 1x modded RTX 4090 48GB (PCIe passthrough)
Qwen3-Coder-30B-A3B-Instruct-AWQ HF LMDeploy 0.14.0 (Docker, openmmlab/lmdeploy:latest) 4x Tesla V100-SXM2-16GB
Qwen3.6-27B-int4-AutoRound HF vLLM 0.24.0 (Docker, vllm/vllm-openai:v0.24.0) NVIDIA DGX Spark GB10

The 'str' object has no attribute 'items' error was the only
error type observed in the affected result files, at meaningful
volume across all three (different engines, different hardware):

Model Occurrences (4 multi_turn subcategories)
Devstral-Small-2-24B-Instruct-2512-AWQ-4bit 2530
Qwen3-Coder-30B-A3B-Instruct-AWQ 3284
Qwen3.6-27B-int4-AutoRound 116

Since this reproduces identically across two engines (vLLM, LMDeploy)
and three hardware platforms, it's clearly the harness's own bug, not
an engine/model/quantization issue. It likely understates Multi-Turn
accuracy for any FC-mode model evaluated through
OpenAICompletionsHandler, MistralHandler, or FireworksHandler
whenever the model correctly stops calling tools mid-conversation.

Fix

Check tool_calls explicitly instead of relying on exception
fallback. When there are none, return model_responses = [], which
is_empty_execute_response() already recognizes as "no further tool
calls" and handles as a normal turn-ending case — no exception, no
misreported error.

Testing

No existing unit tests cover this handler. Verified manually with a
mocked api_response:

  • tool_calls=None + plain-text contentmodel_responses == []
    (previously: raw string, crashing downstream)
  • Normal tool_calls present → output unchanged from before this fix

Scope note

CohereHandler already does this correctly (explicit
if len(tool_calls) > 0 check) and wasn't touched.
local_inference/minicpm_fc.py has a structurally different
(non-exception-based) fallback to raw content that may warrant a
separate look, but I didn't want to touch a different code path/model
family without dedicated testing — flagging here for maintainers'
awareness rather than bundling an unverified change into this PR.


Co-authored with Claude Code (Sonnet 5).

… error

OpenAICompletionsHandler._parse_query_response_FC() (and the same
copy-pasted pattern in MistralHandler / FireworksHandler) wraps the
tool_calls extraction in a bare `except:` and falls back to the raw
text `content` whenever `tool_calls` is falsy. This isn't only an
error path -- a model that has already finished the task and replies
in plain text (no further tool calls needed) hits the exact same
fallback.

That raw string then flows into decode_execute() ->
convert_to_function_call(), which assumes a list[dict] of function
calls. Iterating a string yields its characters, and calling
`.items()` on each one raises `'str' object has no attribute 'items'`
-- silently miscounted as "Failed to decode the model response" even
though the model behaved correctly.

Found while running BFCL v4 multi_turn locally: 100% of the
`'str' object has no attribute 'items'` errors across three different
models/backends traced back to this single fallback path, not to any
model or serving-engine issue.

Fix: check `tool_calls` explicitly and return `model_responses = []`
when there are none, matching what decode_execute/
is_empty_execute_response already expect for "no further tool calls."

Co-authored-by: Claude Code Sonnet-5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant