Skip to content

fix(agents): tolerate unparsable tool call arguments #4339

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BlueX888 wants to merge 2 commits into
base: master
Choose a base branch
from
+82 −1
Open

fix(agents): tolerate unparsable tool call arguments #4339

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5b1cded
fix(agents): tolerate unparsable tool call arguments
BlueX888 Sep 16, 2026
e97d759
fix(agents): treat non-object tool call arguments as empty
BlueX888 Sep 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion camel/agents/chat_agent.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4036,7 +4036,16 @@ def _handle_batch_response(
for tool_call in tool_calls:
tool_name = tool_call.function.name # type: ignore[union-attr]
tool_call_id = tool_call.id
args = json.loads(tool_call.function.arguments) # type: ignore[union-attr]
try:
args = json.loads(tool_call.function.arguments) # type: ignore[union-attr]
except json.JSONDecodeError:
# `arguments` may be empty or truncated on the wire.
args = {}

@ebarkhordar ebarkhordar Sep 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran this against 5b1cded65d7acda268599fb434f93ae55a4f8822 in a clean container (python:3.12-slim, pip install -e .).

The guard catches a decode error, but arguments is only required to be a string, not an object. A value that parses as valid JSON that is not a dict skips the except and reaches ToolCallRequest(args=args) at :4046-4048, where pydantic raises instead, so it leaves step() the same way the decode error did.

Your test script, same ping tool, varying only the arguments string, at the PR head:

empty string (your case 1)  -> OK   args={} result='pong'
truncated (your case 2)     -> OK   args={} result='pong'
valid JSON null             -> ValidationError out of step()
valid JSON list             -> ValidationError out of step()
valid JSON scalar 123       -> ValidationError out of step()
valid JSON string "ping"    -> ValidationError out of step()
normal object {}            -> OK   args={} result='pong'

The last four behave identically with your change reverted, so the PR does not introduce this.

It does bear on the consistency point in the description. I called _execute_tool_from_stream_data directly with those same five strings, not through a real stream, and it returned a record every time and logged the failure rather than raising. So for this input class the two paths still disagree.

One line if you want it in scope:

if not isinstance(args, dict):
    args = {}

I have not seen a provider send arguments: "null", so that part is argued from the spec wording you quote rather than observed.

@BlueX888 BlueX888 Sep 17, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed, and fixed in e97d759.

I re-ran your matrix from this PR's own test harness (ChatAgent.step() driven by a stub model), at the head you tested:

empty string           -> OK   args={}                result='pong'
truncated              -> OK   args={}                result='pong'
valid JSON null        -> ValidationError out of step()
valid JSON list        -> ValidationError out of step()
valid JSON list [1, 2] -> ValidationError out of step()
valid JSON number 123  -> ValidationError out of step()
valid JSON true        -> ValidationError out of step()
valid JSON "ping"      -> ValidationError out of step()

I also measured the streaming side rather than assuming it, calling _execute_tool_from_stream_data with the same five strings: it returns without raising in every case (null, [], 123, "ping" and "" all come back as a None record after the logged failure). So the two paths disagreed exactly where you said they did.

The guard now covers both classes, since neither payload can fill ToolCallRequest.args:

try:
    args = json.loads(tool_call.function.arguments)
except json.JSONDecodeError:
    # `arguments` may be empty or truncated on the wire.
    args = {}
if not isinstance(args, dict):
    # `arguments` may also be valid JSON that is not an
    # object (`null`, `[]`, a bare scalar), which `args`
    # cannot hold any more than a malformed payload.
    args = {}

The regression test is now parametrized over ["", '{"city": "San Fra', "null", "[]", "123", '"ping"'], so both the JSONDecodeError path and the non-object path are pinned. test/agents/test_chat_agent.py is 40 passed / 4 failed on this head, with the same 4 failures on master (they need live API credentials), and ruff reports the same three pre-existing RUF059 findings on master and here.

For the record, I agree with the caveat you flagged: I have not seen a provider send arguments: "null" either, so this is argued from the contract rather than observed. The reason I took the one-liner is that the failure mode is asymmetric - a stray non-object payload costs the whole step() today, and the streaming path already made the other choice.

@ebarkhordar ebarkhordar Sep 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that closes it. At e97d759 ToolCallRequest is built in exactly one place, chat_agent.py:4051, inside the function you guarded, so there is no second path left where a non-object payload can reach args.

if not isinstance(args, dict):
# `arguments` may also be valid JSON that is not an
# object (`null`, `[]`, a bare scalar), which `args`
# cannot hold any more than a malformed payload.
args = {}
extra_content = getattr(tool_call, 'extra_content', None)

tool_call_request = ToolCallRequest(
Expand Down
72 changes: 72 additions & 0 deletions test/agents/test_chat_agent.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
BaseModelBackend,
ModelFactory,
OpenAIModel,
StubModel,
)
from camel.terminators import ResponseWordsTerminator
from camel.toolkits import (
Expand Down Expand Up @@ -2371,3 +2372,74 @@ def test_rate_limit_retry_respects_anthropic_error_when_installed():
from openai import RateLimitError as OpenAIRateLimitError

assert _RATE_LIMIT_ERRORS == (OpenAIRateLimitError,)


@pytest.mark.parametrize(
"arguments",
["", '{"city": "San Fra', "null", "[]", "123", '"ping"'],
ids=["empty", "truncated", "null", "list", "number", "string"],
)
def test_chat_agent_step_tolerates_unusable_tool_arguments(arguments):
r"""A tool call whose `arguments` string cannot fill `args` is executed.

OpenAI-compatible servers (vLLM, Ollama, Together, ...) send
`arguments: ""` for zero-argument tools, and arguments can be truncated
on the wire. A payload may also be well-formed JSON that is not an
object (`null`, `[]`, a bare scalar), which `ToolCallRequest.args` cannot
hold either. The streaming path lets none of them escape, so the
non-streaming path must not either.
"""

def ping() -> str:
r"""A zero-argument tool."""
return "pong"

model_backend_rsp_tool = ChatCompletion(
id='mock_id_123456',
choices=[
Choice(
finish_reason='tool_calls',
index=0,
logprobs=None,
message=ChatCompletionMessage(
content=None,
role='assistant',
tool_calls=[
ChatCompletionMessageFunctionToolCall(
id='call_mock_123456',
function=Function(
arguments=arguments,
name='ping',
),
type='function',
),
],
),
)
],
created=1730752528,
model='stub',
object='chat.completion',
usage=CompletionUsage(
completion_tokens=5,
prompt_tokens=10,
total_tokens=15,
),
)

model = StubModel(ModelType.STUB, model_config_dict={"stream": False})
agent = ChatAgent(
system_message="You are a help assistant.",
model=model,
tools=[ping],
max_iteration=1,
)
agent.model_backend.run = MagicMock(return_value=model_backend_rsp_tool)

response = agent.step("Ping the tool.")

tool_calls = response.info['tool_calls']
assert len(tool_calls) == 1
assert tool_calls[0].tool_name == 'ping'
assert tool_calls[0].args == {}
assert tool_calls[0].result == 'pong'
Loading