Skip to content

BFCL multi-turn: endpoint-controlled tool-call names reach bfcl-eval eval() (RCE on evaluator host) #391

Description

@Palanivelg

From PR #346 review (bfcl_v4_execution.py:235).

Decoded tool-call strings (function name + arguments straight from the endpoint's tool_calls response) reach bfcl-eval's eval(func_call), guarded only by an upstream name denylist (kill/exit/quit/remove/unlink/popen/Popen/run). A tool call named __import__('os').system bypasses it (split("(")[0] yields __import__, not on the list), so a compromised or untrusted --endpoint gets arbitrary code execution on the evaluator host.

This is upstream bfcl-eval behavior, but this PR is what wires arbitrary-endpoint HTTP responses into it.

Options:

  • At minimum: document that multi-turn eval must only target trusted endpoints.
  • Better: validate tc["name"] against the entry's known tool names in process_response before handing calls to bfcl-eval.

Deferred out of PR #346 to keep it focused; the multi-turn live-execution path is opt-in (its own CLI).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions