Skip to content

ToolSearch from deferred tool loading is not blocked by disallowed_tools, and costs ~10x on single-tool sessions #1147

Description

@MarcellZamboHu

Summary

When a session exposes exactly one in-process MCP tool and lists the built-in tools in disallowed_tools, the CLI still loads a built-in ToolSearch meta-tool through deferred tool loading. The model then calls ToolSearch to discover the single MCP tool that was already named in allowed_tools. This costs an extra API round trip and about 10x the total cost of the identical task run with tools=[].

ToolSearch is not in the public built-in tool list, so a user writing a denylist has no way to know they should name it.

Environment

  • claude-agent-sdk 0.2.128 (latest on PyPI at time of writing)
  • bundled Claude Code CLI 2.1.220
  • Python 3.12.3, Linux x86_64
  • model: claude-sonnet-5

Reproduction

import anyio
from claude_agent_sdk import (ClaudeAgentOptions, ResultMessage, create_sdk_mcp_server,
                              query, tool)

BUILTINS = ["Bash", "Read", "Write", "Edit", "Glob", "Grep", "WebFetch", "WebSearch",
            "NotebookEdit", "Task", "TodoWrite", "BashOutput", "KillShell", "SlashCommand"]


@tool("get_secret_number", "Returns a secret number for demonstration.", {"seed": str})
async def get_secret_number(args: dict) -> dict:
    return {"content": [{"type": "text", "text": f"secret={len(args['seed'])}"}]}


server = create_sdk_mcp_server(name="demo", version="1.0.0", tools=[get_secret_number])
FULL = "mcp__demo__get_secret_number"


async def run(zero_base: bool) -> None:
    kwargs = dict(
        model="claude-sonnet-5",
        system_prompt="Answer in one sentence. Use the demo tool when asked.",
        mcp_servers={"demo": server},
        allowed_tools=[FULL],
        disallowed_tools=BUILTINS,
        setting_sources=[],
        permission_mode="bypassPermissions",
        max_turns=5,
    )
    if zero_base:
        kwargs["tools"] = []                     # the only difference between the variants
    async for msg in query(prompt="Call the demo tool with seed 'demo', then summarise.",
                           options=ClaudeAgentOptions(**kwargs)):
        for block in getattr(msg, "content", []) or []:
            if getattr(block, "type", None) == "tool_use":
                print("tool_use:", block.name)
        if isinstance(msg, ResultMessage):
            print("turns:", msg.num_turns, "cost_usd:", msg.total_cost_usd, msg.usage)


anyio.run(run, False)     # variant A - denylist only
anyio.run(run, True)      # variant B - zero-base allowlist

Observed

Variant A (disallowed_tools only):

tool_use: ToolSearch          <- not requested, not in the public tool list
tool_use: mcp__demo__get_secret_number
turns: 3   cost_usd: 0.0683255   cache_creation_input_tokens: 14774
elapsed: 7.79 s

Variant B (identical, plus tools=[]):

tool_use: mcp__demo__get_secret_number
turns: 2   cost_usd: 0.00712225
usage: input_tokens=4, cache_creation_input_tokens=1235,
       cache_read_input_tokens=1140, output_tokens=100
elapsed: 4.82 s

Same task, same model, same tool: 0.0683 USD vs 0.0071 USD, a factor of 9.6, almost entirely cache-write tokens spent on loading a tool-discovery mechanism for a session that had one tool.

Expected

Any one of these would resolve it:

  1. ToolSearch is suppressible by name through disallowed_tools, like other built-ins.
  2. Deferred tool loading does not engage when the session's total tool count is trivially small — discovering one tool cannot pay for itself.
  3. The documentation states plainly that disallowed_tools is a denylist and is not sufficient for isolation, and that tools=[] is the way to restrict what is available.

Why this matters beyond cost

Guides and examples commonly show disallowed_tools as the way to constrain a session. It is a denylist, so any built-in not named — including one absent from the published tool list — stays reachable. The type stub does say the right thing (types.py:1770-1771: "To restrict which tools are available at all, use tools"), but that line lives in the stub, and the option named allowed_tools reads as though it were the restriction. For anyone building a constrained agent, the difference between "these run without prompting" and "these are the only ones that exist" is the whole design.

The related default is worth a docs line too: setting_sources=None loads every filesystem source, including the operator's ~/.claude. For a headless service that must be deterministic, setting_sources=[] has to be set explicitly. (#794 and #802 fixed the case where the empty list was ignored; this is about the default itself.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions