feat(rlm): split tools into sandbox_tools and native_tools - #95
Merged
Conversation
RLM exposed every user tool inside the sandbox: bound into the snippet namespace and advertised in the prompt's tools catalog, while the provider's own tool schema carried only `run_python_code`. That is the right placement for a tool a snippet *composes* with — one whose result feeds the next line of Python — and the wrong one for a tool the LM only wants to *consult*, whose answer shapes the snippet it is about to write. Forcing the latter through the sandbox costs a round-trip through generated code, and the catalog advertises names that are not natively callable, which models try anyway and get `Unknown tool` back. Add `native_tools=` for tools the LM calls directly, alongside `run_python_code`, and `sandbox_tools=` as an explicit spelling of the existing behaviour. Native tools reuse the dispatch path the subagent tools already take, so they land in the provider schema, in `_dispatch_tool_calls`, and in the `Unknown tool` message's list of what is callable — while staying out of the sandbox namespace and the catalog. `tools=` is unchanged and still means sandbox tools, so existing agents keep their behaviour; it is concatenated with `sandbox_tools=`. A name may not be in both halves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
RecursiveLanguageModelAgentexposes every user tool inside the sandbox: bound into the snippet namespace and advertised in the prompt's tools catalog, while the provider's own tool schema carries onlyrun_python_code.That is the right placement for a tool a snippet composes with — one whose result feeds the next line of Python. It is the wrong one for a tool the LM only wants to consult: a lookup whose answer shapes the snippet it is about to write. Forcing that through the sandbox costs a round-trip through generated code just to read an answer.
It also has a concrete failure mode. The catalog presents sandbox tools as a
toolslist with names, descriptions and JSON-schema parameters — which looks exactly like a tool-call spec — so models emit native tool calls for them and get:Observed 24 times in a single agent task on Qwen3.8-27B, and in every task of a 19-task run.
What
native_tools=— tools the LM calls directly, alongsiderun_python_code.sandbox_tools=— an explicit spelling of the existing behaviour, for agents that have some of each and wheretools=would read ambiguously.Native tools reuse the dispatch path the subagent tools already take (
extra_native_tools), so they land in the provider schema, in_dispatch_tool_calls, and in theUnknown toolmessage's list of what is callable — while staying out of the sandbox namespace and out of the catalog.Compatibility
tools=is unchanged: it still means sandbox tools, so existing agents keep their behaviour. It is concatenated withsandbox_tools=. A name may not appear in both halves, and native tools get the same reserved-name and public-name checks as sandbox tools.Tests
Added to
rlm_agent_test.py:test_tools_still_means_sandbox_tools— back-compat pin:tools=lands in the sandbox half, and an agent that never mentionsnative_toolshas none.test_tools_and_sandbox_tools_are_concatenatedtest_sandbox_tools_is_a_spelling_of_toolstest_native_tools_are_kept_out_of_the_sandbox_settest_native_and_sandbox_tools_coexisttest_same_name_in_both_halves_rejectedtest_reserved_names_rejected_for_native_tools_tootest_native_tool_is_dispatched_not_rejected— end-to-end: the LM emits a native tool call and gets the tool's result back.test_sandbox_tool_called_natively_is_still_rejected— the other half of the contract.synalinks/src/modules/agents/: 195 passed before, 197 after (plus the constructor tests).🤖 Generated with Claude Code