feat: Built-in fetch tool#399
Draft
andrii-novikov wants to merge 5 commits into
Draft
Conversation
andrii-novikov
changed the title
Add design doc for `internal_file_fetch`, a self-contained internal tool that fetches a web URL. A `save` flag selects between loading content inline (default) and persisting it as a workspace DIAL file for the existing file tools. Reuses ExternalUrlFetcher and the two-tier egress policy; independent of the large-tool-response offload processor.
Split the feature into two dedicated tools after review: - internal_web_fetch: load a text resource inline (internal_tooling module, gated by features.web_fetch.enabled, configurable max_inline_size). - internal_file_download: persist any resource into the agent-home workspace via the file-family write path (gated by features.dial_files.enabled_tools). Both feature-gated and preview-gated. Reuses ExternalUrlFetcher + the two-tier egress policy; independent of the offload processor. Status: Approved.
Move internal_web_fetch out of the deprecated internal_tooling module into a new self-contained, feature-gated web_tooling module. Drop module-name references to other in-flight work and note the prerequisite PRs (#368, #393) that merge first: the file-family subclass API (_resolve_appdata_url / _to_display_path) that internal_file_download relies on is preserved by #368, and edits to the shared wiring files are additive alongside both PRs.
Collapse the two-tool proposal (internal_web_fetch + internal_file_download) into a single internal_web_fetch(url, save_path=None): omit save_path to read text inline, provide it to persist under the agent home. Approved.
Implement the approved one-tool design: internal_web_fetch(url, save_path). Omit save_path to return the fetched text inline (textual, size-guarded); provide it to persist the resource under the agent home via the shared HomePathResolver + DialFileService.write_bytes, returning the workspace path. - Promote HomePathResolver to shared/home_path (HomePathModule in shared_module), injected by both the file tools and the web tool. - Remove the superseded internal_file_download tool and its dial_files wiring. - Gate the tool on external egress at init: when web_fetch is enabled but egress is disabled, the tool is not exposed and a hard ToolInitializationException is surfaced, instead of failing on every call. - Reject files/-prefixed save_path; uniquify on collision (overwrite=False). - Regenerate schemas; dump_internal_tools enables egress so the tool materializes.
andrii-novikov
force-pushed
the
feat/344-web-fetch-tool
branch
from
July 3, 2026 14:18
ecf65aa to
270b548
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Applicable issues
Description of changes
Adds the design doc for a built-in web fetch capability, split after review into two dedicated internal tools:
internal_web_fetch— loads a text resource inline (internal_toolingmodule), gated byfeatures.web_fetch.enabled, with a configurablemax_inline_size.internal_file_download— persists any resource into the agent-home workspace via the existing file-family write path, gated byfeatures.dial_files.enabled_tools.Both tools are feature-gated and preview-gated, and reuse
ExternalUrlFetcherplus the existing two-tier external-egress policy — independent of the large-tool-response offload processor. Status: Approved.Checklist
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.