Memoize ToolSearch token counts

Author: SqlRush

docs/cc-100-roadmap.md

@@ -199,7 +199,7 @@ M8/M6 补充：compact plan 现在会把 compact 前已发现的 `tool_reference
 
 M8/M2 补充：当 `ToolSearch` 可用且存在 deferred 工具时，conversation request 现在会按官方 dynamic tool loading 过滤请求工具：未发现 deferred 工具不发送 schema，已发现 deferred 工具作为 loaded tool 发送，`ToolSearch` 保持可调用，并在 API messages 前置 `<available-deferred-tools>` 名称列表；没有 deferred 工具时会从请求中移除 `ToolSearch`。
 
-M8/M2 补充：conversation request 现在会在发送 beta `defer_loading` / `tool_reference` shape 前执行官方 ToolSearch enablement gate：Haiku 模型、`CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS`、falsy `ENABLE_TOOL_SEARCH`、`ENABLE_TOOL_SEARCH=auto:100`、未显式启用 ToolSearch 时的非一方 `ANTHROPIC_BASE_URL`、以及 `ENABLE_TOOL_SEARCH=auto` 下 deferred 工具描述体积低于阈值都会回落为标准 inline tool schema；显式 `ENABLE_TOOL_SEARCH=true` 仍可让支持 beta shape 的自定义网关 opt in。auto 模式已接入 Anthropic `/v1/messages/count_tokens` 精确计数路径，成功时减去官方固定 tool overhead 后按 token 阈值判断，失败时会用 Haiku message-create usage 作为官方第二层 fallback，再失败才按官方字符 fallback 计算阈值，覆盖 `auto:N` 百分比、`[1m]` context 和 ant-only context-window override；token-count VCR/cache memoization 仍未宣称完成。
+M8/M2 补充：conversation request 现在会在发送 beta `defer_loading` / `tool_reference` shape 前执行官方 ToolSearch enablement gate：Haiku 模型、`CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS`、falsy `ENABLE_TOOL_SEARCH`、`ENABLE_TOOL_SEARCH=auto:100`、未显式启用 ToolSearch 时的非一方 `ANTHROPIC_BASE_URL`、以及 `ENABLE_TOOL_SEARCH=auto` 下 deferred 工具描述体积低于阈值都会回落为标准 inline tool schema；显式 `ENABLE_TOOL_SEARCH=true` 仍可让支持 beta shape 的自定义网关 opt in。auto 模式已接入 Anthropic `/v1/messages/count_tokens` 精确计数路径，成功时减去官方固定 tool overhead 后按 token 阈值判断，失败时会用 Haiku message-create usage 作为官方第二层 fallback，再失败才按官方字符 fallback 计算阈值，覆盖 `auto:N` 百分比、`[1m]` context 和 ant-only context-window override；deferred tool token count 现在按官方 deferred tool name 列表做 in-process memoization，成功值和不可用结果都会缓存。token-count VCR 以及更完整的 MCP 生命周期失效策略仍未宣称完成。
 
 M8/M2 补充：当本次 request 未启用 `ToolSearch` 时，conversation request 现在会从 API user `tool_result.content` 中剥离历史 `tool_reference` blocks；纯 reference 结果会替换为官方占位文本 `[Tool references removed - tool search not enabled]`，剥离发生在 discovered-tool 扫描之后，避免影响后续 loaded 工具恢复。
 

docs/claude-code-go-rewrite-plan.md

@@ -294,7 +294,7 @@ test/parity/                 # golden tests against TS/official behavior
 - 本轮补充：conversation `BuildRequest` 会扫描历史 `tool_result.content` 中的 `tool_reference`，并把已发现工具在后续 API request 中作为 loaded tool 发送，不再携带 `defer_loading`；扫描兼容运行时 `ToolReference` 值和 transcript/JSON 解码后的 map 形态。完整官方 tool-reference expansion/filtering 仍未宣称完成。
 - 本轮补充：compact plan 会把 compact 前已发现的 `tool_reference` 名称快照进 `compactMetadata.preCompactDiscoveredTools`，session transcript alias/resume 转换会保留该 metadata，conversation `BuildRequest` 可在 tool-result 消息被 summary 替换后继续从 compact boundary 恢复已发现工具并取消 `defer_loading`。完整官方 compact/snipping 边界策略仍未宣称完成。
 - 本轮补充：当 request registry 同时存在 `ToolSearch` 和 deferred 工具时，conversation request 会按官方 dynamic tool loading 过滤工具列表：未发现 deferred 工具不再发送 schema，已发现 deferred 工具作为 loaded tool 发送，`ToolSearch` 保持可调用，并在首条 API user message 前注入 `<available-deferred-tools>` 名称列表；没有 deferred 工具时会从请求中移除 `ToolSearch`。
-- 本轮补充：conversation request 现在会在发送 beta `defer_loading` / `tool_reference` shape 前执行官方 ToolSearch enablement gate：Haiku 模型、`CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS`、falsy `ENABLE_TOOL_SEARCH`、`ENABLE_TOOL_SEARCH=auto:100`、未显式启用 ToolSearch 时的非一方 `ANTHROPIC_BASE_URL`、以及 `ENABLE_TOOL_SEARCH=auto` 下 deferred 工具描述体积低于阈值都会回落为标准 inline tool schema；显式 `ENABLE_TOOL_SEARCH=true` 仍可让支持 beta shape 的自定义网关 opt in。auto 模式已接入 Anthropic `/v1/messages/count_tokens` 精确计数路径，成功时减去官方固定 tool overhead 后按 token 阈值判断，失败时会用 Haiku message-create usage 作为官方第二层 fallback，再失败才按官方字符 fallback 计算阈值，覆盖 `auto:N` 百分比、`[1m]` context 和 ant-only context-window override；token-count VCR/cache memoization 仍未宣称完成。
+- 本轮补充：conversation request 现在会在发送 beta `defer_loading` / `tool_reference` shape 前执行官方 ToolSearch enablement gate：Haiku 模型、`CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS`、falsy `ENABLE_TOOL_SEARCH`、`ENABLE_TOOL_SEARCH=auto:100`、未显式启用 ToolSearch 时的非一方 `ANTHROPIC_BASE_URL`、以及 `ENABLE_TOOL_SEARCH=auto` 下 deferred 工具描述体积低于阈值都会回落为标准 inline tool schema；显式 `ENABLE_TOOL_SEARCH=true` 仍可让支持 beta shape 的自定义网关 opt in。auto 模式已接入 Anthropic `/v1/messages/count_tokens` 精确计数路径，成功时减去官方固定 tool overhead 后按 token 阈值判断，失败时会用 Haiku message-create usage 作为官方第二层 fallback，再失败才按官方字符 fallback 计算阈值，覆盖 `auto:N` 百分比、`[1m]` context 和 ant-only context-window override；deferred tool token count 现在按官方 deferred tool name 列表做 in-process memoization，成功值和不可用结果都会缓存。token-count VCR 以及更完整的 MCP 生命周期失效策略仍未宣称完成。
 - 本轮补充：当本次 request 未启用 `ToolSearch` 时，API message normalization 会从历史 user `tool_result.content` 中剥离 `tool_reference` blocks；纯 reference 结果会替换成官方占位文本 `[Tool references removed - tool search not enabled]`，但剥离发生在 discovered-tool 扫描之后，不影响后续启用 ToolSearch 时恢复 loaded 工具。
 - 本轮补充：Anthropic request tool 转换会保留 contract 的 `strict`、`eager_input_streaming`、`cache_control` 与 `should_defer`，把 deferred 工具序列化为 API `defer_loading`，并让 `always_load` 覆盖 deferred hint；API tool description 会按 description、prompt、searchHint 顺序 fallback，conversation runner 构造请求时会保留 `Task` 等 deferred tool 的 strict/defer_loading 元数据。完整 deferred/lazy tool discovery 仍未宣称完成。
 - 本轮补充：tool executor 会在未通过 `ToolSearch` 发现的 deferred 工具发生输入 schema 校验失败时追加 schema-not-sent 恢复提示，引导模型先调用 `ToolSearch` 的 `select:<tool>` 再重试；runner 会把当前 turn messages 传入工具 metadata，提示判断兼容 `tool_reference` 结果和 compact boundary 的已发现工具快照。

docs/first-second-parity-audit.md

@@ -93,6 +93,7 @@ Anthropic API 和 conversation：
 - Compact planning now snapshots discovered `tool_reference` names into `compactMetadata.preCompactDiscoveredTools`; resume/transcript conversion preserves that metadata and request construction reloads those tools after compacted `tool_result` messages have been summarized away.
 - When `ToolSearch` is available and deferred tools exist, request construction now follows dynamic tool loading: undiscovered deferred schemas are omitted from `tools`, discovered deferred tools are loaded, `ToolSearch` stays callable, and an `<available-deferred-tools>` user meta message advertises the deferred tool names; if no deferred tools exist, `ToolSearch` is omitted from the request.
 - Request construction now also applies the official ToolSearch enablement gates before emitting beta `defer_loading` / `tool_reference` shapes: Haiku models, `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS`, falsy `ENABLE_TOOL_SEARCH`, `ENABLE_TOOL_SEARCH=auto:100`, unset ToolSearch behind non-first-party `ANTHROPIC_BASE_URL`, and `ENABLE_TOOL_SEARCH=auto` requests below the deferred-tool size threshold all fall back to standard inline tool schemas; explicit `ENABLE_TOOL_SEARCH=true` can still opt a custom gateway back in. Auto mode now uses the Anthropic `/v1/messages/count_tokens` endpoint when the client supports it, subtracts the official fixed tool overhead, falls back to Haiku message-create usage when count-tokens fails, and then falls back to the official character threshold when token counting is unavailable, including `auto:N` percentages, 1M context suffixes, and ant-only context-window overrides.
+- Deferred tool token counts in ToolSearch auto mode are now memoized in-process by the official deferred tool name list key, including both successful counts and unavailable/null-equivalent results.
 - When ToolSearch is inactive for a request, API message normalization strips historical `tool_reference` blocks from user `tool_result` content and replaces pure-reference results with the official placeholder text, while preserving history scanning before stripping so discovered tools can still be loaded when ToolSearch is active.
 - Anthropic request tool conversion now preserves contract-level `strict`, `eager_input_streaming`, `cache_control`, and API-level `defer_loading` hints from deferred tool definitions, with `always_load` overriding `should_defer`; it also falls back from `description` to `prompt` to `searchHint` when building API tool descriptions. Runner request construction preserves these fields for tools such as `Task`.
 - Tool validation now appends an official-style schema-not-sent recovery hint when an undiscovered deferred tool is invoked with invalid typed input and `ToolSearch` is available, including compact-boundary discovered-tool metadata so already loaded tools do not get a false hint.
@@ -114,7 +115,7 @@ The following items remain incomplete and must not be treated as done:
 - Auto mode / YOLO classifier: transcript construction, two-stage classifier, XML/tool-use parsing, prompt dump, denial circuit breaker, model gating, and fallback behavior.
 - Interactive permission prompt flow: REPL dialogs, bridge/channel/swarm permission relays, user feedback images, prompt race handling, cancellation.
 - Full filesystem permission parity gaps that remain: skill-scope allow suggestions, remaining local/local-jsx slash-command execution wiring beyond the currently covered no-query builtins, command permission UI/SDK surfacing, forked/remote/MCP/bundled SkillTool behavior and remaining plugin SkillTool UI/SDK polish beyond current metadata preservation, skill prompt shell injection, complete auto-memory override policy, and deeper platform-specific Windows/WSL bypass handling.
-- Full tool execution parity gaps that remain: complete hook runtime policy beyond the current settings command hook path, MCP elicitation, complete SDK control event surface beyond current progress NDJSON, full deferred/lazy tool discovery beyond current local registry BM25 search, API `defer_loading` hints, model/env/proxy ToolSearch gates, remaining `ENABLE_TOOL_SEARCH=auto` edge behavior such as official token-count VCR/cache memoization, remaining delta-attachment details, mid-call cancellation for concrete tools, background task behavior, telemetry, and concrete tool-specific semantics.
+- Full tool execution parity gaps that remain: complete hook runtime policy beyond the current settings command hook path, MCP elicitation, complete SDK control event surface beyond current progress NDJSON, full deferred/lazy tool discovery beyond current local registry BM25 search, API `defer_loading` hints, model/env/proxy ToolSearch gates, remaining `ENABLE_TOOL_SEARCH=auto` edge behavior such as official token-count VCR and full MCP lifecycle invalidation details, remaining delta-attachment details, mid-call cancellation for concrete tools, background task behavior, telemetry, and concrete tool-specific semantics.
 - Complete Anthropic API parity gaps that remain: dynamic beta-header latching for any remaining provider-specific feature gates beyond prompt-cache/cache-editing/strict-output/1M-context requests, ant-only dump gating and remote `/issue` submission integration beyond the current local context summary, full official cost tracker/session restore lifecycle beyond current transcript usage restore, post-handshake streaming failure semantics, deeper gateway/proxy auth and routing behavior beyond static custom headers, first-party/Bedrock/Vertex/Foundry client setup, deeper OAuth account/profile lifecycle beyond request-time refresh retry, fast-mode retry/cooldown semantics, persistent unattended retry heartbeats, full prompt-cache editing lifecycle, and provider-specific cache behavior.
 - Full conversation/query loop: stop hooks, compact/auto-compact, token budget escalation, resume, SDK JSON/NDJSON control events, status updates, rate-limit handling, model switch breadcrumbs, side questions.
 - Full settings parity gaps that remain: complete Zod-equivalent validation messages, full remote managed-settings watcher/non-daemon background refresh lifecycle beyond current turn-start and daemon-heartbeat refresh, remaining forked agent/frontmatter execution and UI edge cases beyond current policy metadata filtering, complete marketplace TUI/UI/background lifecycle beyond current local/settings/directory/file/URL catalog/git/github/npm cache manifest load, URL catalog cache fallback, git/github fetch/pull cache refresh, npm pack cache refresh, and background watcher/continuous app-state sync beyond current turn-start local settings reload.

internal/conversation/request.go

@@ -8,6 +8,7 @@ import (
 	"sort"
 	"strconv"
 	"strings"
+	"sync"
 	"time"
 
 	"ccgo/internal/api/anthropic"
@@ -82,6 +83,13 @@ func (r Runner) buildRequest(ctx context.Context, history []contracts.Message, m
 
 type deferredToolTokenCounter func([]contracts.ToolDefinition) (int, bool)
 
+type deferredToolTokenCountCacheEntry struct {
+	Tokens int
+	OK     bool
+}
+
+var deferredToolTokenCountCache sync.Map
+
 func filterToolSearchDefinitions(definitions []contracts.ToolDefinition, history []contracts.Message, model string, tokenCounter deferredToolTokenCounter) ([]contracts.ToolDefinition, []string, bool) {
 	if len(definitions) == 0 {
 		return definitions, nil, false
@@ -151,21 +159,53 @@ func (r Runner) deferredToolTokenCounter(ctx context.Context, modelName string)
 		if len(tools) == 0 {
 			return 0, true
 		}
-		if counter, ok := r.Client.(TokenCountingMessageClient); ok && counter != nil {
-			response, err := counter.CountTokens(ctx, anthropic.CountTokensRequest{
-				Model:    modelName,
-				Messages: []contracts.APIMessage{{Role: "user", Content: []contracts.ContentBlock{contracts.NewTextBlock("foo")}}},
-				Tools:    tools,
-			})
-			if err == nil && response != nil && response.InputTokens > 0 {
-				return normalizeToolTokenCount(response.InputTokens), true
+		cacheKey := deferredToolTokenCountCacheKey(definitions)
+		if cacheKey != "" {
+			if cached, ok := deferredToolTokenCountCache.Load(cacheKey); ok {
+				entry := cached.(deferredToolTokenCountCacheEntry)
+				return entry.Tokens, entry.OK
 			}
 		}
-		if tokens, ok := r.countToolTokensViaHaikuFallback(ctx, tools); ok {
-			return tokens, true
+		tokens, ok := r.countDeferredToolTokens(ctx, modelName, tools)
+		if cacheKey != "" {
+			deferredToolTokenCountCache.Store(cacheKey, deferredToolTokenCountCacheEntry{Tokens: tokens, OK: ok})
+		}
+		return tokens, ok
+	}
+}
+
+func (r Runner) countDeferredToolTokens(ctx context.Context, modelName string, tools []anthropic.ToolDefinition) (int, bool) {
+	if len(tools) == 0 {
+		return 0, true
+	}
+	if counter, ok := r.Client.(TokenCountingMessageClient); ok && counter != nil {
+		response, err := counter.CountTokens(ctx, anthropic.CountTokensRequest{
+			Model:    modelName,
+			Messages: []contracts.APIMessage{{Role: "user", Content: []contracts.ContentBlock{contracts.NewTextBlock("foo")}}},
+			Tools:    tools,
+		})
+		if err == nil && response != nil && response.InputTokens > 0 {
+			return normalizeToolTokenCount(response.InputTokens), true
 		}
-		return 0, false
 	}
+	if tokens, ok := r.countToolTokensViaHaikuFallback(ctx, tools); ok {
+		return tokens, true
+	}
+	return 0, false
+}
+
+func deferredToolTokenCountCacheKey(definitions []contracts.ToolDefinition) string {
+	names := make([]string, 0, len(definitions))
+	for _, definition := range definitions {
+		if toolDefinitionDeferred(definition) {
+			names = append(names, definition.Name)
+		}
+	}
+	return strings.Join(names, ",")
+}
+
+func resetDeferredToolTokenCountCache() {
+	deferredToolTokenCountCache = sync.Map{}
 }
 
 func (r Runner) countToolTokensViaHaikuFallback(ctx context.Context, tools []anthropic.ToolDefinition) (int, bool) {