mcp: tool calls in a turn execute serially — latency is the sum, not the max

[MitulShah1]

Follow-up to the MCP stdio transport shipped in v1.3.0.

Problem

Executor.ResolvePendingToolCalls runs a turn's tool calls one at a time, so latency is the sum rather than the max. Measured: 8 calls at 100 ms each → 802 ms.

v1.3.0 capped a turn at 64 calls (maxToolCallsPerTurn), which bounds the blast radius but also makes the worst case explicit: 64 × the 30 s default per-call timeout ≈ 32 minutes of blocking on the request path when no request_timeout is configured.

Nothing in the code justifies the serial choice — no comment, no TODO.

Concurrency safety is already established

Checked before filing, since it is the precondition:

• The HTTP transport is built for it — internal/mcp/client.go guards the session ID with a sync.RWMutex whose doc comment states "concurrent CallTool invocations only contend for a read lock", and request IDs come from an atomic.Int64.
• The stdio transport documents concurrency-safety as well.

So the fan-out is viable. Shape: a bounded worker group (not go per call), with results written into a pre-sized slice by index so ordering and tool_call_id pairing survive for the assistant message.

Why this is not a straightforward patch

Concurrency is an observable change for the MCP server, not just for the gateway. A stateful server — a database writer, a file editor, anything with ordering assumptions — currently receives calls serialized and would start receiving them in parallel. That can break correctness on the server side, silently, on a patch upgrade.

Two options:

1. Ship as a minor version.
2. Ship in a patch behind a per-server max_concurrent_tool_calls, defaulting to 1 — current behaviour preserved, opt-in fan-out. This also gives operators a throttle for servers that cannot handle parallelism.

Option 2 seems preferable, but it is a judgement call worth agreeing before implementation.