Merge branch 'main' into fix-gateway-streaming-tool-call-id

Author: alexkroman

REFERENCE.md

@@ -139,3 +139,27 @@ writes:
 derives that title from the transcript via the LLM Gateway once the stream ends,
 renaming the files to match (the timestamp stem is kept if the title is empty).
 The two are mutually exclusive.
+
+## Live agent tools (MCP)
+
+`assembly live` answers each spoken turn with a tool-using agent, so it can reach
+external tools mid-conversation. Out of the box it loads its built-in URL fetch,
+the AssemblyAI docs, and a curated, no-auth MCP toolset: `time` and `fetch`
+(`uvx`), `memory` and `filesystem` (`npx`, the latter rooted at the working
+directory), and an NWS-backed `weather` server.
+
+Firecrawl web search also loads when a `FIRECRAWL_API_KEY` is set; without it the
+session prints a one-line notice and runs without web search (every other default
+tool needs no key).
+
+`--mcp-config FILE` adds your own servers on top of the defaults, from a standard
+`mcpServers` JSON file — the same
+`{"mcpServers": {"name": {"command": "…", "args": […]}}}` shape Claude Desktop and
+Claude Code use. Repeat the flag to merge several files; a later file (or a config
+entry sharing a default's name) wins on a clash. Remote servers use `{"url": "…"}`
+instead of `command`/`args`.
+
+Each server is launched independently and best-effort: one that won't start (a
+missing `npx`/`uvx`, an offline host) drops only its own tools, so a single broken
+tool never sinks the session. MCP tools are a live-run feature and are not
+reflected in `--show-code` output.

aai_cli/AGENTS.md

@@ -151,7 +151,7 @@ heavily-reworked commands with long bodies; small commands keep the inline
 - **`streaming/`** + `client.stream_audio` — v3 realtime API. Event callbacks run on the SDK reader thread and guard against `BrokenPipeError` (`stdio.silence_stdout()`) so a closed pipe never dumps a thread traceback.
 - **`core/sync_stt.py`** + **`core/signals.py`** + `commands/dictate/` — `assembly dictate`: headless dictation over the **Sync STT API** (`Environment.sync_base`, one POST `/transcribe` per utterance with the required `X-AAI-Model: u3-sync-pro` header; 80 ms–120 s of PCM/WAV). It needs no terminal: recording starts immediately and `dictate_exec._record` polls `signals.stop_on_terminate` between ~100 ms mic chunks for a SIGTERM, which finishes the utterance (clean exit 0) — so a hotkey tool like Hammerspoon can launch it as a background task and `kill -TERM`/`task:terminate()` to transcribe. SIGINT (Ctrl-C) still cancels (exit 130). Both boundaries (the stop latch, mic, HTTP) are injectable, so the suite never needs a real signal or microphone (`tests/test_dictate_exec.py` scripts the SIGTERM latch). Contrast `signals.terminate_as_interrupt` (used by `stream`/`agent`/`speak`), which routes SIGTERM into the *cancel* path instead.
 - **`agent/`** — full-duplex voice agent (mic in, TTS out via `voices.py`).
-- **`agent_cascade/`** + `commands/agent_cascade/` — `assembly agent-cascade`: the same live terminal conversation as `assembly agent`, but **client-orchestrated** — `engine.run_cascade` wires Streaming STT → the LLM Gateway → streaming TTS itself instead of talking to the Voice Agent endpoint, mirroring what the `agent-cascade` `assembly init` template does server-side. **Sandbox-only** (streaming TTS has no prod host; guarded via `tts.session.require_available`). Reuses the agent slice's `DuplexAudio`/`AgentRenderer` and `core.client.stream_audio`/`core.llm.complete`/`tts.session.synthesize`; the three network legs are injected through `engine.CascadeDeps` (the `tts/session.py` seam) so the cascade — greeting, per-sentence TTS, barge-in, history window — is unit-tested against fakes with no sockets/mic/speaker.
+- **`agent_cascade/`** + `commands/agent_cascade/` — `assembly agent-cascade`: the same live terminal conversation as `assembly agent`, but **client-orchestrated** — `engine.run_cascade` wires Streaming STT → the LLM Gateway → streaming TTS itself instead of talking to the Voice Agent endpoint, mirroring what the `agent-cascade` `assembly init` template does server-side. **Sandbox-only** (streaming TTS has no prod host; guarded via `tts.session.require_available`). Reuses the agent slice's `DuplexAudio`/`AgentRenderer` and `core.client.stream_audio`/`core.llm.complete`/`tts.session.synthesize`; the three network legs are injected through `engine.CascadeDeps` (the `tts/session.py` seam) so the cascade — greeting, per-sentence TTS, barge-in, history window — is unit-tested against fakes with no sockets/mic/speaker. The LLM leg is a deepagents graph (`brain.py`); under `-v` (`debuglog.active()`) `brain._run_graph` *streams* that graph instead of `invoke`-ing it and logs each tool call/result/interim line as it lands (reusing `code_agent.events.message_events`), so a spoken turn that stalls mid-tool is debuggable — plain `invoke` runs the whole loop internally and `-v` would otherwise show only the httpx lines.
 - **`tts/`** + `commands/speak.py` — `assembly speak` synthesizes text to speech over the sandbox streaming-TTS WebSocket (`streaming-tts.sandbox000.…`). **Sandbox-only:** `session.is_available()` is false in production (empty `Environment.streaming_tts_host`), so the command exits 2 with a `--sandbox` hint. `session.synthesize` drives a Begin→Generate→Flush→Audio→Terminate protocol with an injectable `connect` for hermetic tests (mirrors `agent/session.py`); `audio.py` plays the PCM (default) or writes a WAV (`--out`). The single-voice default-playback path **streams**: `synthesize`'s `on_audio(chunk, sample_rate)` callback is wired to `audio.PcmPlayer.feed`, so speech starts on the first Audio frame (it opens the device lazily, since the rate is only known at Begin) instead of after the whole text — the win for a long `--url` page. `--out` (needs the full buffer) and the multi-voice dialogue path (`synthesize_dialogue` → `_output_audio` → buffered `play_pcm`) stay buffered; `synthesize` still returns the complete PCM for the summary regardless.
 - **`code_agent/`** + `commands/code/` — `assembly code`: a terminal coding agent (a bespoke port of langchain-ai/deepagents' `code` agent) that talks **only** to the LLM Gateway. `model.py` pins the model to `ChatOpenAI` against `llm_gateway_base`; `agent.py` builds the deepagents graph over a cwd-scoped `LocalShellBackend` (filesystem + shell tools), plus extra tools: the custom `assembly` CLI tool (`cli_tool.py`, runs `python -m aai_cli` with the key via child env, never argv), a URL `fetch_url` tool (`fetch_tool.py`), Tavily web search when `TAVILY_API_KEY` is set (`web_search.py`), an `ask_user` tool routed through an `AskBridge` to the front-end (`ask_tool.py`), and best-effort docs MCP tools (`docs_mcp.py`). Middleware adds installed skills (`skills.py`) and long-term memory (`memory.py`), each over its own dedicated backend. Sessions persist via a SQLite checkpointer (`store.py`) keyed by `--session`, so conversations resume. Approval gates the mutating tools (write/edit/execute/`assembly`/`fetch_url`); the general-purpose `task` subagent comes from deepagents by default. `session.py` drives the graph turn-by-turn (interrupt/resume = human approval), emitting framework-agnostic `events.py` to either the Textual TUI (`tui.py`, modeled on deepagents-code: transcript + input + approval/ask modals + clipboard copy) or the Rich fallback (`render.py`). The whole orchestration is tested by driving the **real** graph with a fake `BaseChatModel` (`tests/test_code_agent.py`), so no network/TTY is needed. **Voice is the default front-end in an interactive TTY** (`voice.py` + `_exec._run_voice`): `VoiceSession.listen` captures one spoken turn over Streaming STT (gating the mic shut the instant a turn finalizes) and `VoiceSession.speak` reads each assistant reply back over streaming TTS. It runs the **Rich REPL** loop (not the keyboard TUI) with a voice `read_line` + a reply-speaking sink. Readback needs streaming TTS, so it's **sandbox-only** (`tts.session.is_available`); in production the mic input still works and replies stay on screen. A mic-less box degrades to typed input on the first `AUDIO_ERROR_TYPES` `CLIError`; `--no-voice` selects the TUI, and a non-TTY (pipe/CI) the headless loop. Both legs (STT/TTS) are injected like the cascade's, so `tests/test_code_voice.py` drives it with fakes — no mic/speaker/socket.
 - **`code_gen/`** — backs `--show-code` on `transcribe`/`stream`/`agent`: builds a ready-to-run Python SDK script from exactly the flags passed (no API key needed; generated code reads `ASSEMBLYAI_API_KEY`).

aai_cli/agent_cascade/brain.py

@@ -16,18 +16,31 @@
 
 from __future__ import annotations
 
+import logging
 from collections.abc import Callable, Sequence
 from typing import TYPE_CHECKING
 
 from aai_cli.agent_cascade.config import CascadeConfig
 from aai_cli.code_agent.agent import CompiledAgent
 from aai_cli.code_agent.fetch_tool import FETCH_TOOL_NAME
-from aai_cli.code_agent.web_search import WEB_SEARCH_TOOL_NAME
+from aai_cli.code_agent.firecrawl_search import WEB_SEARCH_TOOL_NAME
+from aai_cli.core import debuglog
 
 if TYPE_CHECKING:
     from langchain_core.tools import BaseTool
     from openai.types.chat import ChatCompletionMessageParam
 
+# Verbose (`-v`) flow logging for the agent's tool loop. `invoke` runs the whole loop
+# internally, so without this `-v` only shows the httpx request lines and never which
+# tools the agent reached for or what they returned — exactly what you need to see when
+# a spoken turn stalls mid-tool. Logged at INFO so plain `-v` surfaces it.
+_FLOW_LOG = logging.getLogger("aai_cli.agent_cascade.brain")
+
+# Tool outputs (a fetched page, a search payload) can be huge; cap what we log per result
+# so a single tool call doesn't bury the rest of the flow in stderr. The exact cap is an
+# arbitrary tuning knob — a +-1 shift is behaviorally equivalent, so no test can kill it.
+_RESULT_LOG_CAP = 500  # pragma: no mutate
+
 # Closes every guidance variant: the reply is spoken, so it must stay short and plain.
 _SPOKEN_TAIL = (
     "Your reply is read aloud, so keep it short and spoken — no markdown, lists, code, or raw URLs."
@@ -60,8 +73,8 @@ def _tool_capabilities(tools: Sequence[BaseTool]) -> list[str]:
     """The spoken-capability phrases backed by an actually-present tool.
 
     Derived from the resolved tool names so the prompt never advertises a capability the
-    agent can't perform: web search is present only with a ``TAVILY_API_KEY``, and the docs
-    tools are best-effort (absent when the docs host is unreachable).
+    agent can't perform: web search is present only with a ``FIRECRAWL_API_KEY``, and the
+    docs tools are best-effort (absent when the docs host is unreachable).
     """
     names = {tool.name for tool in tools}
     capabilities: list[str] = []
@@ -74,15 +87,35 @@ def _tool_capabilities(tools: Sequence[BaseTool]) -> list[str]:
     return capabilities
 
 
-def build_system_prompt(persona: str, *, tools: Sequence[BaseTool]) -> str:
+def _extra_capability(extra_tools: Sequence[BaseTool]) -> str | None:
+    """The spoken-capability phrase for user-configured MCP tools, listing them by name.
+
+    The deepagents graph already shows the model each tool's schema, so this only has to
+    name the tools so the guidance doesn't claim "no external tools" when MCP tools are
+    bound — and so the model knows to reach for them.
+    """
+    names = sorted(tool.name for tool in extra_tools)
+    if not names:
+        return None
+    return f"use your connected tools ({', '.join(names)})"
+
+
+def build_system_prompt(
+    persona: str, *, tools: Sequence[BaseTool], extra_tools: Sequence[BaseTool] = ()
+) -> str:
     """The live agent's system prompt: the user's persona plus tool guidance.
 
-    The guidance is tailored to ``tools`` so the model is only told about capabilities it
-    actually has — advertising a missing tool (web search without a ``TAVILY_API_KEY``) made
-    the agent announce an action it then couldn't take, leaving the turn hanging with no
-    answer. With no tools at all the model is told to answer from its own knowledge.
+    The guidance is tailored to the bound tools so the model is only told about
+    capabilities it actually has — advertising a missing tool (web search without a
+    ``FIRECRAWL_API_KEY``) made the agent announce an action it then couldn't take, leaving
+    the turn hanging with no answer. ``tools`` are the built-in legs (web search, URL
+    fetch, AssemblyAI docs); ``extra_tools`` are user-configured MCP tools, advertised
+    generically by name. With no tools at all the model answers from its own knowledge.
     """
     capabilities = _tool_capabilities(tools)
+    extra = _extra_capability(extra_tools)
+    if extra is not None:
+        capabilities.append(extra)
     if not capabilities:
         return f"{persona}\n\n{_NO_TOOLS_GUIDANCE}"
     guidance = (
@@ -100,12 +133,12 @@ def build_live_tools() -> list[BaseTool]:
     All three are reused from the coding agent's tool modules. Unlike there they are
     *not* approval-gated — a spoken turn can't wait for a keyboard confirmation, so the
     live agent only gets read-only tools and runs them automatically. Web search is
-    present only when ``TAVILY_API_KEY`` is set; the docs MCP is best-effort (an empty
+    present only when ``FIRECRAWL_API_KEY`` is set; the docs MCP is best-effort (an empty
     list when the host is unreachable), so neither blocks a session.
     """
     from aai_cli.code_agent.docs_mcp import load_docs_tools
     from aai_cli.code_agent.fetch_tool import build_fetch_tool
-    from aai_cli.code_agent.web_search import build_web_search_tool
+    from aai_cli.code_agent.firecrawl_search import build_web_search_tool
 
     tools: list[BaseTool] = [build_fetch_tool()]
     search = build_web_search_tool()
@@ -116,27 +149,36 @@ def build_live_tools() -> list[BaseTool]:
 
 
 def build_graph(
-    api_key: str, config: CascadeConfig, *, tools: Sequence[BaseTool] | None = None
+    api_key: str,
+    config: CascadeConfig,
+    *,
+    tools: Sequence[BaseTool] | None = None,
+    mcp_tools: Sequence[BaseTool] | None = None,
 ) -> CompiledAgent:
     """Compile the deepagents graph for one live session over the gateway model.
 
     Reuses the coding agent's gateway-bound ``ChatOpenAI`` (so the live agent can only
     ever reach AssemblyAI), threading the cascade's ``--max-tokens``/``--llm-config``
-    through it. ``tools`` defaults to :func:`build_live_tools`; tests pass an explicit
-    (possibly empty) list to skip the network-touching docs probe.
+    through it. ``tools`` defaults to :func:`build_live_tools`; ``mcp_tools`` defaults to
+    the tools of the servers in ``config.mcp_servers``. The two are kept apart so the
+    system prompt advertises the built-in legs and the MCP tools differently, but the
+    model is bound to both. Tests pass explicit (possibly empty) lists to skip the
+    network-touching docs/MCP probes.
     """
     from deepagents import create_deep_agent
 
+    from aai_cli.agent_cascade.mcp_tools import load_mcp_tools
     from aai_cli.code_agent.model import build_model
 
     model = build_model(
         api_key, model=config.model, max_tokens=config.max_tokens, extra=config.llm_extra
     )
-    resolved = build_live_tools() if tools is None else list(tools)
+    builtin = build_live_tools() if tools is None else list(tools)
+    extra = load_mcp_tools(config.mcp_servers) if mcp_tools is None else list(mcp_tools)
     return create_deep_agent(
         model=model,
-        tools=resolved,
-        system_prompt=build_system_prompt(config.system_prompt, tools=resolved),
+        tools=builtin + extra,
+        system_prompt=build_system_prompt(config.system_prompt, tools=builtin, extra_tools=extra),
     )
 
 
@@ -147,18 +189,79 @@ def build_completer(
 
     The cascade prepends its own ``system`` message to the history each turn; the graph
     already owns the system prompt, so we drop it before invoking. The graph runs the
-    full tool loop and we return its final spoken text. ``graph`` is injected in tests
-    so the per-turn wiring runs against a fake with no network.
+    full tool loop and we return its final spoken text. Under ``-v`` the loop is streamed
+    so each tool call/result is logged as it lands (see :func:`_run_graph`). ``graph`` is
+    injected in tests so the per-turn wiring runs against a fake with no network.
     """
     resolved = build_graph(api_key, config) if graph is None else graph
 
     def complete_reply(messages: list[ChatCompletionMessageParam]) -> str:
         conversation = [message for message in messages if message.get("role") != "system"]
-        return _reply_text(resolved.invoke({"messages": conversation}))
+        return _reply_text(_run_graph(resolved, conversation))
 
     return complete_reply
 
 
+def _run_graph(
+    graph: CompiledAgent, conversation: list[ChatCompletionMessageParam]
+) -> dict[str, object]:
+    """Run one turn through the graph, returning its end state.
+
+    Normally a single ``invoke`` (the whole tool loop runs internally). Under verbose
+    mode, and when the graph can stream, drive it as incremental state snapshots instead
+    so :func:`_log_flow` can surface each tool call/result on stderr as it happens — which
+    is what makes a stalled spoken turn debuggable. The test fakes only implement
+    ``invoke``, so they (and the non-verbose path) take the plain branch.
+    """
+    graph_input = {"messages": conversation}
+    if debuglog.active() and hasattr(graph, "stream"):
+        last: dict[str, object] = {}
+        seen = 0
+        for chunk in graph.stream(graph_input, None, stream_mode="values"):
+            seen = _log_flow(chunk, seen)
+            last = chunk
+        return last
+    return graph.invoke(graph_input)
+
+
+def _log_flow(state: dict[str, object], seen: int) -> int:
+    """Log the tool calls/results added to ``state`` since the first ``seen`` messages.
+
+    Reuses the coding agent's message→event vocabulary so the flow log knows the same
+    AIMessage/ToolMessage shapes the TUI does. Returns the new high-water message count
+    so the next snapshot only logs what it added.
+    """
+    from aai_cli.code_agent.events import AssistantText, ToolCall, ToolResult, message_events
+
+    messages = state.get("messages")
+    if not isinstance(messages, list):
+        return seen
+    for message in messages[seen:]:
+        for event in message_events(message, announce_calls=True):
+            if isinstance(event, ToolCall):
+                _FLOW_LOG.info("tool call %s args=%s", event.name, event.args)
+            elif isinstance(event, ToolResult):
+                _FLOW_LOG.info("tool result %s -> %s", event.name, _clip(event.content))
+            elif isinstance(event, AssistantText):
+                _FLOW_LOG.info("llm: %s", event.text)
+    return len(messages)
+
+
+def _clip(text: str) -> str:
+    """Flatten a tool result onto one line and truncate it for the flow log.
+
+    Tool output is untrusted external content (a fetched page, a search payload), so its
+    whitespace — newlines especially — is collapsed before logging: a result can't then
+    forge extra ``[aai_cli.…]`` log lines, and each result stays on one readable line. The
+    length is capped so a multi-KB payload can't bury the rest of the flow. (Secrets are
+    separately masked by the debuglog formatter across every record.)
+    """
+    flattened = " ".join(text.split())
+    if len(flattened) <= _RESULT_LOG_CAP:
+        return flattened
+    return f"{flattened[:_RESULT_LOG_CAP]}… ({len(flattened)} chars)"
+
+
 def _reply_text(result: dict[str, object]) -> str:
     """The agent's final spoken reply: the last assistant message that carries text.