Assistant Brain

A custom Home Assistant conversation agent backed by a Cloudflare Worker that reasons with Claude and can call personal MCP tools — Home Assistant itself and Joplin notes — plus a set of native tools built into the Worker (clock, math, unit conversion, random). Say something into Home Assistant's Assist pipeline and get back an answer that may have involved checking or changing the state of the house, reading your notes, or exact date/arithmetic/unit-conversion work the model can't reliably do on its own.

Architecture

Android (HA Companion App)
  └── Assist  (home button hold / wake word / lock screen)
        └── HA Voice Pipeline
              ├── STT — Whisper, local
              ├── conversation.AssistantBrainAgent  (Python, custom_components)
              │     └── HTTPS POST ──► assistant-brain Worker  (TypeScript)
              │                          ├── Claude via AI Gateway (/ai/v1/messages)
              │                          ├── KV — short-lived conversation history
              │                          └── tool loop over native + MCP tools
              │                                ├── brain__*              → src/tools/ (in-Worker)
              │                                ├── HA_MCP_SERVICE        → home-assistant-mcp
              │                                └── JOPLIN_MCP_SERVICE    → joplin-mcp
              └── TTS — Piper, local

Two components live in this repository:

• src/ — the assistant-brain Cloudflare Worker. Exposes POST / behind a bearer token, loads short conversation history from KV, connects to the MCP servers over service bindings, and runs a bounded tool-calling loop against Claude via Cloudflare's AI Gateway. src/tools/ adds general-purpose native tools (brain__clock, brain__math, brain__convert_units, brain__random) that need no keys or network, presented to Claude alongside the MCP tools.
• custom_components/assistant_brain/ — a Home Assistant custom integration exposing one ConversationEntity that POSTs the transcribed text to the Worker and speaks the reply.

Prerequisites

• A Cloudflare account with the MCP Workers already deployed: home-assistant-mcp and joplin-mcp (each mounted at /mcp, each callable as a Worker-to-Worker service binding).
• A Cloudflare API token with Workers AI: Read permission (used to call the AI Gateway /ai/v1/messages endpoint — no Anthropic API key needed, see below).
• A Home Assistant instance with the Assist voice pipeline configured (Whisper for STT, Piper for TTS).

Worker setup

npm install
npx wrangler kv namespace create CONVERSATIONS
# copy the printed id into wrangler.jsonc's kv_namespaces[0].id

npx wrangler secret put BRAIN_TOKEN      # a long random string, e.g. `openssl rand -hex 32`
npx wrangler secret put CF_API_TOKEN     # the Cloudflare token with Workers AI: Read

npm run deploy

Config values to change if you fork this

• ACCOUNT_ID in wrangler.jsonc's vars — your Cloudflare account id.
• The three services[].service names — your own MCP server Worker names.
• The routes[0].pattern — your own hostname on your own zone.

Home Assistant setup

1. Copy the whole custom_components/assistant_brain/ folder into your HA config directory's custom_components/:

   \\<your-ha-host>\config\custom_components\assistant_brain\
   

2. Restart Home Assistant.
3. Settings → Devices & Services → Add Integration → search for Assistant Brain, and fill in the form:
   • Worker URL — https://<your-worker-route>/
   • Bearer token — the BRAIN_TOKEN value from above

The integration is set up via a config entry (added through the UI), not configuration.yaml — recent Home Assistant versions require conversation agents to be registered this way; YAML platform discovery silently never creates the entity.

Testing it

• Developer Tools → States — confirm conversation.assistant_brain exists.
• Developer Tools → Actions → conversation.process (YAML mode):

  action: conversation.process
  data:
    agent_id: conversation.assistant_brain
    text: What rooms are in my house?

• Settings → Voice assistants → your pipeline → Conversation agent → select Assistant Brain, then hold the home button in the phone app and speak a request.

Development commands

npm run dev          # wrangler dev — local Worker dev server
npm test              # vitest run
npm run typecheck      # tsc --noEmit
pytest python_tests/   # Python integration tests (needs pytest installed)

How it works

• Tool loop (src/brain.ts) — sends the conversation to Claude with the aggregated tool list. If Claude responds with tool_use blocks, every call in that turn runs concurrently and all their results go back in a single tool_result user message (splitting them across messages trains the model out of calling tools in parallel). The loop is bounded by both an iteration cap (MAX_ITERATIONS) and a wall-clock budget (TIME_BUDGET_MS), because either alone can run away.
• Namespacing (src/mcp/naming.ts) — each MCP server's tools are prefixed with the server name and a double underscore (home_assistant__get_entities) so Claude sees one flat, collision-free tool list across all three servers, and calls route back to the right server by parsing the prefix.
• Native tools (src/tools/) — general-purpose tools implemented in the Worker itself, namespaced brain__* the same way and listed first so the cached prompt prefix survives even if an MCP server drops out. brain__clock covers date/time/date-math, brain__math arithmetic, brain__convert_units unit conversion, and brain__random dice/coin-flip/pick — things LLMs are unreliable at doing themselves.
• Memory (src/memory.ts) — the last few turns of a conversation are cached in KV under the conversation_id, short-lived (HISTORY_TTL_S). Only plain-text turns are stored; tool traffic is dropped, because replaying a stale tool result as if it were current would be actively misleading.

No Anthropic API key is used anywhere in this project. Claude is called through Cloudflare's AI Gateway (/ai/v1/messages), authenticated with a Cloudflare API token, and billed through Cloudflare Unified Billing.