Skip to content

feat(web): WebSocket channel with MCP tool proxy and structured annotations#13

Open
bobbymarko wants to merge 6 commits into
longbkit:mainfrom
bobbymarko:pr/web-websocket-channel
Open

feat(web): WebSocket channel with MCP tool proxy and structured annotations#13
bobbymarko wants to merge 6 commits into
longbkit:mainfrom
bobbymarko:pr/web-websocket-channel

Conversation

@bobbymarko

@bobbymarko bobbymarko commented May 28, 2026

Copy link
Copy Markdown
Contributor

What this enables

Most agent interfaces are a single chat thread. The web channel opens up a richer pattern: browser UIs that wire multiple agent conversations to structured data, where each piece of data gets its own isolated AI session and the UI stays in control of state.

A concrete example of what you can build: a kanban-style task manager where each task card opens a dedicated agent conversation. The UI loads all tasks directly from a data source via invoke_tool — no agent round-trip, just a fast MCP call over the persistent WebSocket. When the user taps a card and starts chatting, the agent gets the task context injected into the message. As the agent researches (finding a phone number, a price, a contact), it emits [[CHANNEL_EVENT:brief:...]] annotations that flow back to the card as a live summary — without cluttering the chat transcript. Each card's conversation is scoped by contextId, so tasks never bleed into each other's history.

The channel is intentionally minimal: it handles auth, session routing, streaming, and tool proxying. Everything else — what the UI looks like, what MCP servers are connected, what context gets injected into messages — is the client's concern.


Summary

Adds a new web channel — a WebSocket-based channel that connects any browser-side app to a clisbot agent. Includes a generic MCP tool proxy so the UI can call tools directly, and a structured annotation mechanism so agents can push typed events to the client without leaking them into visible chat text.

What's included

New channel (src/channels/web/)

  • WebSocket server with API-key auth (HTTP upgrade, ?key= query param)
  • message — routes text to the configured agent; contextId (optional) scopes to a per-context conversation thread
  • get_history — replays the agent's Claude Code session JSONL back to the client so the chat UI can restore history on reconnect
  • invoke_tool — proxies any tools/call to a configured MCP server over stdio; the client can call databases, APIs, or anything MCP-accessible without going through the agent

mcp-client.ts — lightweight JSON-RPC 2.0 stdio MCP client

  • Spawns an MCP server process, handles the initialize handshake, maps pending requests by ID
  • Unwraps the content[0].text envelope so callers get plain JSON back
  • Persistent per-server connection, lazy-initialized and reused across requests

history-reader.ts — reads Claude Code session JSONL files for history replay

  • Strips client-injected context (everything before [USER_INPUT] marker)
  • Strips [[CHANNEL_EVENT:...]] annotations from historical messages

Generic [[CHANNEL_EVENT:key:value]] annotations

  • The agent can embed [[CHANNEL_EVENT:key:value]] anywhere in its reply
  • The channel strips them from the visible text stream (handles partial markers mid-stream)
  • Emits { type: "annotation", contextId, key, value } WS events
  • Client apps define their own annotation keys — the channel has zero knowledge of their meaning

Config additions

  • bots.web.mcpServers — map of server name → { command, args, env } spawn configs; only declared servers are accessible via invoke_tool
  • bots.web.senderName — display name for the human sender (default "Owner")

WebSocket protocol

Client → server

{ "type": "message", "text": "...", "contextId": "optional-thread-id" }
{ "type": "get_history", "contextId": "..." }
{ "type": "invoke_tool", "server": "my-tools", "tool": "search", "args": { "q": "..." } }

Server → client

{ "type": "ready" }
{ "type": "chunk", "id": "wc1", "text": "partial agent response..." }
{ "type": "update", "id": "wc1", "text": "revised agent response..." }
{ "type": "done" }
{ "type": "tool_result", "server": "my-tools", "tool": "search", "result": {...} }
{ "type": "annotation", "contextId": "...", "key": "my-key", "value": "..." }
{ "type": "history", "contextId": "...", "messages": [{"role":"user","text":"..."},...] }
{ "type": "error", "message": "..." }

Example config

"bots": {
  "web": {
    "apiKey": "your-secret-key",
    "port": 3099,
    "senderName": "Alice",
    "mcpServers": {
      "my-tools": {
        "command": "python3",
        "args": ["/path/to/my-mcp-server.py"]
      }
    }
  }
}

🤖 Generated with Claude Code

Bobby Marko and others added 4 commits May 28, 2026 18:43
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ide context

- Add invoke_tool WebSocket handler: proxies to any configured MCP server via stdio
- Add mcp-client.ts: lightweight JSON-RPC 2.0 stdio client with pending-map concurrency
- Add get_history handler: reads Claude Code session JSONL for chat history replay
- Replace BRIEF_UPDATE with generic [[CHANNEL_EVENT:key:value]] extraction:
  strips markers from the visible stream, emits typed 'annotation' WS events
- Move context injection out of the channel into the client (text field now carries full context)
- Add web.senderName config field (defaults to 'Owner')
- Add web.mcpServers config schema for declaring allowed MCP servers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- All WS protocol fields use contextId (generic opaque thread ID)
- history-reader strips context via [USER_INPUT] marker only — no app-specific prefix checks
- history-reader strips [[CHANNEL_EVENT:...]] only — no legacy BRIEF_UPDATE

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@bobbymarko
bobbymarko marked this pull request as ready for review May 28, 2026 18:56
Bobby Marko and others added 2 commits May 29, 2026 00:13
…T annotations

message-tool responseMode waits 3s for a tool-final reply that never arrives (Claude CLI
just outputs text, no send_message tool), then returns early without calling
renderResponseText on the completed snapshot. This left accumulatedText as the last
streaming chunk value, which was unreliable for CHANNEL_EVENT extraction.

capture-pane takes the paneManagedDelivery path which always calls renderResponseText
with the final completed snapshot before returning, guaranteeing accumulatedText contains
the full response including any [[CHANNEL_EVENT:brief:...]] markers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…fences

Claude Code renders markdown in the terminal, so tmux pane capture strips
code fence language identifiers (e.g. \`\`\`a2ui:contact\`). Reading the
final assistant message directly from the session JSONL (raw API response)
ensures code fences reach the web client intact.

Also add 'web' as a passthrough platform in renderPlatformInteraction so
the web channel bypasses Slack/Telegram formatting entirely — the web
client handles its own markdown rendering.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@bobbymarko

Copy link
Copy Markdown
Contributor Author

@longbkit I realize this is a pretty obscure use case of wanting to interact with Clisbot via a web interface. But I have found it to be a powerful unlock for expanding beyond the boundaries of what messaging apps support. On top of this I built my own todo app with each todo item being able to spawn its own conversation. I also built support for A2UI so my responses can have interactive visual components instead of being text. But if this is too specific, maybe we could build a federated channel plugin framework so others can add their own channels without having to add them to the codebase?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant