Hermes-Relay — Decisions & Implementation Guide

Updated: 2026-04-06

Read this before SPEC.md — it tells you what to build, what was deferred, and why.

Framework Decision: Kotlin + Jetpack Compose

Chosen over: React Native, Flutter, Kotlin + XML (upstream)

Why Compose:

80% of the app is native Android services (AccessibilityService, PTY, foreground services, biometrics). Cross-platform frameworks would still need Kotlin native modules for all of that, plus a bridge layer.
Compose is declarative like React — same mental model (state → UI), different syntax. Familiar to React developers.
Material 3 / Material You theming is first-class. Dynamic color from wallpaper, proper motion system.
OkHttp WebSocket supports wss:// natively. No bridge layer to debug.
Background foreground service keeps the WSS connection alive when the phone locks. React Native's background execution is fragile.
Single language (Kotlin) for the entire app — services, UI, networking.

Why not React Native:

Would need native modules for: AccessibilityService, foreground service, biometric auth, EncryptedSharedPreferences, MediaProjection. That's most of the app written in Kotlin anyway, plus JS bridge overhead.
Background reliability issues on Android are well-documented.
Only makes sense if the UI were 80%+ of the codebase. Here it's ~20%.

Why not Flutter:

Same native bridge problem as RN, but with Dart (new language) instead of familiar React patterns.
Platform channels for every native API.
Smaller Android-specific ecosystem for security/biometric libraries.

Why not staying with Kotlin + XML (upstream):

XML layouts are legacy. Compose is the modern Android UI toolkit.
Upstream UI is functional but not polished. Full rewrite needed anyway.
Compose gives us better animation, theming, and state management primitives.

Architecture Decisions

1. Single WSS Connection with Channel Multiplexing

Decision: One WebSocket connection carries relay real-time channels via typed message envelopes. The original set was chat, terminal, and bridge; later releases added TUI over the relay envelope, while voice uses Relay-protected HTTP routes with the same session/grant model.

Why: Simpler connection management, single auth flow, single reconnect handler. Mobile networks are flaky — one connection is easier to keep alive than three.

Trade-off: If one channel floods (e.g., terminal output), it could delay others. Mitigated by: terminal output batching (16ms frames), priority queuing (system > chat > terminal > bridge).

2. Relay Server as Separate Service (Port 8767)

Decision: New Python relay service, separate from the existing bridge relay (8766) and the Hermes gateway.

Why:

The existing relay is single-purpose (bridge only). Extending it risks breaking upstream compatibility.
Separate service means we can deploy/restart independently of the gateway.
Future: can merge into gateway as a platform adapter if it stabilizes.

Alternative considered: Adding as a gateway platform adapter. Deferred — too coupled to gateway lifecycle for MVP.

3. Chat via Direct API, Not Relay Proxy

Status: Superseded as the standard route by ADR 38 (2026-07-18). Direct API chat remains the automatic fallback and an advanced headless compatibility mode; the upstream Dashboard/Gateway is now the primary connection surface.

Decision: ~~Chat channel proxies through the relay to the WebAPI.~~ Updated: Chat now connects directly from the Android app to the Hermes API Server via HTTP/SSE. The relay server is only used for bridge and terminal channels.

Why (original relay approach):

WebAPI already handles session management, agent creation, SSE streaming, tool progress events.
Gateway integration would require implementing a new platform adapter. Much more work.
WebAPI is stable, documented, and used by ClawPort.

Why direct API (updated 2026-04-05):

The relay was an unnecessary middleman for chat — it just converted SSE to WebSocket envelopes.
Every other Hermes frontend (Open WebUI, ClawPort, LobeChat, etc.) connects directly to the API server.
Direct connection is simpler, removes the relay as a single point of failure for chat, and reduces latency.
The relay remains for bridge (device control) and terminal (tmux/PTY) which require custom bidirectional protocols.

Streaming endpoints (updated 2026-04-07):

The app supports two streaming endpoints, selectable in Settings:

Endpoint	Tool Calls	Event Format
Sessions (`/api/sessions/{id}/chat/stream`)	Inline text annotations (`💻 terminal`) — client parses from markdown	Hermes-native SSE (assistant.delta, tool.progress, etc.) or OpenAI-format (delta.content)
Runs (`/v1/runs` + `/v1/runs/{run_id}/events`)	Structured events (tool.started, tool.completed) — real-time tool cards	Hermes lifecycle events (message.delta, tool.started, tool.completed, run.completed)

Important upstream note: The /api/sessions CRUD/chat endpoints are now in upstream Hermes core via focused PR #33134, which salvaged the useful session-control portion of #29302 and covers session list/create/read/update/delete, messages, fork, chat, and chat stream. Read-only skill/toolset discovery is also native via #33016. The bootstrap still ships for older core builds and for surfaces that remain compatibility-only (config, memory, legacy skill detail/toggle, available-models, slash middleware), but sessions and read-only skill lists should now be upstream-first.

The app's probeCapabilities() returns a per-endpoint snapshot, and ConnectionViewModel.resolveStreamingEndpoint() collapses streamingEndpoint = "auto" (the default for new installs) to a concrete "sessions" or "runs" choice based on what the server actually exposes.

Tool call transparency: In /v1/chat/completions streaming, tool calls are NOT emitted as separate SSE events. They are injected as inline markdown text (e.g., `💻 pwd`). The app's annotation parser (ChatHandler.parseAnnotationLine) detects these and renders them as tool progress cards. The /v1/runs endpoint is the only path that provides structured tool.started/tool.completed events.

Architecture:

Phone (HTTP/SSE) → Hermes API Server (:8642)   [chat — direct]
Phone (WSS)      → Relay Server (:8767)          [bridge, terminal]

Auth: Optional Bearer token (API_SERVER_KEY) stored in EncryptedSharedPreferences on device. Most local Hermes setups don't require a key.

4. xterm.js in WebView for Terminal

Decision: Use xterm.js running in a local WebView for the terminal emulator, not a native Compose canvas renderer.

Why:

xterm.js is battle-tested — handles all ANSI escape sequences, Unicode, colors, scrollback.
A native Compose terminal renderer would be weeks of work for inferior rendering.
The WebView is a single composable in an otherwise fully native app — acceptable trade-off.
Can replace with native renderer later if WebView performance is insufficient.

5. tmux for Terminal Session Management

Decision: Terminal channel attaches to tmux sessions, not raw PTY.

Why:

tmux gives persistence — disconnect from the app, reconnect, session is still there.
Named sessions let you manage multiple contexts (different projects, different servers).
Shared sessions — agent and user can see the same terminal (future collaboration).

6. Auth: Pairing Code → Session Token

Decision: Initial pairing via 6-char code (upstream pattern), then long-lived session token for subsequent connections.

Why:

Pairing codes are user-friendly and don't require pre-shared secrets.
Session tokens avoid re-pairing on every app restart.
Tokens stored in EncryptedSharedPreferences (Android Keystore-backed AES-256-GCM).
An explicit re-pair replaces older sessions and trusted credentials for the same non-empty device ID; it does not accumulate duplicate entries for one app installation. Other devices and legacy entries without an identity remain independent.

6a. QR Carries Both API and Relay Credentials (updated 2026-05-03)

2026-07-18 amendment: ADR 38 makes Dashboard/Gateway the normal connection anchor. This API-first QR remains a backward-compatible import and Relay-pairing format. New setup flows should carry an explicit Dashboard/Gateway URL and must not require an API endpoint or API bearer when dashboard chat is ready.

Decision: The Hermes pairing QR payload bundles the API server credentials AND the relay URL + pairing code into a single scan. The pair command (hermes pair, /hermes-relay-pair, or the compatibility hermes-pair shell shim, all backed by plugin/pair.py) runs on the Hermes host; if a relay is reachable at localhost:RELAY_PORT, the command mints a fresh 6-char code, pre-registers it with the relay via a new loopback-only POST /pairing/register endpoint, and embeds {url, code} under a nullable relay key alongside the existing host/port/key/tls fields. The dashboard pairing flow uses the relay's loopback-only POST /pairing/mint endpoint instead; when the dashboard omits api_key, the relay reads the same host-local Hermes API key config as hermes pair and places it in top-level key.

Trust anchor: the operator with shell access on the host. Only a process running on the same machine as the relay can hit /pairing/register — the handler rejects any non-loopback request.remote with HTTP 403. A LAN attacker cannot inject codes. This matches the model we already rely on for reading ~/.hermes/.env and ~/.hermes/config.yaml: if you have shell access to the host, you have enough privilege to authorize a device.

Why the change was necessary:

Previously the phone generated its own 6-char pairing code locally via AuthManager.generatePairingCode() and sent it to the relay on WSS connect. The relay had no way to know what code to accept, so relay pairing was effectively broken — only direct API chat pairing worked via the QR.
Pushing the code flow through the host means the operator always has the source of truth, and a single scan configures both chat and terminal/bridge with no manual steps.

Schema evolution:

Old API-only QRs ({hermes, host, port, key, tls}) still parse cleanly — the relay field is nullable and kotlinx.serialization runs with ignoreUnknownKeys = true.
When --no-relay is passed to the pair command, or the relay isn't running, the QR omits the relay block and the command prints an [info] pointing at hermes relay start.
Top-level key is always the Hermes API bearer token for direct chat/session HTTP. The relay pairing code lives only at relay.code; putting an empty API key in a dashboard-minted QR will pair voice/relay successfully but leaves direct chat unauthenticated when the gateway requires API_SERVER_KEY.
Top-level dashboard_url is optional. When present, Android stores it for Manage and Vanilla Hermes dashboard voice instead of deriving the conventional same-host :9119 URL from the API server.

Pairing alphabet change: PAIRING_ALPHABET in plugin/relay/config.py was widened from ABCDEFGHJKLMNPQRSTUVWXYZ23456789 (32 chars, no ambiguous 0/O/1/I) to the full A-Z / 0-9 (36 chars) to match the phone-side AuthManager.PAIRING_CODE_CHARS. The old restriction only mattered when a human had to retype a code from a display; now that the code flows phone ↔ server through a QR + HTTP, the restriction silently rejected ~12% of valid codes and had to go.

Phase 3 symmetry: POST /pairing/register is written generically enough that the bridge channel's phone-generates-code, host-approves flow (symmetric to the current server-generates, phone-scans flow) can reuse it from the opposite direction. Phone-side generatePairingCode() in AuthManager.kt is retained for that reason.

References:

plugin/pair.py — pair_command(), register_relay_code(), probe_relay()
plugin/relay/server.py — handle_pairing_register, handle_pairing_mint
plugin/dashboard/plugin_api.py — dashboard proxy for /pairing/mint
plugin/relay/auth.py — PairingManager.register_code()
app/.../ui/components/QrPairingScanner.kt — HermesPairingPayload / RelayPairing

7. Biometric Gate for Terminal Only

Decision: Biometric/PIN required before terminal access. Chat and bridge don't require it.

Why:

Terminal = shell access to your server. Highest privilege.
Chat is conversational — no more dangerous than Discord.
Bridge is controlled by the agent, not the user — gating it behind biometrics doesn't help.

8. Dynamic Personalities over Hardcoded Profiles

Terminology note (2026-04-18): The word "Profiles" in this decision's title predates the multi-server / multi-agent work landed in §19 and §21. Today's vocabulary:

Connection (§19) — a paired Hermes server. What earlier drafts sometimes called a "profile".

Profile (§21) — an upstream-Hermes agent directory (~/.hermes/profiles/<name>/). Overlays model + SOUL on chat turns.

Personality (this decision) — a system-prompt preset within one agent's config.

This decision is unchanged; only the surrounding vocabulary shifted.

Decision: Fetch personalities from GET /api/config → config.agent.personalities (name → system prompt map). Display active personality name on chat bubbles. Send selected personality's system prompt via system_message field.

Why:

Hermes stores personalities as system prompt strings in config.yaml under agent.personalities. The server's active personality is in config.display.personality (set by the user during hermes setup).
Previous approach sent a profile parameter that the server ignored — personality switching was non-functional.
Now the app fetches the full personality map, shows the server default first in the picker, and sends the system prompt directly when a non-default personality is selected.
Personalities are maintained in ~/.hermes/config.yaml — changing them doesn't require an app update.
Note: personalities only change the system prompt. Memory, sessions, tools, and model stay the same. For full agent identity separation, use hermes profiles (separate gateway instances).

Trade-off: Requires server connectivity to populate the picker. Fallback is "Default" with no personality override.

8b. Dynamic Command Palette over Hardcoded Slash Commands

Decision: Replace hardcoded slash command list with dynamic sources. Add a searchable command palette (bottom sheet) alongside inline autocomplete.

Why:

Commands come from three sources: 29 gateway built-in commands (from hermes_cli/commands.py — GATEWAY_KNOWN_COMMANDS), dynamic personality commands (from config.agent.personalities), and server skills (from native GET /v1/skills).
The built-in gateway commands are currently hardcoded because hermes-agent has no /api/commands endpoint. This is a potential upstream contribution (see below).
The command palette provides browse-by-category, search, and multi-line descriptions — essential when there are 120+ commands.
Inline autocomplete remains for fast typing: type / and it filters immediately.

Upstream opportunity: Propose a GET /api/commands endpoint for hermes-agent that exposes GATEWAY_KNOWN_COMMANDS from hermes_cli/commands.py. This would let the app fetch built-in commands dynamically instead of hardcoding 29 entries. Filed as a future contribution in docs/upstream-contributions.md.

9. In-App Analytics (Stats for Nerds)

Decision: Add an AppAnalytics singleton that collects performance metrics in-memory (no persistence, no network), displayed via canvas bar charts in Settings.

Why:

Users debugging latency or token consumption need data, not guesses.
Metrics tracked: TTFT (time to first token), completion time, token usage per message, health check latency, stream success/failure rates.
Canvas-rendered bar charts with purple gradient — lightweight, no charting library dependency.
In-memory only — data resets on app restart. No privacy concerns, no storage cost.

Alternative considered: Third-party analytics SDK (Firebase, Mixpanel). Rejected — too heavy for a developer-focused app, and users would rightfully object to telemetry.

10. Command Palette + Inline Autocomplete

Decision: Two complementary command discovery interfaces: inline autocomplete (type / to filter) and a full searchable command palette (bottom sheet via / button).

Why:

120+ commands across three sources (29 gateway built-ins, dynamic personalities, 90+ server skills) — too many for a simple dropdown.
Inline autocomplete serves the "I know what I want" flow — type /mod and /model appears.
Command palette serves the "what's available?" flow — browse by category, search, read descriptions.
Commands are grouped by category: session, configuration, info, personality, then skill categories (creative, devops, etc.).
All slash commands are sent to the server as regular messages. The server handles them as skills or built-in commands. Client stays thin.

Sources:

Gateway built-ins (29): from hermes_cli/commands.py GATEWAY_KNOWN_COMMANDS. Currently hardcoded — no API endpoint exists. See docs/upstream-contributions.md for a proposed GET /api/commands.
Personalities: generated dynamically from GET /api/config → config.agent.personalities.
Skills: fetched from native GET /v1/skills with name and description; legacy detail metadata is optional compatibility data.

11. Animated Splash Screen

Decision: Custom AnimatedVectorDrawable splash with scale + overshoot + fade animation. Hold-while-loading pattern.

Why:

Android 12+ requires a splash screen. Rather than the default static icon, we use a custom animated vector.
The splash holds until DataStore preferences are loaded (determines whether to show onboarding or main app). This prevents a flash of wrong content.
Separate splash_icon.xml at 0.9x scale of ic_launcher_foreground.xml — the splash icon spec uses a different safe zone than adaptive icons.
Smooth fade-out exit animation via SplashScreen.setOnExitAnimationListener.

Deferrals

Feature	Reason	When
iOS support	Android-first, platform-specific APIs	v2+
Multi-device	Single-device simplifies auth and state	Phase 6
On-device model	Complexity, unclear value for relay use case	Phase 6
Voice mode	Depends on Hermes TTS/STT maturity	Phase 6
Notification listener	Not needed for app core	Phase 6
File transfer	Can use agent tools (terminal) as workaround	Phase 6
Android as Hermes platform/channel	See ADR 12 — shipped as the `phone` plugin platform; unified-session "Threads" UI in progress	Shipped 2026-06-28

12. Android as a Hermes Platform/Channel — "Threads" (Shipped 2026-06-28; unified-session model 2026-06-29)

Status (2026-08-12): SHIPPED as the phone platform plugin (plugin/phone_platform.py) — registered via ctx.register_platform with no fork (the ~16-file upstream change sketched below was avoided; the original research predates the open plugin-platform registry). Two-way reply is device-verified. Agent-facing entry is send_message target=phone (the stale target=mobile:<device_id> syntax below is superseded); standalone cron delivery uses the registered sender, and the adapter publishes its canonical home destination through upstream's channel directory. Live cron certification remains tracked in TODO.md.

Decision (2026-06-29) — unified-session "Threads", not a separate surface: the proactive agent↔phone conversation is not a separate app lane/tab/segment. It is a source-tagged session inside the one Chat surface — a Thread (source=phone). The three things distinguishing a Thread from a normal gateway chat are session properties, not a separate UI: (a) the agent can initiate a turn, (b) it rides the relay proactive transport and is relay-gated, (c) it's a standing/named DM. Scrollback = the gateway session store (same read path Chat uses); live receive = the relay proactive push (→ notification); send = proactive.reply. The local ProactiveInboxStore is demoted to a live-push cache + outbox (no parallel history). The Thread capability is surfaced in the connection best-path/capability UI (a relay-tier capability, like terminal/bridge/voice) and as a clean Threads entry (thread-spool icon, NOT a phone glyph) pinned atop the session drawer when active — never a connection-wizard step. Degrades cleanly: no relay plugin → no source=phone sessions → Chat is unchanged (standard-path-safe). This supersedes the earlier "separate Agent lane / 4th nav segment" sketch and folds in the "show chat source/platform attribution in Chat" goal in one stroke.

Outbound-first refinement (2026-08-14): an outbound platform send does not itself create a gateway session; the source=phone session is created only when the phone first replies. Android groups the bounded proactive cache by connection + chat_id and renders a provisional Thread during that gap. A notification tap opens that exact provisional Thread, and the first reply uses the existing phone-platform path before promoting the view to the real gateway session. Once /phone/threads reports the mapping, the provisional row disappears. This cache is a bootstrap transcript, not a second long-term history store.

Reconnect refinement (2026-08-15): Relay explicitly marks messages flushed from its bounded offline queue and follows a completed flush with one count envelope. Android persists that provenance, labels the affected Thread bubbles, and announces one accessible batch summary. This does not attempt background delivery after an Android force-stop; delivery still occurs only when the paired app subscribes again.

Why unified, not separate: a separate "agent chat" tab is redundant — a Thread is just a chat session the agent can also start. One Chat surface (sessions tagged by source) matches the Discord/messaging-app model the product targets. Two distinct senses of "gateway" to keep straight: the messaging gateway / platform layer (gateway/platforms/* — phone/Discord/Slack as platforms; this is the Thread's source) vs. the dashboard gateway transport (/api/ws tui_gateway — how live chat bytes flow). A Thread is defined by its platform/source, not its transport; the two are orthogonal.

Original research (2026-04-07, retained as lineage — the mobile:<device_id> syntax and the ~16-file upstream-fork registration below are SUPERSEDED by the no-fork plugin path):

Decision: Register the Android app as a Hermes platform adapter — like Discord, Telegram, or Slack — so the agent can proactively push messages TO the phone, not just respond to requests.

Current architecture (request/response):

Phone → POST /v1/runs → Agent processes → SSE stream → Phone
         (user initiates every exchange)

Future architecture (bidirectional channel):

Phone ↔ Relay Server ↔ Hermes Gateway (as "mobile" platform)
         (agent can initiate conversations, push notifications, send files)

What this enables:

Agent-initiated messages — "Your build finished," "Reminder: meeting in 10 min"
Cross-platform relay — "Send this to my phone" from Discord/Telegram
Background task results — agent completes a long-running task and pushes the result
Proactive notifications — agent monitors something and alerts the phone
The phone appears in channel_directory.py as a reachable destination
Cron job results delivered directly to phone
send_message tool can target mobile:<device_id>

Upstream Platform Adapter Architecture (Researched 2026-04-07)

Base class: BasePlatformAdapter in gateway/platforms/base.py

Required abstract methods:

Method	Purpose
`connect() → bool`	Start listeners, return True on success
`disconnect()`	Stop listeners, cleanup
`send(chat_id, content, reply_to?, metadata?) → SendResult`	Send a text message
`get_chat_info(chat_id) → Dict`	Return `{name, type, chat_id}`

Optional methods (with default stubs): send_typing, send_image, send_document, send_voice, send_video, edit_message

Key data classes: MessageEvent (inbound), SendResult (outbound), SessionSource (origin tracking)

Registration is NOT plugin-based — requires changes to ~16 files in hermes-agent. There is an official guide: gateway/platforms/ADDING_A_PLATFORM.md (16-step checklist). Key files:

File	Change
`gateway/platforms/mobile.py`	CREATE — `MobileAdapter(BasePlatformAdapter)`
`gateway/config.py`	Add `MOBILE = "mobile"` to Platform enum + env overrides
`gateway/run.py`	Add to `_create_adapter()` factory + authorization maps
`gateway/channel_directory.py`	Add `"mobile"` to session-based discovery list
`tools/send_message_tool.py`	Add to `platform_map` + send routing
`cron/scheduler.py`	Add to `platform_map` for cron delivery
`agent/prompt_builder.py`	Add `PLATFORM_HINTS["mobile"]`
`toolsets.py`	Add `hermes-mobile` toolset

Inbound flow: Adapter receives message via WSS → creates MessageEvent → calls self.handle_message(event) → gateway runs agent → response sent back via adapter.send()

Outbound flow: Agent uses send_message tool → platform_map["mobile"] → adapter.send(chat_id, content) → relay pushes to phone via WSS

Implementation Approaches

Approach	Pros	Cons
A. Upstream PR — Add `mobile.py` to hermes-agent	Full integration, maintained upstream, benefits community	Requires ~16 file changes, PR review process
B. Fork hermes-agent — Add locally	Quick iteration, full control	Maintenance burden, diverges from upstream
C. Relay as bridge — Relay server acts as adapter sidecar	No gateway changes, uses existing relay	Less integrated (no cron delivery, no channel_directory)

Recommended: Approach A (upstream PR) for long-term. Start with Approach C (relay bridge) for prototyping. The relay server already has the persistent WSS connection — it just needs to expose a local HTTP API that the gateway's send_message tool can call.

Session key pattern: agent:main:mobile:dm:<device_id>

Why deferred to Phase 2+:

Current chat works fine as request/response via HTTP/SSE
Terminal and bridge channels are higher priority for MVP
Upstream PR needs careful design and community alignment

13. Skill Distribution via `external_dirs`, not Hub Publish or Hand-Copy (2026-04-11)

Decision: The hermes-relay-pair skill is distributed by adding the Hermes-Relay clone's skills/ directory to skills.external_dirs in ~/.hermes/config.yaml, rather than publishing to the Hermes skill hub or hand-copying files into ~/.hermes/skills/. The skill itself lives at skills/devops/hermes-relay-pair/SKILL.md in the repo (category subdirectory matching metadata.hermes.category: devops, per Hermes canonical layout).

Why:

Atomic updates — a single git pull inside ~/.hermes/hermes-relay/ updates the skill, the plugin (via pip install -e), and the docs in lock-step. There is no drift between the skill and the plugin module it depends on (plugin/pair.py).
No hub round-trip — the skill is an implementation detail of this project, not a generally-useful standalone skill. Publishing to the hub would be overkill and would make updates slower (hub publish step + hermes skills update on every change).
No hand-copy — cp -r skills/... ~/.hermes/skills/ was the old approach, but it means every update is a manual step and users can silently end up with stale skills. external_dirs is scanned fresh on every hermes-agent invocation, so there's nothing to forget.
Canonical category layout — matching the devops subdirectory to the metadata.hermes.category frontmatter lets users browse ~/.hermes/skills/ (and external dirs) by category and mirrors how upstream organizes its own skills.

Trade-off: Users need to trust the installer's YAML edit to ~/.hermes/config.yaml. The installer keeps the edit idempotent and only appends a single line under skills.external_dirs. If a user removes the line by hand, re-running the installer restores it.

Why not hermes skills update: That command targets skills installed via the hub (which writes to ~/.hermes/skills/). Skills discovered via external_dirs aren't tracked by the hub at all — hermes-agent enumerates them on startup. Update flow for external_dirs skills is whatever the external dir's own update mechanism is. For us, that's git pull.

Why the old skills/hermes-pairing-qr/ was deleted: It was the pre-plugin bash script era — hermes-pair as a shell script + a flat-file SKILL.md. The plugin now owns the QR generation (plugin/pair.py, pure Python, no qrencode dependency), the skill at skills/devops/hermes-relay-pair/ owns the slash-command surface, and the shell shim at ~/.local/bin/hermes-pair covers the script-friendly CLI entry point. Keeping the deprecated skill around would have been two sources of truth for the same operation.

Plugin CLI status (updated 2026-06-07): hermes-agent v0.8.0 had a top-level argparse gap where third-party PluginContext.register_cli_command() entries did not reach hermes <subcommand>. Current upstream now discovers plugin CLI registrations in hermes_cli/main.py, so hermes pair and hermes relay are the preferred shell entry points when the plugin is enabled. /hermes-relay-pair and the dashed hermes-pair shim stay as older-build and script compatibility paths until our supported baseline includes the upstream fix.

References:

install.sh — canonical installer
skills/devops/hermes-relay-pair/SKILL.md — skill definition (canonical category layout)
plugin/pair.py — shared implementation
~/.hermes/hermes-agent/website/docs/user-guide/features/skills.md — upstream skill distribution spec

14. Inbound Media via Plugin-Owned Relay Endpoint, Not Upstream Gateway (2026-04-11)

Decision: Agent-initiated media (screenshots, and any future file-producing tool) is delivered to the phone via a new loopback-register + bearer-fetch pair of routes on our own relay server (POST /media/register → GET /media/{token}), not by relying on hermes-agent's upstream extract_media() / send_document() machinery. Tools emit a MEDIA:hermes-relay://<token> marker in their chat response text; the phone parses the marker out of the SSE stream and fetches bytes out-of-band over authenticated HTTPS.

Why:

Upstream doesn't solve this on the HTTP API surface. Verification against ~/AppData/Local/Temp/hermes-agent/gateway/platforms/api_server.py shows APIServerAdapter.send() is an explicit no-op with the comment "API server uses HTTP request/response, not send()". _write_sse_chat_completion (api_server.py:651-757) streams raw stream_q deltas straight into SSE content chunks — it never invokes extract_media() and never routes deltas through GatewayStreamConsumer (which would at least strip MEDIA: tags via _MEDIA_RE at stream_consumer.py:188). The upstream extract_media() / send_document() calls at gateway/run.py:4570, 4747, 4349 are only reachable from non-streaming paths (background tasks, cron, batch) and push-style platform adapters (Telegram, Feishu, WeChat, Slack), all of which override send_document with real platform APIs. The pull-based HTTP adapter inherits the base class default, which falls back to self.send(chat_id, f"📎 File: {file_path}") — which is the no-op. So MEDIA:/tmp/... has always passed through our chat stream as literal text.
Inline base64 in tool output was the obvious alternative but blows up LLM context. A 1280×720 JPEG is ~135 KB base64, and every subsequent turn's context window has to re-ingest the bytes. Scales badly for video or multiple attachments per turn. Opaque tokens are ~25 chars and add essentially zero context cost.
No upstream PR in scope. Fixing this properly upstream would mean implementing send_document on APIServerAdapter (likely via a side-channel SSE event or a new attachment field on the chat-completion chunk shape). That's a community-scoped API change, and user explicitly wanted an in-plugin workaround, not a fork.
Our relay is already the right place. The plugin's relay server (plugin/relay/server.py) is a service we already own, already has HTTP routes (/health, /pairing/register), already uses SessionManager for bearer-auth'd channels, and already lives on the phone's trust boundary (paired via the same QR). Adding file-serving doesn't create a new security surface or a new credential store — it reuses both.

How it works:

Agent tool              Hermes API Server       Relay (:8767)         Phone
──────────              ─────────────────       ─────────────         ─────
android_screenshot()
  │
  ├─ write bytes to /tmp/...
  ├─ POST /media/register  ─────────────────▶   MediaRegistry
  │  (loopback only)                            ◀── {"token": "xyz"}
  │
  └─ returns "MEDIA:hermes-relay://xyz" ──▶ stream_q
                              │
                              └─ SSE content chunk ────────────────▶ ChatHandler
                                                                     scanForMediaMarkers
                                                                     parseAnnotationLine
                                                                     strips line, fires
                                                                     onMediaAttachmentRequested

                                              GET /media/xyz  ◀───── RelayHttpClient
                                              Authorization:         (Bearer = existing
                                                Bearer <session>      session_token)
                                              ──── bytes + ─────▶   MediaCacheWriter
                                              Content-Type          → cacheDir/hermes-media/
                                              Content-Disposition   → FileProvider URI
                                                                    → InboundAttachmentCard

Trust model:

/media/register is loopback-only — 403 for any request.remote other than 127.0.0.1 / ::1. Only a process running on the same host as the relay can inject files. Same pattern as /pairing/register.
/media/{token} requires Authorization: Bearer <session_token> — the same session token issued at pairing and stored in Android's EncryptedSharedPreferences. A LAN attacker who sniffed a token in cleartext insecure mode would also need a valid session token to use it, which they don't have unless they scanned the QR (same trust level as the WSS channel itself). Tokens are secrets.token_urlsafe(16) = 128 bits of entropy.
Path is never exposed to the client. The register endpoint holds the token → path mapping server-side. Clients present only the token on GET, so the fetch endpoint has zero path-traversal surface. Path sandboxing lives entirely on the register side: os.path.realpath() must resolve under an allowed root (default: tempfile.gettempdir() + HERMES_WORKSPACE or ~/.hermes/workspace/ + any RELAY_MEDIA_ALLOWED_ROOTS entries), the file must exist, must be a regular file, and must fit under the configured size cap.

Resource bounds:

TTL: 24 hours (default, RELAY_MEDIA_TTL_SECONDS) — chosen to match within-a-day session scrollback, the actual human use case. Going longer buys nothing since SessionManager is in-memory and any relay restart invalidates all tokens regardless. Going shorter breaks same-day scrollback.
LRU cap: 500 entries (default, RELAY_MEDIA_LRU_CAP) — prevents runaway memory/disk under screenshot spam. Eviction is oldest-first via OrderedDict.move_to_end on every get().
Per-file size cap: 100 MB (default, RELAY_MEDIA_MAX_SIZE_MB) — guards /media/register against accidentally registering a 10 GB file.

Fallback when the relay isn't running: The tool calls register_media() with a 5-second timeout. On any failure (connection refused, non-200, timeout) it logs a warning and returns the legacy bare-path form MEDIA:/tmp/.... The phone's ChatHandler parses this form via a second regex and, as of the bare-path-fetch update below, attempts a direct /media/by-path fetch — rendering the ⚠️ Image unavailable placeholder only if that fetch ALSO fails (relay offline, sandbox violation, file missing). No regression versus the pre-fix behavior; the marker never renders as raw text.

Addendum (2026-04-11, same day): LLM-emitted bare-path markers + /media/by-path. On-device testing exposed a gap: the above token-registration flow only covers markers emitted by tools (which we control and can wire through register_media()). But upstream hermes-agent/agent/prompt_builder.py:266 explicitly instructs the LLM in its base system prompt: "include MEDIA:/absolute/path/to/file in your response. The file..." — so the agent's free-form completions contain MEDIA:/abs/path markers too, and those never go through any tool code. Our tool-only fix left them as raw text.

Rather than patch the system prompt (out of scope — upstream-owned), the fix is a second relay route — GET /media/by-path — that takes an absolute path + bearer auth and streams the file directly. Same path sandbox as /media/register (via shared validate_media_path helper), same trust model (bearer session token). The phone's bare-path marker handler now attempts this fetch first and only falls back to the "unavailable" placeholder on network/404/403 failure. Result: LLM free-form MEDIA markers render inline with zero prompt-hacking.

Security implications of /media/by-path: Adding a path-addressable fetch widens what a paired phone can request by one degree — it can now read any file in the allowed-roots whitelist without host-local process cooperation, where previously it needed a tool to pre-register via /media/register. This does NOT widen the trust boundary because:

The allowed-roots whitelist is the same. Files outside tempfile.gettempdir() / HERMES_WORKSPACE / RELAY_MEDIA_ALLOWED_ROOTS are still unreachable.
On Linux /tmp is already world-readable to every process running as the same user, so the relay exposing it over bearer-auth'd HTTP is no new disclosure to an attacker who could already connect a paired phone.
Bearer auth still requires a valid session token issued at pair time. Unpaired peers get 401.
The path never leaks out of the sandbox via symlinks — os.path.realpath() resolves symlinks before the whitelist check, so a symlink inside the whitelist pointing outside it is rejected.

The bare-path fetch is therefore safe as long as operators treat the allowed-roots whitelist as "directories the paired phone is allowed to read." If that's wrong for a given deployment, tighten RELAY_MEDIA_ALLOWED_ROOTS rather than disabling the endpoint.

Trade-off: session replay across relay restarts doesn't work. If the user scrolls back into a session from yesterday and the relay has restarted since, the tokens stored in the persisted message text are stale and /media/{token} returns 404. The phone renders a FAILED placeholder. Acceptable for MVP — the alternative (phone-side persistent cache indexed by token or content hash) is meaningful new plumbing and the right layer for durability, but out of scope. Filed as a follow-up in DEVLOG.

Trade-off: auto-fetch-threshold slider is persisted but not enforced today. The user-facing Settings → Inbound media section exposes a "auto-fetch threshold" knob (0–50 MB), but the actual fetch path only checks the cellular toggle + the max-size cap. Real threshold enforcement would need either a HEAD preflight (to reject before downloading) or accept the post-hoc waste. Kept the slider as a forward-compatibility placeholder; actual wiring is a follow-up.

Alternative rejected: inline base64 in tool output. Would be simpler (no new endpoint, no phone-side fetcher) but every attachment would bloat the LLM context window on every subsequent turn. For a single screenshot that's ~135 KB of base64; for any non-trivial use case the costs compound. User explicitly rejected this during the design discussion.

Alternative rejected: patch upstream api_server.py. Would be architecturally cleaner — route deltas through GatewayStreamConsumer so _MEDIA_RE strips the tags, then implement send_document via a side-channel SSE event. But it's a community-scoped API change, and user explicitly scoped us to "work with existing/documented methods ideally; if we have to work-around we need to follow our existing path within our plugin/etc." Recorded in docs/upstream-contributions.md as a possible future PR.

References:

plugin/relay/media.py — MediaRegistry, _MediaEntry, MediaRegistrationError
plugin/relay/server.py → handle_media_register, handle_media_get
plugin/relay/client.py → register_media() (stdlib urllib, 5s timeout)
plugin/tools/android_tool.py::android_screenshot — first consumer
app/src/main/kotlin/.../network/RelayHttpClient.kt — phone fetcher
app/src/main/kotlin/.../network/handlers/ChatHandler.kt → scanForMediaMarkers, finalizeMediaMarkers
app/src/main/kotlin/.../ui/components/InboundAttachmentCard.kt — Discord-style rendering

15. Pairing + Security Architecture: grants, user-chosen TTL, Keystore, TOFU, Paired Devices (2026-04-11)

Decision: Replace the minimal pairing model (one-shot code → fixed-30-day session token → no channel separation → EncryptedSharedPreferences storage) with a layered architecture built around four ideas:

User chooses session TTL at pair time — 1 day / 7 days / 30 days / 90 days / 1 year / never expire. The Android TTL picker dialog always opens on QR scan so the user explicitly confirms. Defaults depend on transport: wss or Tailscale → 30d; plain ws → 7d. (Both wss and Tailscale are treated as secure transports here — but for different reasons: wss is TLS, while Tailscale's security comes from WireGuard end-to-end encryption, not TLS. See user-docs/architecture/connection-security.md.) Never-expire is ALWAYS selectable with an inline warning — per operator direction, trust the user's intent rather than gating on secure-transport detection.
Per-channel grants — one session token, separate expiries for chat / terminal / bridge; later releases added tui and split voice grants (voice:config, voice:stt, voice:tts). Blast-radius-heavy channels can have shorter caps, and all grants are clamped to the session lifetime. Chat runs through the hermes-agent API server rather than the relay, so the chat grant is informational only (used by the phone UI to show scope).
Hardware-backed token storage with graceful fallback — KeystoreTokenStore requests StrongBox-backed keys via setRequestStrongBoxBacked(true) on Android 9+ devices that advertise FEATURE_STRONGBOX_KEYSTORE. Falls back to the existing LegacyEncryptedPrefsTokenStore (TEE-backed EncryptedSharedPreferences) on older devices or when the Keystore path throws. Migration is one-shot and lossless — users never lose a session to an app upgrade.
TOFU cert pinning with explicit reset on re-pair — CertPinStore records SHA-256 SPKI fingerprints per host:port on the first successful wss connect. Subsequent connects build an OkHttp CertificatePinner from the stored pin. A user-initiated QR re-pair (applyServerIssuedCodeAndReset(code, relayUrl)) wipes the pin for the target host — re-pair is explicit consent to potentially-new cert material. Plaintext ws:// short-circuits pinning entirely.

Supporting infrastructure:

Device revocation UI — new Paired Devices screen on the phone, backed by GET /sessions (list all paired devices, tokens masked to first 8 chars) and DELETE /sessions/{token_prefix} (revoke). Self-revoke is allowed and flagged via revoked_self: true so the phone can wipe local state and redirect to pairing.
QR payload v2 + HMAC signing — payload version bumped from 1 to 2 when Relay metadata fields are present (ttl_seconds, grants, transport_hint). Signed with HMAC-SHA256 using a host-local secret at ~/.hermes/hermes-relay-qr-secret (32 bytes, 0o600, auto-created). Phone parses and stores the sig field but does NOT verify it yet — full verification requires a secret-distribution mechanism we don't have defined. The server-side infrastructure is in place so phone-side verification can land in a follow-up.
Rate-limit clear on pair — /pairing/register now calls RateLimiter.clear_all_blocks() on success. An operator explicitly re-pairing wants a clean slate; the stale rate-limit state otherwise blocks the legitimate re-pair attempt for 5 minutes. This was an actual bug biting the operator at the start of this session.
Transport security UI — badge component with three states (secure green 🔒 / insecure amber with reason / insecure unknown red), three sizes (chip / row / large). Rendered in Settings Connection section, Session info sheet, and on each Paired Device card.
Insecure ack dialog — first-time toggle-on shows a plain-language threat-model dialog with a reason picker (LAN only / Tailscale or VPN / Local dev only). Reason persists for display purposes; does NOT gate anything per the operator's trust-model direction.
Tailscale detection — TailscaleDetector checks for tailscale0 interface + 100.64.0.0/10 CGNAT addresses + .ts.net hostnames in the relay URL. Shown as a "Tailscale detected" green chip; does NOT auto-change any defaults — informational only.
Phase 3 bidirectional pairing stub — new POST /pairing/approve route, loopback-only, same shape as /pairing/register. Marked with # TODO(Phase 3): — full flow needs a pending-codes store so operators review rather than rubber-stamp. The route and wire shape are committed now so the phone side has something to target when bridge lands.

Why this split:

Grants on a single token (not multiple tokens) — one WSS connection, one auth envelope, one session lookup. Per-channel expiry is checked at channel message dispatch time via Session.channel_is_expired(name). Simpler to reason about than multiple parallel tokens, and the phone only needs one storage slot.
math.inf for never-expire — represents "truly unbounded" in code, serializes to null on the wire (JSON doesn't have an infinity literal, and null maps cleanly to Kotlin's nullable Long?). canonicalize() uses allow_nan=False so accidentally trying to sign a payload with a raw math.inf crashes loudly — callers must explicitly emit None/0. Prevents silent serialization bugs.
Metadata on pairing entries is host-authoritative — when the host operator runs hermes pair --ttl 7d and the phone sends ttl_seconds=30d in the auth envelope (because the user picked a different value on the TTL dialog), the host value wins. If host metadata is absent, the relay uses bounded server defaults; network clients never author session lifetime or grants. The legacy anonymous POST /pairing code-mint route is intentionally not registered, so every accepted code originates from a loopback-only operator flow.
Bearer session-policy changes are monotonic and self-only — a Relay bearer may use PATCH /sessions/{token_prefix} only for its own token and only to shorten its session or grants. It cannot add grant names, lengthen a grant, switch to never-expire, or modify another session. Those authority-increasing changes require a fresh operator-approved pairing flow.
Token prefix (not full token) in /sessions responses — a caller already holds their own full token; they should never see another session's full token. First 8 chars are enough to identify devices in a practical deployment (one operator, 1-3 phones) and enough entropy to avoid collisions. Collisions return 409 with the match count.
Always open the TTL picker (no skip) — even when the QR carries an operator-chosen TTL, the dialog opens with that value preselected. The user is always in the loop for the trust decision. A future "don't ask again if QR specifies a TTL" toggle is a plausible refinement but not in this cut.

Trade-offs:

Any paired phone can revoke any other paired device. A compromised phone could lock out all other devices — DoS territory. Acceptable for the single-operator / 1-2 phones deployment model; multi-user deployments will need a per-device role model (admin vs user) with per-role grant caps.
QR signatures aren't verified on the phone. Raises the bar for QR tampering on the server side (attacker can't inject a fake code via a modified photo if the server requires a valid sig), but the phone currently trusts any signature as long as the parse succeeds. Full verification is a follow-up.
TOFU cert pinning doesn't protect the first connect. By definition, trust-on-first-use accepts whatever certificate is present on the initial handshake. Protects against MITM of subsequent connects only. Acceptable for LAN / Tailscale / VPN deployments where the first connect happens over a trusted path.
StrongBox is opportunistic. Older Android devices or devices without StrongBox hardware fall back to TEE-backed EncryptedSharedPreferences. Still strong, but the attack surface is larger than hardware-backed.

Alternatives rejected:

Separate tokens per channel — clean model but triples the storage + auth flow. One-token-with-grants is the right abstraction.
Gating never-expire on secure transport detection — operator explicitly requested "don't force check, just allow based on user intent." User agency over policy.
Full QR signature verification on the phone in this cut — requires a secret distribution mechanism (pre-shared key? enrollment token? OAuth?) that isn't yet designed. Server-side signing is the prerequisite and it's in place.

References:

plugin/relay/auth.py — Session, PairingMetadata, SessionManager, PairingManager, RateLimiter.clear_all_blocks
plugin/relay/qr_sign.py — canonicalize, sign_payload, verify_payload, load_or_create_secret
plugin/relay/server.py — handle_pairing_register, handle_pairing_approve, handle_sessions_list, handle_sessions_revoke, _detect_transport_hint
plugin/pair.py — build_payload(sign=True), parse_duration, parse_grants, --ttl / --grants flags
app/src/main/kotlin/.../auth/SessionTokenStore.kt — Keystore / legacy fallback
app/src/main/kotlin/.../auth/CertPinStore.kt — TOFU pinning
app/src/main/kotlin/.../auth/PairedSession.kt — phone-side session metadata
app/src/main/kotlin/.../ui/components/SessionTtlPickerDialog.kt — TTL picker
app/src/main/kotlin/.../ui/components/TransportSecurityBadge.kt — secure / insecure indicator
app/src/main/kotlin/.../ui/components/InsecureConnectionAckDialog.kt — first-time insecure consent
app/src/main/kotlin/.../ui/screens/PairedDevicesScreen.kt — list + revoke UI
app/src/main/kotlin/.../util/TailscaleDetector.kt — informational detection

CI/CD Patterns (from ARC)

Adopting from ARC's workflow patterns:

CI workflow: Lint (ktlint) → Build (debug + release matrix) → Test → Upload APK artifact
Release workflow: Tag v* → version validation (build.gradle.kts vs tag) → signed APK → GitHub Release
Concurrency groups: Cancel in-progress CI on new push to same branch
Dependabot: Auto-merge minor/patch dependency updates

16. Runtime API Server Patch via .pth Bootstrap (2026-04-12)

Context: The Android app depends on API-server routes for session history, profile/config metadata, skills, and memory-backed UI. Upstream core moved in focused pieces rather than one large frontend API patch: PR #33134 now covers the canonical /api/sessions/* surface, and PR #33016 covers read-only /v1/skills + /v1/toolsets. Config, memory, legacy skill detail/toggle, available-models, and slash-command preprocessing still remain compatibility routes in this repo until core exposes stable equivalents or the local UI no longer depends on them. Without the bootstrap, users on older vanilla upstream builds lose session browsing, metadata-backed settings, and history-on-restart behavior.

We considered four options:

A. Stay fork-only. Reject vanilla upstream users until the relevant core API surfaces land. Penalises onboarding.
B. Read-only sessions browser via plugin relay. Add GET /api/sessions/* to the relay at port 8767. Forces the client to know which URL each operation goes to.
C. Full parity by porting all 800 lines onto the plugin relay. Same architectural pollution as B, plus duplicates ~250 lines of chat-stream handler with cross-cutting _create_agent / run_conversation dependencies that the fork may have implicitly modified.
D. Runtime injection via Python interpreter startup hook. Ship a .pth file in the venv site-packages that imports a bootstrap module, which installs a sys.meta_path finder for aiohttp.web. When the gateway eventually imports aiohttp.web, our finder wraps the loader and replaces web.Application with a thin subclass. The subclass overrides __setitem__ to detect app["api_server_adapter"] = self (the line in upstream's connect() that gives us a reference to the adapter while the router is still mutable). At that point we register only missing method/path handlers directly onto the same router the gateway is in the middle of populating.

Decision: D, scoped to management endpoints only. The chat-stream handler is intentionally NOT injected — chat goes through vanilla upstream Hermes /v1/runs, which already emits structured tool.started/tool.completed events. This avoids touching _create_agent / run_conversation (the fork's riskiest cross-cutting dependencies) and is arguably an upgrade — /v1/runs has live tool events whereas the sessions chat-stream path required a post-stream message-history reload to render tool cards.

Why this is the right answer despite being a clever hack:

Zero modifications to hermes-agent's filesystem. git pull / hermes update see no local changes, so they always work cleanly. The patch lives entirely in hermes_relay_bootstrap/ inside our own repo.
Single-file containment of all ported logic. _handlers.py is 500 lines of straight-line aiohttp handler code with explicit adapter parameters (closures, not bound methods). Easy to audit, easy to delete.
Feature detection by method/path, not broad route family. Native upstream routes win one method/path at a time. This matters because #33134 landed /api/sessions/* before core had stable config/memory/legacy skill APIs; the bootstrap must not skip those remaining compatibility routes just because a sessions route exists.
Trust model already established. The user installed our plugin into their hermes-agent venv. They've already consented to having the plugin import hermes-agent internals (it does this for relay tools, voice endpoints, media registry). Monkey-patching aiohttp.web.Application is in the same trust bucket.
Surface-by-surface removal. With #33134/#33016 merged, sessions and read-only skill lists should go quiet automatically on current upstream. Config, memory, legacy skill detail/toggle, available-models, and command preprocessing remain until their native replacements exist or the local UI stops depending on them. Full bootstrap deletion happens only after every compatibility route group has a stable core equivalent or a deliberate local removal.
/v1/runs remains the fallback run-control path. Native /api/sessions/{id}/chat/stream is now the preferred session-persisted chat path when advertised. /v1/runs still matters for async run lifecycle/control and for older builds without native sessions chat.

The Android client adapts via streamingEndpoint = "auto". ServerCapabilities returned by HermesApiClient.probeCapabilities() captures per-endpoint presence (sessionsApi, sessionsChatStream, runs, portable, healthy). ConnectionViewModel.resolveStreamingEndpoint() collapses "auto" to "sessions" when native session chat is present, then falls back to OpenAI-compatible completions or runs according to the probe. The setting still supports manual "sessions" / "completions" / "runs" overrides for debugging.

Risks accepted:

Plugin load order — verified: .pth files are processed by Python's site module BEFORE any application code runs, so our import hook is in place before hermes-agent imports aiohttp.web.
Upstream refactor of the route-registration block in connect() — handled by feature detection on route path. Worst case: bootstrap logs a warning and gateway runs without injected routes. The Android client falls back to /v1/runs automatically.
Upstream symbol removal in tools/skills_tool.py — skills_categories was removed in upstream commit 8d023e43 as dead code. The bootstrap no longer imports or re-injects it. The app uses /api/skills?category= for category filtering; the standalone /api/skills/categories endpoint was never called by the app. The bootstrap stays in sync with upstream by not re-introducing removed symbols.
Editable pip install doesn't ship .pth files reliably — verified empirically (test in /tmp/pth-test on the server during scoping). Solved by install.sh copying the .pth directly into the venv's site-packages/ after pip install -e.

File locations:

hermes_relay_bootstrap/__init__.py — installs the meta_path finder (~30 lines)
hermes_relay_bootstrap/_patch.py — _AioHttpWebFinder, _PatchingLoader, _PatchedApplication, _maybe_register_routes (~170 lines)
hermes_relay_bootstrap/_handlers.py — 14 ported handlers + helpers (~500 lines)
hermes_relay_bootstrap.pth — single line: import hermes_relay_bootstrap
install.sh step 2 — copies the .pth into the venv site-packages

Removal path is now per surface:

Sessions: done (2026-07-08, HRUI-002). The sessions CRUD/messages/fork handlers were removed from the bootstrap with no pre-#33134 fallback kept; native /api/sessions/* (#33134) is the only provider. Older core builds degrade via the client capability probe to /v1/chat/completions//v1/runs.
Read-only skills/toolsets: done (2026-07-08, HRUI-002). The legacy GET /api/skills list handler was removed; clients use native /v1/skills and /v1/toolsets (#33016). Legacy detail (/api/skills/{name}) and the 501 toggle stub remain — no native equivalent exists.
Config/memory/available-models/session search: remove those compatibility handlers only after stable core APIs exist or the dependent Android surfaces are redesigned. While they remain, session search is offloaded through upstream AsyncSessionDB (or asyncio.to_thread on older Hermes), and each memory mutation request resets the optional upstream per-turn consolidation failure budget so independent REST calls cannot consume one shared budget.
Slash middleware: remove after native API-server slash preprocessing exists.
Full cleanup: delete hermes_relay_bootstrap/, delete hermes_relay_bootstrap.pth, remove the .pth install block, and update local agent docs only after all compatibility groups have native replacements.
The Android client probeCapabilities() and streamingEndpoint = "auto" plumbing stays — it's permanent infrastructure that handles mixed-version deployments.

17. Wake-lock wrapping for gesture dispatch + multi-window ScreenReader + three-tier tapText cascade (2026-04-13)

Context: While scoping the v0.4 bridge feature expansion (see docs/plans/2026-04-13-bridge-feature-expansion.md), three reliability gaps in the existing Phase 3 ActionExecutor / ScreenReader surfaced as hard prerequisites before any of the ten Tier A tools were worth adding:

Gestures silently fail when the phone screen is off. ActionExecutor.tap / swipe / typeText all dispatch into GestureDescription via the accessibility service. When the screen is off the gesture completes cleanly at the framework level — GestureResultCallback.onCompleted fires — but nothing actually happens on-device. The agent thinks the tap landed and proceeds with the next step. Biting Bailey repeatedly during android_navigate loops that sat idle between iterations.
ScreenReader misses every overlay, popup menu, notification shade, and split-screen secondary window. The existing implementation calls service.rootInActiveWindow and walks a single tree. Android exposes all visible windows via AccessibilityService.windows; system overlays and popup menus live in separate windows from the foreground activity. Any agent flow that needed to see a popup dialog (date pickers, confirmation dialogs, context menus, notification shade content) hit a blank tree.
ActionExecutor.tapText fails on clickable text inside non-clickable wrappers. The single-shot implementation finds a text node via findNodeBoundsByText and calls performAction(ACTION_CLICK) on that node. Real-world Android apps (Uber, Spotify, Instagram, Tinder — verified by comparison-passing raulvidis/hermes-android against a matrix of target apps) wrap clickable content in non-clickable TextView/ImageView ancestors. ACTION_CLICK on the text node itself is a no-op. The fallback approach is to walk up the parent chain to the nearest clickable ancestor, and if nothing clickable is found, fall back to a coordinate tap at the node's bounds center.

Decision: Adopt all three patterns before shipping the v0.4 tool surface. They're applied to the existing Phase 3 code so every new Tier A/B/C tool inherits them for free.

Pattern 1 — WakeLockManager.wakeForAction (A8). New object WakeLockManager at app/src/main/kotlin/com/hermesandroid/relay/power/WakeLockManager.kt. Exposes suspend fun <T> wakeForAction(block: suspend () -> T): T. Uses PowerManager.PARTIAL_WAKE_LOCK (the non-deprecated modern successor to SCREEN_BRIGHT_WAKE_LOCK), with:

Ref counting. A private lockCount: Int tracks concurrent scopes; the underlying WakeLock.release() only fires when the count hits zero. Nested calls don't release each other prematurely.
Hard 10-second timeout. Passed to newWakeLock().acquire(timeoutMillis = 10_000) as a battery safety rail. Long-held wake locks are a notorious source of drain bugs; 10s is longer than any single gesture needs and far shorter than any plausible leak window.
Try/finally release. The wakeForAction body is wrapped in try { block() } finally { release() } so exceptions thrown from inside the action still release the lock.

ActionExecutor.tap / tapText / typeText / swipe / scroll / longPress / drag (the last two new in v0.4) are wrapped in WakeLockManager.wakeForAction { ... }. Read-only calls are deliberately NOT wrapped — readScreen, findNodes, describeNode, screenHash, diffScreen, currentApp, clipboardRead/Write, mediaControl. These don't need the screen on; wrapping them would waste battery and add latency to polls. Requires android.permission.WAKE_LOCK in app/src/main/AndroidManifest.xml.

Pattern 2 — Multi-window ScreenReader (P1). Change ScreenReader.readCurrentScreen (and any helper using rootInActiveWindow) to iterate service.windows.mapNotNull { it.root } and walk each per-window tree. The node-ID scheme is updated to prefix every stable ID with w<windowIndex>: so IDs remain unique across the merged output (w0:42 for the main activity, w1:7 for a popup menu). Per-iteration try/finally blocks recycle each AccessibilityNodeInfo as the walker descends — .parent and .getChild(i) both return fresh instances that leak if not recycled. A single-window fallback kicks in when service.windows is empty, which happens on the googlePlay flavor without flagRetrieveInteractiveWindows (the conservative accessibility config required by Play Store policy review).

Pattern 3 — Three-tier tapText cascade (A9). Rewrite ActionExecutor.tapText:

Find node by text across all windows (benefits from P1). If node.isClickable → performAction(ACTION_CLICK). Return ActionResult(ok=true, data="direct").
Otherwise walk up the parent chain, capped at 8 levels, looking for a clickable ancestor. Each .parent call returns a fresh node that must be recycled before the loop reassigns. If any ancestor is clickable → performAction(ACTION_CLICK) on it. Return ActionResult(ok=true, data="parent").
Otherwise capture the original node's getBoundsInScreen() center before recycling, and fall back to coordinate tap(cx, cy) via the existing gesture path (which itself runs under WakeLockManager.wakeForAction). Return ActionResult(ok=true, data="coords").

The data field tells the activity log and agent trace which tier succeeded, which is actually useful debugging info when a tap lands unexpectedly.

Consequences:

Wake-lock reliability: closes the idle-screen silent-failure class of bugs for every gesture-dispatching tool, present and future. Adds minimum complexity per action (one wakeForAction { ... } wrapper call). The battery cost is tightly bounded by the 10s timeout and the PARTIAL lock level.
Multi-window visibility: accessibility tree size grows modestly (typical overhead: 1–3 extra windows at ~20 nodes each — the notification shade and system UI). MAX_NODES=512 cap in ScreenReader absorbs it. The node-ID prefix change is a breaking change to the ID format, but IDs are opaque and session-local — agents pass them back within a single turn, never persist them, so the breakage is invisible to callers.
tapText success rate: validated against raulvidis's target-app matrix (Uber, Spotify, Instagram, Tinder, Maps, Settings). Direct-click path still wins on well-structured accessibility trees (Google Maps, Android Settings). Parent-walk catches the ~60% of cases where the clickable wrapper is within 1–2 levels of the text node. Coordinate fallback is the last-resort and backs ~10% of cases on the hardest apps.
android_navigate throughput: Pattern 1 + Pattern 3 together eliminate the two biggest "tap fails for no obvious reason" failure modes inside the vision-driven navigate loop. Combined with A5 android_screen_hash as a cheap change-detection primitive (covered in the tool surface but not a pattern ADR), navigate iterations get faster and more reliable without changing the vision-model integration.
Test coverage: Pattern 1 is tested implicitly via the existing ActionExecutor tests — the wake-lock wrap is transparent to the gesture completion path. Pattern 2 adds multi-window fixtures to ScreenReader tests where available; the single-window fallback path ensures the unit tests that use the old fixture shape still pass. Pattern 3 is hard to unit-test cleanly (accessibility node traversal is not easy to mock) so it's primarily verified via instrumentation tests on a real device.

Alternatives rejected:

SCREEN_BRIGHT_WAKE_LOCK instead of PARTIAL_WAKE_LOCK. Deprecated since API 17. The agent doesn't need the screen visible — it just needs the input path warm. PARTIAL_WAKE_LOCK is the modern, non-deprecated choice.
Single-root with window enumeration as a fallback. Leaks overlays when the overlay is the primary point of interaction (e.g. a full-screen dialog). Iterating all windows up-front is simpler and the overhead is bounded.
Only walk the parent chain, no coordinate fallback. Fails on the ~10% of apps where nothing in the chain is clickable but the bounds are valid. Coordinate tap is the last-ditch option and is cheap to try.
Wake-lock the entire bridge command handler. Over-broad — every android_screen call would grab the lock. Scope it to the gesture path only.

References:

app/src/main/kotlin/com/hermesandroid/relay/power/WakeLockManager.kt (new in v0.4)
app/src/main/kotlin/com/hermesandroid/relay/accessibility/ActionExecutor.kt — tap / tapText / typeText / swipe / scroll / longPress / drag wrapped in wakeForAction, tapText cascade implementation
app/src/main/kotlin/com/hermesandroid/relay/accessibility/ScreenReader.kt — service.windows iteration, w<windowIndex>:<sequentialIndex> node-ID scheme
app/src/main/kotlin/com/hermesandroid/relay/accessibility/ScreenHasher.kt (new in v0.4 — SHA-256 content fingerprint primitive, covered in docs/spec.md §6.4.2 but not a separate ADR since it's additive rather than a cross-cutting pattern)
docs/plans/2026-04-13-bridge-feature-expansion.md — A8, A9, P1 units
docs/spec.md §6.4.2 — architectural-patterns subsection

18. Unattended-access visibility: in-app banner vs. system-overlay chip (2026-04-17)

Context: v0.4.1's unattended-access mode (sideload-only) lets the agent wake the screen and dismiss the keyguard while the user is physically away from the phone. That's a meaningful expansion of agent authority — the user should have a prominent, always-on affordance telling them the mode is live so they can disable it at a glance. The existing v0.4.1 unattended work shipped the WindowManager BridgeStatusOverlayChip in an "amber Unattended ON" variant to cover this, but that surface only renders when the Hermes-Relay app is backgrounded (the floating chip is drawn via SYSTEM_ALERT_WINDOW and the user doesn't see their own app's overlay on top of itself). While the user is IN Hermes-Relay — on any tab — there was no always-on affordance at all; they had to scroll the Bridge tab down to the Unattended Access card to check status.

Decision: Ship two complementary affordances with distinct visibility windows, kept deliberately separate rather than trying to unify them.

UnattendedGlobalBanner (in-app) — Compose-drawn 28dp amber strip at the top of RelayApp's scaffold, rendered unconditionally on every tab when masterEnabled && unattendedEnabled && BuildFlavor.isSideload. Theme-aware colours (amber-on-dark / dark-amber-on-pale-amber), pulsing dot, chevron → navigates to Bridge. Handles the app-foregrounded case.
BridgeStatusOverlayChip (existing) — WindowManager overlay via SYSTEM_ALERT_WINDOW, renders the amber "Unattended ON" variant when unattended is on. Forced visible whenever unattended is on, independent of the user's regular "show status overlay" preference. Handles the app-backgrounded case.

Why split, not unified:

OS constraint. WindowManager overlays don't render on top of the owning app's UI in the foregrounded state — Android deliberately hides a package's own overlays when that package is itself foreground. One surface cannot cover both states.
Different visual budgets. The in-app banner sits inside RelayApp's scaffold and can occupy a full-width strip with text + chevron. The system chip is a floating pill with strict size limits (~180dp wide) to avoid obscuring content in the host app. Text copy differs accordingly.
Different z-order semantics. The in-app banner composes with the rest of the app's UI and respects tab navigation. The system chip is user-positionable via drag and persists across app switches. Unifying them would force the weaker of the two behaviours onto the stronger surface.
Different dismissibility. The in-app banner is non-dismissible and always renders when the preconditions are met (it's a fixed affordance, not a notification). The system chip is user-draggable but non-tap-dismissable while unattended is on. Both fail-closed.

What the user sees (sideload only, unattended on):

On any Hermes-Relay tab → amber banner pinned to the top of the scaffold. Tap the chevron to jump to the Bridge tab and disable.
App in the background (any other app foregrounded, or home screen) → floating amber "Unattended ON" chip in the system overlay. Tap to return to Hermes-Relay.

Consequences:

Correct coverage. There is now no app-state window in which the user can miss that unattended mode is on.
Two places to maintain. Copy changes need to ripple to both surfaces. Mitigated by keeping the strings short and similar but not identical (the in-app banner has more room and uses "agent can wake and drive this device"; the system chip uses the tighter "Unattended ON"). Both surfaces are driven by the same unattendedEnabled StateFlow, so state can't drift.
Theme story differs. The in-app banner honours the Material 3 theme (light / dark / dynamic colour). The system chip uses a fixed amber for visibility on arbitrary host apps. This is a feature, not a defect — the system chip has no knowledge of the underlying app's theme and amber on amber would vanish.

Alternatives rejected:

One unified overlay chip, rendered even while the app is foregrounded. Requires a custom always-foreground overlay mode via TYPE_APPLICATION_OVERLAY + manual z-order tricks; Android actively fights this on modern targets. Cost / benefit not worth it when a Compose-drawn banner solves the foreground case trivially.
Persistent foreground notification as the only affordance. We already have one (BridgeForegroundService), and it's the authoritative in-sight kill switch. But (a) it lives in the shade until the user pulls it down and (b) it's for the master-toggle-on state, not for unattended specifically. A per-mode notification channel was considered and rejected as notification shade clutter.
In-app banner only, drop the system chip. Regresses the backgrounded-app case. The system chip is the only surface that can tell the user at a glance that unattended is on while their phone is running a different app — exactly the case unattended is designed for.

References:

app/src/main/kotlin/com/hermesandroid/relay/ui/components/UnattendedGlobalBanner.kt (new in v0.4.1 polish pass)
app/src/main/kotlin/com/hermesandroid/relay/ui/RelayApp.kt — banner mount point at scaffold top
app/src/main/kotlin/com/hermesandroid/relay/bridge/BridgeStatusOverlay.kt — BridgeStatusOverlayChip amber variant (shipped with the initial v0.4.1 unattended work)
docs/spec.md §5 Bridge Tab — user-facing description

19. Multi-Connection Support — one app, many Hermes servers (2026-04-18)

2026-07-18 amendment: ADR 38 replaces the endpoint-shaped identity described below. A connection still represents one Hermes installation, but its stable ID is independent of apiServerUrl; Dashboard/Gateway is the standard surface and API/Relay endpoints are optional capabilities. The original model and migration notes remain here as implementation history.

Terminology note (2026-04-18): earlier drafts of this design called these "profiles" — that's been renamed to "Connection" to avoid collision with Hermes's in-config agent profiles concept (agent.profiles in config.yaml defines name + model + description for each agent personality). A follow-up pass will introduce the agent-profile layer on top of the connection layer described here.

Decision: Formalize the implicit single-server connection as a first-class Connection entity, with a ConnectionStore that holds N connections in DataStore and persists the active one. Switching connection is a heavy context swap: cancel in-flight SSE, disconnect relay WSS, rebuild HermesApiClient, update URL StateFlows (which RelayHttpClient/RelayVoiceClient providers already re-read lazily), rebuild AuthManager against a connection-scoped EncryptedSharedPreferences file, reconnect, reprobe capabilities, reload sessions/personalities, restore per-connection last-active session.

Why:

Users with multiple Hermes installs (home + work, dev + prod, multiple NAS hosts) want to switch targets without wiping pairing state or re-running onboarding.
A "connection" in this app's terminology is a separate gateway instance — there is no /api/connections endpoint to enumerate. The honest model is "connection = baseUrl + bearer + pairing record". This also matches how Discord achieves multi-agent-in-one-channel: each "agent" is a distinct bot backed by its own gateway.
Personalities (Decision 8) are orthogonal and remain per-connection — each server exposes its own personality map via /api/config.

How:

Connection model: UUID id, editable label (default = hostname), apiServerUrl, relayUrl, tokenStoreKey (EncryptedSharedPreferences file name), pairedAt, lastActiveSessionId, transportHint, expiresAt. Serialized as a JSON array in DataStore key connections_v1; active connection in active_connection_id.
Migration: on first launch of the new version, ConnectionStore.migrateLegacyConnectionIfNeeded() seeds connection 0 with the existing hermes_companion_auth_hw file as its store — zero re-pair, zero token migration, fully transparent to the user. Additionally, the DataStore key rename (profiles_v1 → connections_v1, active_profile_id → active_connection_id) migrates in-place on first launch after the rename pass.
Auth layer: SessionTokenStore.tryCreate() accepts a prefsName parameter; AuthManager gains a connectionId constructor parameter and picks the store file via Connection.buildTokenStoreKey(id). CertPinStore is unchanged — pins are already keyed by host:port and correctly shared across connections targeting the same server.
Network rebind: RelayHttpClient and RelayVoiceClient take provider lambdas that read URL + token at call time, so updating the backing MutableStateFlows is sufficient — no client recreation. HermesApiClient is recreated (private OkHttp pool + SSE factory), using the existing rebuildApiClient() path in ConnectionViewModel.
UI: top-bar connection chip → ConnectionSwitcherSheet bottom sheet with radio list + "Manage connections…" link. Screen.ConnectionsSettings hosts card CRUD: rename (inline), re-pair (reuses ConnectionWizard with connectionId nav arg), revoke, remove.
Remove semantics: ConnectionStore.removeConnection() calls context.deleteSharedPreferences(tokenStoreKey) to wipe the connection's auth material. Cert pin survives in the global TOFU map keyed by host:port so re-adding the same server is still trusted.

Scope — what's per-connection vs. global:

Per-connection	Global
API baseUrl + bearer	Theme, dev-mode toggles
Sessions, messages, search	Bridge safety vocabulary (blocklist, destructive verbs, timer duration)
Bridge capability policy (ADR 63)	Status-overlay presentation
Personalities (`/api/config`)	Feature flags / DataStore overrides
Skills, memory	Notification companion enabled/disabled
Relay WSS endpoint + cert pin (shared by host)	Keystore itself (one keystore, many entries)
Voice transcribe/synthesize endpoint
Bridge command target
Last-active session ID

Amended by ADR 63 — Bridge capability grants are per-connection. The blocklist, destructive vocabulary, confirmation timeout, and timed-window duration remain phone-wide safety preferences. Actual Always/Never/Timed authority is keyed by Connection ID, so server A cannot inherit grants made for server B.

Trade-off — "both agents in one channel" (Discord-style) deferred: a unified chat view showing interleaved messages from two connections is possible but semantically fraught: sessions, memory, and tool calls don't merge on the server side, so the unified view would be purely client-side theater. Deferred until there's a concrete use case; v1 users switch contexts with one tap and carry on.

Upstream opportunity: a real /api/profiles endpoint that returns [{id, name, endpoint, model}] from a single config would let one hermes install expose multiple agent profiles without the user running multiple daemons. That's the upcoming Pass 2 work on top of this connection layer — note that upstream's concept ("agent profile") is different from the multi-server concept this decision records ("connection"). Filed alongside /api/commands as a future contribution in docs/upstream-contributions.md.

References:

app/src/main/kotlin/.../data/ConnectionData.kt — Connection + helpers
app/src/main/kotlin/.../data/ConnectionStore.kt — StateFlows + CRUD + migration
app/src/main/kotlin/.../auth/AuthManager.kt — connectionId ctor. (The sessionLabels field mentioned in earlier drafts has been replaced by agentProfiles: StateFlow<List<Profile>> in §21.)
app/src/main/kotlin/.../viewmodel/ConnectionViewModel.kt — switchConnection() orchestration
app/src/main/kotlin/.../ui/components/ConnectionSwitcherSheet.kt — bottom-sheet switcher
app/src/main/kotlin/.../ui/screens/ConnectionsSettingsScreen.kt — CRUD screen

20. Dashboard plugin: single plugin with internal tabs + pre-built IIFE bundle (2026-04-18)

Context: The upstream Dashboard Plugin System landed on hermes-agent's axiom branch in three commits (01214a7f plugin system, 3f6c4346 theme, 247929b0 OAuth providers). The gateway now scans ~/.hermes/plugins/<name>/dashboard/manifest.json at startup and exposes registered plugins as tabs in the web UI, with a frontend SDK exposed at window.__HERMES_PLUGIN_SDK__ (React + shadcn subset + hooks) and a plugin-registration global at window.__HERMES_PLUGINS__.register(name, Component). Several deferred items from the MVP audit (cron manager, skills browser, memory viewer) previously blocked on "needs relay extension" flip with this system — most belong in the upstream dashboard directly. The relay plugin only needs to surface the four items that only the relay knows about: paired-device state, bridge command history, push delivery (future), and active MediaRegistry tokens. The question was how to ship those four surfaces cleanly.

Decision: Ship one plugin at plugin/dashboard/ exposing a single tab /relay that hosts four internal sub-tabs (Relay Management, Bridge Activity, Push Console, Media Inspector), with the frontend authored in JSX under src/, bundled to a committed IIFE at dist/index.js via esbuild, and a thin FastAPI proxy at plugin_api.py forwarding to loopback-only relay HTTP routes.

Four sub-decisions captured together:

Single plugin with internal Tabs, not four plugins. The upstream manifest allows exactly one tab.path per plugin. Four plugins would fragment the dashboard nav (four "Relay — …" entries between Skills and whatever comes next), confuse operators ("where's paired-device state?"), and multiply the manifest / rescan surface for no UX gain. One plugin at /relay with an internal shadcn Tabs component keeps relay concerns grouped and gives us a single header (Auto-refresh toggle, health pill, version string) across all four sub-surfaces.
Pre-built IIFE bundle, committed to git. The upstream example plugin (plugins/example-dashboard/dashboard/) uses plain React.createElement with no build step. That's readable for a one-tab proof of concept; for four non-trivial tabs with shared lib/ helpers (sentence formatters, relative-time, byte-size, API wrappers) it's actively painful to maintain. Decision: author JSX under plugin/dashboard/src/, bundle with esbuild (target es2020, format iife, React coming from the SDK global — NOT bundled), minify, and commit the ~16 KB dist/index.js alongside the source. Operators who git pull get a ready-to-serve bundle; the dashboard <script src> loads it verbatim and never runs a build. esbuild is the only dev dep, listed in plugin/dashboard/package.json.
Loopback-only for the three new relay routes. /bridge/activity, /media/inspect, /relay/info are gated with the same _require_loopback() helper pattern as /bridge/status, /pairing/register, and /media/register — any request.remote other than 127.0.0.1 / ::1 gets HTTP 403. The plugin backend runs inside the gateway process (itself bound to localhost) and calls the relay at http://127.0.0.1:{HERMES_RELAY_PORT}/... — no bearer minting, no new credentials, no new attack surface. We also added a loopback-exempt branch to the existing bearer-gated /sessions handler so the plugin can list paired devices without the relay needing to hand itself a token. Media paths are sanitized (basename-only) at the MediaRegistry.list_all() layer so a future decision to expose these routes externally wouldn't leak filesystem layout.

2026-07-12 amendment: /relay/info now has a second, paired-device bearer-authenticated path for Android Diagnostics. The loopback dashboard path is unchanged. The remote response is the same sanitized contract and contains no tokens, secrets, config contents, or filesystem paths; /bridge/activity and /media/inspect remain loopback-only.
Dashboard backend is a thin proxy; relay is source of truth. plugin_api.py exposes five routes at /api/plugins/hermes-relay/{overview,sessions,bridge-activity,media,push} and forwards to the relay over httpx.AsyncClient with a 5-second timeout. No business logic — the plugin never maintains its own state, never caches, never retries. Relay connect-error / timeout / 5xx translate to HTTPException(502, detail=…) carrying the relay address, so the UI can render a "relay unreachable at 127.0.0.1:8767" banner; 4xx passes through verbatim. The one exception is /push — since FCM isn't wired, this route is a static stub returning {configured: false, reason: "FCM not yet wired; …"} with no network call. Keeps the four-tab nav layout correct for when FCM lands; swapping in real data only touches PushConsole.jsx + plugin_api.py::get_push.

Consequences:

Zero upstream dependency. We ship against the already-published axiom plugin-system contract (scanner + manifest shape + SDK globals) without needing to patch hermes-agent. If upstream changes the icon whitelist or the SDK surface, we adjust the manifest / src/ and rebuild — no relay changes.
Installer already covers discovery. install.sh step 3 symlinks ~/.hermes/plugins/hermes-relay → <repo>/plugin, so ~/.hermes/plugins/hermes-relay/dashboard/manifest.json resolves automatically on any host that's already installed us. No new installer step; the plugin is live after the next hermes-gateway restart.
One extra dev dep (esbuild). Listed in plugin/dashboard/package.json under devDependencies. Not a runtime dep — the gateway never runs npm install. Developers who want to modify the frontend run cd plugin/dashboard && npm install && npm run build, which regenerates dist/index.js; CI doesn't run the build because the committed bundle is the shipped artifact.
Committed build artifact policy. dist/index.js is committed to git alongside the source. Unusual — most projects would .gitignore it — but required here because the dashboard scans live filesystem paths and has no build step of its own. The esbuild output is deterministic given the same source + minify flags, so PR review can diff the source without fighting the bundle; operators treat the bundle as read-only.
Loopback-branch divergence on /sessions. The bearer-gated /sessions path still returns is_current: bool (true for the caller's own bearer); the loopback branch returns the same session rows without is_current (no caller context). Documented in docs/relay-server.md and in the handler's docstring so future maintainers don't lose the divergence.
Push Console UX before FCM. The stub tab renders an "FCM not configured" banner + link to the deferred-items doc, which is honest but does occupy a nav slot for a non-working feature. Accepted trade-off: the alternative is shipping three tabs now and a four-tab reshuffle later, which churns muscle memory and breaks any localStorage tab-selection persistence.

Alternatives rejected:

Four separate plugins (one per sub-tab). Rejected — fragments nav, multiplies manifest surface, no UX win.
No build step; plain React.createElement in dist/index.js directly. Readable for one tab; actively hostile for four tabs + shared helpers. Source unreviewable once it hits ~500 LOC.
Bundle React into the IIFE. Would bloat the bundle from ~16 KB to ~150 KB and introduce React-version drift risk vs. the dashboard shell's React. Using the SDK's window.__HERMES_PLUGIN_SDK__.React keeps us pinned to whatever the dashboard ships.
Plugin mints its own "dashboard key" bearer via a new relay config field. Would work but adds a new credential to manage (rotation, storage, revocation) for no security benefit over the existing loopback gate — both layers already require localhost access.
Skills browser / cron manager / memory viewer as relay-plugin tabs. Rejected — these belong in upstream dashboard plugins, not a relay plugin. The relay has no unique visibility into them; upstream knows everything we'd know.
Hot-reload the Python plugin_api.py without a gateway restart. Upstream supports hot-reload for the frontend bundle (POST /api/dashboard/plugins/rescan) but Python modules still need a full gateway restart. Accepted — the backend is a thin proxy that rarely changes.

References:

plugin/dashboard/manifest.json — plugin manifest (name, label, icon, tab config, entry bundle, api module)
plugin/dashboard/plugin_api.py — FastAPI router; five proxy routes + structured 502 error translation
plugin/dashboard/src/index.jsx — React root registering via window.__HERMES_PLUGINS__.register("hermes-relay", …)
plugin/dashboard/src/tabs/{RelayManagement,BridgeActivity,PushConsole,MediaInspector}.jsx — four tab components
plugin/dashboard/src/lib/{api,formatters}.js — SDK fetchJSON wrappers + time/byte formatters
plugin/dashboard/dist/index.js — committed IIFE bundle (~16 KB minified), loaded verbatim by the dashboard shell
plugin/dashboard/test_plugin_api.py — 10 backend proxy tests
plugin/relay/server.py — handle_bridge_activity, handle_media_inspect, handle_relay_info, _require_loopback(), loopback branch on handle_sessions_list
plugin/relay/channels/bridge.py — BridgeCommandRecord dataclass + recent_commands deque + get_recent() (commit 777a06a)
plugin/relay/media.py — MediaRegistry.list_all() (commit 2212fbc)
docs/spec.md §10.1 Dashboard plugin — user-facing overview + route table
docs/relay-server.md HTTP Routes — wire-shape details for /bridge/activity, /media/inspect, /relay/info, /sessions loopback branch

21. Agent Profile picker — Hermes profile API routing with overlay fallback (2026-04-18, updated 2026-05-18)

Decision: The relay auto-discovers upstream Hermes profiles by scanning ~/.hermes/profiles/*/ (plus a synthetic "default" entry for the root ~/.hermes/config.yaml). Each discovered profile is advertised in the auth.ok payload as {name, model, description, system_message, api_server_*}. When a selected profile advertises api_server_url, Android routes chat/session API traffic to that profile's Hermes API server so memory, sessions, tools, .env, and model config follow upstream Hermes isolation. When no isolated API route is advertised, Android falls back to the older compatibility overlay: send the selected profile's model and SOUL.md (system_message) on the active Connection's API server.

Why this supersedes the original §21 design:

The first pass read from a fictional top-level profiles: / agents: list in ~/.hermes/config.yaml. Upstream Hermes has never used that schema — profiles upstream are isolated directory instances at ~/.hermes/profiles/<name>/, each with its own config.yaml, .env, SOUL.md, memory, sessions, skills, and (optionally) its own gateway daemon. The old path always returned an empty list on real installs, which is why nothing ever populated in the picker.

Rather than invent our own schema, match upstream's layout. The relay scans the directory, reads each profile's config.yaml + SOUL.md, and reports what's really there.

Three-layer model (unchanged at the UI level):

Connection (§19) — which Hermes server + gateway. One scan per server.
Profile (this decision) — which upstream-layout agent directory on that server. Routes to that profile's API server when advertised; otherwise overlays model + SOUL as fallback.
Personality (§8) — which system-prompt preset within the agent's config.

How:

Server side (plugin/relay/config.py, plugin/relay/server.py):

_load_profiles rewritten to walk ~/.hermes/profiles/*/. For each profile: name = dir.name; model = config.yaml/model.default || "unknown"; description = config.yaml/description || first non-blank line of SOUL.md || ""; system_message = SOUL.md content || null. Plus a synthetic "default" entry from the root config.
_load_profiles also reads profile-local API server metadata from config.yaml (platforms.api_server / api_server) and .env (API_SERVER_ENABLED, API_SERVER_HOST, API_SERVER_PORT, API_SERVER_KEY). It advertises api_server_enabled, api_server_url, api_server_host, api_server_port, and api_server_key_present, but never the API key value. Local bind hosts (127.0.0.1, localhost, 0.0.0.0, ::1) are rewritten through RelayConfig.webapi_url; operators should set RELAY_WEBAPI_URL to the phone-reachable base API URL. Android also rewrites loopback profile URLs against the active Connection API URL as a defensive fallback for stale or host-local payloads.
New RelayConfig.profile_discovery_enabled: bool = True. Set to false in the relay's config to skip the scan and advertise an empty list — matches our configurability pattern of "opt-out defaults" for discovery features. Disabled state is logged at INFO on startup.
auth.ok payload shape: each entry is {"name": str, "model": str, "description": str, "system_message": str | null, "api_server_enabled": bool, "api_server_url": str | null, "api_server_host": str | null, "api_server_port": int | null, "api_server_key_present": bool} (snake_case on the wire).

Client side (app/src/main/kotlin/.../data/ProfileData.kt, auth/AuthManager.kt, viewmodel/ChatViewModel.kt):

Profile includes the apiServer* metadata and exposes hasIsolatedApi.
AuthManager.parseAgentProfiles reads the new api_server_* fields with safe defaults so older relays remain compatible.
ConnectionViewModel keeps a base API client for server health/settings and a chat-routed API client for actual chat. Selecting a profile with apiServerUrl swaps chat/session calls to that profile API URL using the Connection's stored API bearer token.
Multiplex routing (2026-07-19, amended 2026-07-28). When dashboard /api/status positively reports gateway_mode=multiplex and the selected non-default profile appears in its profiles list, the chat-routed API client uses the shared listener at /p/<encoded-profile>. Dedicated api_server_url metadata still wins. Missing/older topology and the server-default selection stay on the root API URL, so no prefix is guessed. A known multiplex profile route uses a separately encrypted per-Connection/per-profile API key configured in the profile sheet; if none exists, Android sends no bearer rather than reusing the root Connection key. The optional compatibility bootstrap recognizes the prefix for slash-command route matching only; upstream middleware remains responsible for authorization and profile runtime scope.
ChatViewModel send path omits profile, model, and profile SOUL.md overrides when the selected profile has an isolated API route. The profile API server owns its own default model, SOUL, sessions, memory, tools, and .env; Android still appends phone context when enabled. If no isolated API route exists, it keeps the compatibility overlay behavior.
Chat session browsing is scoped to the selected profile route. On profile switch Android clears the old session list, refetches through the routed API client, and labels the drawer with the active profile/API fallback.
Gateway session creation, resume, and recovery for a named profile require the authoritative result's info.profile_name to match the requested profile. A missing or different owner fails closed before a prompt is submitted; this prevents an older gateway that ignored params.profile, or a profile removed after discovery, from being treated as the selected agent. The launch/default path remains compatible with older results because it does not claim a named profile.
The Gateway-native inspector tracks profiles.describe and profiles.configure as separate capabilities. A host may remain a valid read source after a configure method-not-found response; Android retains pending drafts, stops offering Gateway-owned writes for that inspector, and keeps Relay-owned memory plus the older all-Relay fallback unchanged.
Voice requests carry the selected profile to both upstream dashboard audio routes and relay-owned /voice/* routes. Standard /api/audio/transcribe, /api/audio/speak, /api/audio/speak-stream, and provider-catalog requests use the selected profile query; Relay /voice/config resolves profile-local tts/stt where present, while /voice/output/* and /voice/realtime/* resolve experimental voice_output / realtime_voice sections from the selected profile config and fall back to relay defaults with explicit config_scope metadata.

Trade-offs / v1 scope:

Isolation when the profile API is running; overlay only as fallback. Proper Hermes profile switching requires each profile's API server/gateway to be running and discoverable. If a profile has no API route, the app can still provide the older model/SOUL overlay, but that does not isolate memory, sessions, tools, provider auth, or cron jobs.
Credential isolation. The relay exposes only api_server_key_present, never the key. Dedicated profile API origins retain the Connection credential contract. Shared /p/<profile> routes use encrypted profile credentials and never inherit the root key.
SOUL.md size. Some SOUL files are multi-KB (Mizu's is 8 KB). The full content ships as system_message only in overlay fallback mode; isolated profile APIs should rely on their own configured SOUL/default prompt.
Persisted per Connection. The selected profile name and last session id are persisted per Connection/profile context. Switching Connections clears the in-memory object, then rehydrates the destination Connection's persisted profile once its advertised profile list arrives.
Voice is profile-aware but still relay-mediated. Voice output/realtime settings can follow the selected profile, but provider secrets stay server-side and the Hermes chat/tool loop still owns the final assistant text. Bridge commands remain unrelated to model choice.
Config toggle. profile_discovery_enabled = false lets a server op keep the phone picker empty (e.g. if the operator wants Connections-only semantics). Per our configurability pattern of "enable useful defaults, let the operator opt out."

Earlier (abandoned) design, for the record:

First attempt parsed a top-level profiles: / agents: list from one YAML. That schema does not exist upstream and always returned empty on real installs. The data class + picker + modelOverride wiring from that attempt are preserved; only the data source + the addition of system_message changed. See commits 0303a4f (initial parse), b9d2914 (build fix), and the R1/R2 pass that followed this decision rewrite.

References:

app/src/main/kotlin/.../data/ProfileData.kt — the data class
app/src/main/kotlin/.../auth/AuthManager.kt — parseAgentProfiles companion
app/src/main/kotlin/.../viewmodel/ChatViewModel.kt — precedence rule
app/src/main/kotlin/.../network/HermesApiClient.kt — modelOverride
app/src/main/kotlin/.../ui/components/ProfilePicker.kt — chip + dropdown
plugin/relay/config.py:_load_profiles — directory-scan source of truth
plugin/relay/server.py — auth.ok payload emission (enriched shape)
Upstream doc: ~/.hermes/hermes-agent/website/docs/user-guide/profiles.md (canonical profile-layout definition)

22. Profile-scoped read-only config + skills API (2026-04-18)

Decision: Add two relay-native HTTP routes — GET /api/profiles/{name}/config and GET /api/profiles/{name}/skills — that read <profile_home>/config.yaml and walk <profile_home>/skills/<category>/<name>/SKILL.md respectively. Both are read-only, use the existing loopback-or-bearer auth pattern (matching /notifications/recent), and resolve name via the same default → ~/.hermes / otherwise → ~/.hermes/profiles/<name> helper used by _load_profiles. For PUT /api/skills/toggle (the upstream toggle shape the phone's capability probe checks), the bootstrap registers a stubbed handler that always returns 501 Not Implemented with a structured {"error": "skill_toggle_not_implemented", ...} body.

Why relay-native, not a dashboard proxy:

The hermes-agent dashboard exposes /api/config and /api/skills via hermes_cli/web_server.py — a separate loopback-only web server from the chat API at :8642. Two problems if we proxied through the dashboard:

No profile scoping. The dashboard's /api/config operates on the active profile only; there's no way to read another profile's config without switching first. Our relay already has the layout knowledge (_load_profiles scans the tree); duplicating that as "switch profile, read, switch back" is fragile and racy.
Secrets leakage risk. Credentials normally live in ~/.hermes/.env or ~/.hermes/auth.json, but Hermes also supports some credentials in config.yaml and extensions may add their own sensitive fields. A purpose-built read route therefore keeps the remote shape explicit: paired remote clients receive only description and model.default, while loopback operator callers can inspect the complete parsed file. The response remains {profile, path, config, readonly: true}, with path: "config.yaml" remotely so host layout is not disclosed.

Both endpoints trust the same boundary as every other phone-facing relay route: bearer-auth for remote callers, loopback for in-process dashboard proxy calls. The config endpoint additionally enforces an explicit remote response schema rather than key-name redaction, so new or nested extension sections cannot silently become public. .env and auth.json are never read or returned by these routes.

Why read-only in v0.7:

Write support would require an active_profile routing layer on the relay — picking which profile's config to mutate, flushing caches, notifying any running gateway daemon for that profile. hermes-desktop handles this by shelling out to hermes profile use <name> + hermes config set ..., which bakes the lifecycle into the CLI. Doing the equivalent from the relay means either (a) shelling out (new subprocess surface, env handling, error classification) or (b) reimplementing the write path in Python (duplicates upstream logic, risks drift). Neither belongs in this pass. The readonly: true field in the response is a deliberate contract — callers that probe it can hide edit UI cleanly when (eventually) a v0.8 server drops the flag.

Why 501 on PUT /api/skills/toggle instead of omitting the route:

tools.skills_tool (the upstream module the bootstrap already imports for skills_list / skill_view) has no clean enable/disable hook — that logic lives on hermes_cli.web_server.py, which the relay doesn't proxy. Three options:

Don't register the route → 404 → Android capability probe can't tell "toggle not implemented" from "wrong URL / server too old."
Register with full behavior → requires either shelling to the CLI or duplicating upstream persistence. Same objections as write-path above, worse because skill toggle state is per-profile and upstream persists it in a JSON sidecar whose shape we don't want to pin.
Register a 501 stub with a stable structured body → capability probe sees the route, reads error: "skill_toggle_not_implemented", renders the toggle UI as disabled with a tooltip. Client logic stays a simple switch on error code; no magic version sniffing.

We pick (3). When a focused upstream follow-up exposes a real toggle on api_server.py, we flip the stub to call through — the Kotlin client sees the 501 disappear and the toggle unlocks.

Trade-offs:

Profile scoping is a directory lookup, not an active-profile check. The request says "read profile X"; we do not require X to be the currently-active profile for the dashboard or for any gateway daemon. This matches the read-only intent — the phone picker shows every profile's shape, even ones no gateway is running against.
Skill enabled: true is hardcoded. We don't track disabled state server-side yet; the field is part of the response only so the shape aligns with upstream's eventual toggle response. Keep it stable so the Kotlin model doesn't need reshuffling when toggles land.
No pagination on skills. A profile with thousands of skills would get a fat response, but that's well outside current usage (the populous profiles on our server carry 20-40 skills). Revisit if this stops being true.
Path-traversal guard is light. Profile name is rejected if it contains /, \\, ., or ... That's enough for the profiles/<name>/ join we do, and matches how _load_profiles already trusts iterdir() results — names that arrive over the wire get the extra check.

Why not just surface _load_profiles's new gateway_running / has_soul / skill_count everywhere?

Those fields (added in the same commit series — see auth.ok profile shape) are intentionally summary-only. The picker uses them to render status; the detail screen pulls the full config + skill list via the new endpoints. Splitting summary (cheap, piggybacks on existing auth.ok) from detail (expensive enough to warrant on-demand HTTP) keeps pairing snappy even for servers with many skills per profile.

References:

plugin/relay/server.py — handle_profile_config, handle_profile_skills, _resolve_profile_home
plugin/relay/config.py:_load_profiles — summary-field source (§21)
hermes_relay_bootstrap/_handlers.py:toggle_skill — 501 stub
docs/spec.md §6.1 — HTTP routes table rows
TEMP-hermes-desktop-analysis.md (session scratch, soon deleted) — hermes-desktop patterns we cross-referenced for the read/write split and credential-pool separation

Scope addendum (2026-04-18): Followed up with two more profile-scoped read routes — GET /api/profiles/{name}/soul and GET /api/profiles/{name}/memory — to feed the phone Profile Inspector's four-section layout (config / SOUL / memory / skills). Both reuse _resolve_profile_home, the loopback-or-bearer gate, and the profile_not_found 404 shape from the original two endpoints. Content is capped inline at 200KB for SOUL.md and 50KB per memory file; larger bodies flag truncated: true and clients see the on-disk size_bytes so they know real dimensions without fetching the full body. The caps exist because the Inspector is a viewer, not a diff or edit tool — phone-safe wire sizes matter more than lossless fidelity, and anyone who needs a full dump has the dashboard. Memory listing is intentionally non-recursive (one iterdir pass, no rglob) so subdirectories under memories/ — used by some users for archival snapshots — don't spam the response. MEMORY.md and USER.md (per upstream hermes_cli/profiles.py) sort first; the rest are alphabetical, so the ordering is stable across filesystems. Absent SOUL.md returns 200 with exists: false; absent memories/ returns 200 with an empty entries array — both let the Inspector render the section rather than mask "no content yet" as a transport failure.

Scope addendum (2026-07-15): Added GET /api/profiles/{name}/avatar for an explicit user action that imports an agent badge image from the Hermes host. Discovery is deliberately shallow and deterministic: direct-child files with conventional avatar/profile names win, followed by compatible aliases, across PNG, JPEG, WebP, and GIF. The resolved path must stay inside the profile home after symlink resolution and satisfy the Relay media-size cap. Android then copies the bytes into its existing connection-and-profile-scoped icon store. This keeps the host filesystem read behind Relay pairing and avoids coupling the vanilla upstream chat path or live UI rendering to host availability.

Voice Mode — Architecture

Context: We wanted real-time voice conversation with the Hermes agent — user speaks, agent listens, agent speaks back, orb reacts. Hermes-agent has six TTS providers and five STT providers fully implemented in tools/tts_tool.py and tools/transcription_tools.py, plus a CLI voice_mode.py that uses them for push-to-talk. At the time, core did not expose those functions through the API server. Upstream PR #8199 is now the canonical path for native /v1/audio/transcriptions and /v1/audio/speech; Relay should treat that as the long-term execution surface while keeping /voice/* as the paired mobile facade.

The four decisions:

1. Voice endpoints live on the relay, not upstream hermes-agent

Three options were considered:

Patch upstream gateway/platforms/api_server.py — add native audio routes directly. The current canonical shape is PR #8199's OpenAI-style /v1/audio/transcriptions and /v1/audio/speech, not a competing /api/audio/* or core /voice/* route family.
Relay plugin hosts the endpoints — plugin/relay/voice.py imports tools.tts_tool.text_to_speech_tool and tools.transcription_tools.transcribe_audio directly from the hermes-agent venv (where the relay is editable-installed) and wraps them in async handlers. Chosen.
Phone calls provider APIs directly — ElevenLabs / OpenAI / Groq SDKs on Android. Rejected: would require API keys stored on the phone, different providers per-device, no unified config, and the phone would need to replicate the provider-selection logic that tts_tool.py / transcription_tools.py already implement.

Why (2):

The relay already runs inside the hermes-agent venv (editable install per install.sh — pip show hermes-relay confirms). from tools.tts_tool import text_to_speech_tool just works.
Provider config (tts: and stt: sections) is already in ~/.hermes/config.yaml — the tools read it internally. The phone gets whatever provider the operator configured on the server, with zero per-device keys.
No upstream dependency. Ships as part of hermes-relay; updates via git pull in the plugin clone.
If upstream native /v1/audio/* endpoints are present, the relay can switch /voice/transcribe and /voice/synthesize to proxy those endpoints before falling back to private helper imports, without phone changes.

Trade-offs accepted:

Uses private upstream helpers (_load_tts_config, _load_stt_config) for the /voice/config endpoint because they're the cleanest way to report what's configured. Marked clearly as private/unstable in the handler — if upstream refactors these, our /voice/config response shape is the only thing that needs a patch.
Voice endpoints fail with 503 on servers where the hermes-agent venv doesn't have the providers' optional deps installed. Documented — operator's responsibility to run pip install "hermes-relay[voice]" or the equivalent.

2. Buffer-not-stream TTS responses (sentence-level client chunking)

The plan initially suggested streaming audio/opus over chunked HTTP — sentence-by-sentence during agent streaming. Two problems:

text_to_speech_tool is sync and returns a file path, not a chunked byte generator. Streaming output would require rewriting the function or calling provider SDKs directly (MiniMax/ElevenLabs both support streaming, Edge TTS doesn't, NeuTTS doesn't).
Android MediaPlayer prefers complete files — chunked Opus works but requires AudioTrack + manual Opus decode, which is a different layer of complexity.

Decision: The relay returns whole .mp3 files per request. The client does sentence-boundary detection on the SSE chat stream (VoiceViewModel.startStreamObserver diffs messages: StateFlow), extracts complete sentences as they arrive, POSTs each sentence to /voice/synthesize, queues the resulting mp3 file into a Channel<String>, and a consumer coroutine plays them one after another with awaitCompletion between. First audio plays within one sentence of the agent starting to respond — effectively the same UX as true streaming.

Why:

Zero upstream changes required.
Works across all six TTS providers uniformly (not just the 3 with streaming SDKs).
MediaPlayer handles mp3 natively — no AudioTrack + decoder ceremony.
Bad sentences (failed TTS) don't corrupt the stream — the queue just skips.

Trade-off: If the agent response is one giant sentence, latency is dominated by that sentence's synthesis time. In practice responses have frequent punctuation so this is a non-issue.

3. `.m4a` (MPEG-4/AAC), not `.webm`, for recorder output

The plan suggested WebM/Opus from MediaRecorder (API 29+). Rejected:

WebM support in Android's MediaRecorder arrived in API 29 but encoder availability is vendor-dependent on older builds. AAC in MPEG-4 is guaranteed since API 18 and universally supported.
OpenAI's whisper-1 and Groq's whisper-large-v3-turbo both accept .m4a / .mp4 via the upstream _validate_audio_file path in transcribe_audio.
Faster-whisper (local STT default) transparently handles m4a via ffmpeg.
M4a at 16kHz/64kbps mono is ~8KB/second — small enough for LAN upload, indistinguishable from Opus at that sample rate.

Use .m4a.

4. ChatViewModel integration via observation, not callbacks

The plan called for adding a // VOICE HOOK callback to ChatViewModel so VoiceViewModel could see streaming deltas in real time and feed them to the sentence-detection buffer.

Rejected. Instead VoiceViewModel.startStreamObserver collects chatVm.messages: StateFlow<List<ChatMessage>> (already public on ChatHandler) and diffs the last assistant message's content length on each emission to extract streaming deltas. Zero changes to ChatViewModel or ChatHandler.

Why:

Keeps chat code decoupled from voice. If voice mode is ripped out tomorrow, ChatViewModel is untouched.
Observes the same state the chat UI observes — no divergence risk.
Transcribed user text routes through the existing chatVm.sendMessage(text) path, so voice utterances appear as normal user messages in chat history. Load the session on another device and you see the transcript.

Follow-up (2026-07-25): the observer no longer relies on a single isStreaming=true assistant message. A per-turn cursor follows every new assistant bubble until the run-level ChatViewModel.isStreaming state ends, so tool handoffs and the final answer are narrated in order. The cursor fences the pre-turn stable UI identities and submitted user-turn/session identity, and only speaks strict content suffix growth, preventing StateFlow/history reconciliation or a pending-new-chat session switch from replaying old or rewritten text. Each newly observed assistant bubble also inserts a speech boundary, so an interim fragment without punctuation cannot run into the final answer.

Alternatives explicitly rejected

Android SpeechRecognizer (on-device Google speech recognition) for STT — would bypass the relay entirely and lose the "server provider" consistency. Also Google-Play-Services-dependent, which contradicts the "works on degoogled Android" goal.
In-app TTS (android.speech.tts.TextToSpeech) — same problem, plus vastly inferior voice quality compared to ElevenLabs / OpenAI. No synergy with the server's provider config.
Dedicated voice WebSocket channel on the relay alongside chat/terminal/bridge — heavier than REST for a request-response modality with no true bidirectional streaming needs. REST + SSE (via the existing chat path) is sufficient.
Sphere voiceMode expansion to 1.0× (plan target) — physically impossible with current baseRadius/data-ring geometry. Capped at 1.08×, which is still perceptually "bigger" and preserves the data ring's orbit math. Documented in the MorphingSphere KDoc.

References:

plugin/relay/voice.py — VoiceHandler, handle_transcribe, handle_synthesize, handle_voice_config; route auth is delegated to plugin/relay/voice_auth.py::require_voice_auth
plugin/relay/server.py — route registration alongside /media/*
plugin/tests/test_voice_routes.py — 14 unit tests, unittest-based (pytest conftest issue documented in CLAUDE.md)
app/src/main/kotlin/.../audio/VoiceRecorder.kt — MediaRecorder amplitude StateFlow
app/src/main/kotlin/.../audio/VoicePlayer.kt — Media3 ExoPlayer (gapless TTS queue) + Visualizer amplitude StateFlow with OEM fallback; audioSessionId served from a thread-safe @Volatile cache (read off-main by barge-in)
app/src/main/kotlin/.../network/RelayVoiceClient.kt — OkHttp multipart + JSON clients
app/src/main/kotlin/.../viewmodel/VoiceViewModel.kt — turn state machine, sentence detection, TTS queue consumer, ChatViewModel observation pattern
app/src/main/kotlin/.../ui/components/VoiceModeOverlay.kt — full-screen overlay, VoiceState→SphereState mapping, three interaction modes
app/src/main/kotlin/.../ui/components/MorphingSphere.kt — Listening/Speaking states, voiceAmplitude/voiceMode params, @Preview functions for iteration
Obsidian plan: 3. System/Projects/Hermes-Relay/Plans/Voice Mode.md

23. Branch policy: `main` + `dev` (2026-04-19)

Evolution of the earlier "Feature branches" policy (2026-04-13): the original rule was "feature branches merge into main continuously, main is always shippable." That worked while the project was small but produced two operational frictions as it grew:

Staging had no home. The server pulled main for dogfood, so any feature that passed CI was immediately exercised against real pairing / bridge / voice state. When a merged-but-rough feature hit a bug on the server, rolling back meant reverting a merge commit on the shippable branch, which fought the "main is always shippable" invariant.
Release cadence and merge cadence were entangled. Cutting a release meant version-bumping on main, which required a branch-protection carve-out (release: vX.Y.Z direct pushes) because atomic bump + tag couldn't round-trip through a PR without racing another merge.

New policy:

main = released state only. Every commit corresponds to a tag or a release-merge from dev.
dev = integration branch. Feature branches target dev; the [Unreleased] CHANGELOG section lives there.
Server pulls dev for staging. Users and hermes-relay-update track main and tags.
Releases are opened as PRs from dev into main, merged --no-ff, then tagged from main. No direct-push carve-out is needed — the release PR is just a normal PR.

Trade-offs accepted:

Hotfix sync step. When a hotfix lands on main from a tag (bypassing dev), you have to merge main back into dev so the next release doesn't collide on appVersionCode. Documented in RELEASE.md hotfix recipe.
Two branches to keep up to date. Feature-branch authors need to remember that the base is dev, not main. Mitigated by branch protection on both and by CI running on both.
Server tracks pre-release state. Post-policy the server pulls dev, so the real shift is that the Play Store / sideload-update audience now trails dev by a release cadence, not a merge cadence.

CI impact: the same workflow files trigger on both branches via path-filtered triggers (ci-android.yml / ci-plugin.yml after the 2026-04-19 split). docs.yml stays main-only — docs publish represents shipped state, not integration state. Surface release workflows are tag-triggered and branch-agnostic, unchanged.

References:

CLAUDE.md — "Git" section + "Testing / CI is split by path" note
RELEASE.md — "Branching policy" + Release Process step 4 + Hotfix recipe
CONTRIBUTING.md — "Commit Conventions"
.github/workflows/ci-android.yml, .github/workflows/ci-plugin.yml
scripts/bump-version.sh — prints the dev → main release flow

24. Multi-endpoint pairing payload + network-aware switching (2026-04-19)

Problem: pairing QRs today carry a single host:port (API) plus a single relay.url. That works for LAN-only operators, but the moment the phone moves between LAN / Tailscale / a public reverse-proxy hostname, the single URL is wrong half the time. Upstream hermes-agent's stance (SECURITY.md) is "use VPN / Tailscale / firewall or don't expose" — they don't plan to own a remote-access story. We do.

Decision: Extend the QR schema with an optional ordered list of endpoint candidates. The phone picks the highest-priority reachable candidate at connect time and re-evaluates on network change. Old single-endpoint QRs remain valid — the phone synthesizes a single priority-0 candidate from the top-level fields when endpoints is absent.

Wire format (v3 — additive):

{
  "hermes": 3,
  "host": "hermes.tail-scale.ts.net",
  "port": 8642,
  "key": "optional-api-key",
  "tls": true,
  "relay": { "url": "wss://hermes.tail-scale.ts.net:8767", "code": "ABC123",
             "ttl_seconds": 2592000, "grants": {...},
             "transport_hint": "wss" },
  "endpoints": [
    { "role": "tailscale", "priority": 0,
      "api":   { "host": "hermes.tail-scale.ts.net", "port": 8642, "tls": true },
      "relay": { "url": "wss://hermes.tail-scale.ts.net:8767",
                 "transport_hint": "wss" } },
    { "role": "public",    "priority": 1,
      "api":   { "host": "hermes.example.com", "port": 443, "tls": true },
      "relay": { "url": "wss://hermes.example.com/relay",
                 "transport_hint": "wss" } },
    { "role": "lan",       "priority": 2,
      "api":   { "host": "192.168.1.100", "port": 8642, "tls": false },
      "relay": { "url": "ws://192.168.1.100:8767",
                 "transport_hint": "ws" } }
  ],
  "sig": "base64-hmac-sha256"
}

Semantics (locked):

Generated defaults are secure-first (amended 2026-08-12). An available pinned Hermes Secure Link, Tailscale Serve, and configured public TLS candidates precede plain LAN. LAN remains a signed, independently acknowledged fallback. An explicit operator preference can still change the strict order.
role is an open string. Known values lan / tailscale / public get styled chips + icons on the phone. Anything else (wireguard, zerotier, netbird-eu, whatever the operator wants) renders with a generic "Custom VPN" treatment and the raw role label. No enum, no validation, no normalization — role is preserved exactly as emitted so HMAC canonicalization round-trips.
priority is strict, 0 = highest. If priority-0 is reachable the phone uses it; reachability never promotes a lower-priority candidate over a higher one. Reachability is only the tiebreaker for candidates that share the same priority. This is the DNS SRV priority/weight contract — known-good semantics, nothing new to debate.
Reachability probes are per surface. Standard routing probes Dashboard /api/status when present, otherwise an explicitly configured API /health, with Relay /health used only for Relay-only records. Relay socket selection probes the candidate's own Relay /health; a healthy Dashboard or API listener never vouches for Relay on another port. Results are cached independently by candidate and surface, so a Relay outage cannot poison healthy standard chat, Manage, sessions, or Vanilla Hermes voice.
Relay endpoint normalization is path-aware and idempotent (amended 2026-08-20). A Relay candidate may name its base, its terminal /ws socket route, or its terminal /health route using HTTP(S) or WS(S). Android removes that terminal route segment once and derives sibling <base>/ws and <base>/health routes. Root bases therefore map to /ws and /health, while /relay, /relay/ws, and /relay/health all map to /relay/ws and /relay/health. Arbitrary path probing is prohibited. Authorities, ports, IPv6 literals, and custom path prefixes are preserved; user info, queries, fragments, malformed URLs, relative segments, and ambiguous encoded separators fail closed. Plain HTTP/WS continues to require the existing explicit trusted-LAN/VPN consent.
Network-change re-probe: Android's ConnectivityManager .NetworkCallback.onAvailable / onLost triggers a re-probe. If a higher-priority candidate became reachable after a network transition, the phone reconnects to it; if the current one became unreachable, the phone falls through to the next candidate in priority order.
Relay retry state is route-aware. Transport-failure streaks are scoped to the Relay socket URL, so one LAN failure plus one Tailscale failure cannot evict Tailscale as though it had failed twice. Automatic route replacement preserves the accumulated exponential-backoff attempt; explicit user/network handoff and a successful socket open reset it. If every Relay surface is unavailable, the standard route remains stable while Relay retries the last configured socket with bounded backoff.
TTL defaults by role (informational — operator can override at pair time): lan → 7 days, tailscale → 30 days, public → 30 days, unknown role → 7 days (conservative). The longer tailscale default reflects that the tailnet is a secure transport (WireGuard end-to-end encryption + device identity) — not that the link is TLS. A tailscale candidate can carry a plain transport_hint = "ws" and still be encrypted; that's WireGuard, not TLS. See user-docs/architecture/connection-security.md. Plaintext-ws:// consent still gates any candidate with transport_hint = "ws".

Canonicalization for the HMAC signature: canonicalize() in plugin/relay/qr_sign.py uses json.dumps(sort_keys=True, separators=(",",":"), ensure_ascii=True, allow_nan=False). Nested dicts are sort-keyed; arrays preserve their emitted order (priority is meaningful, not alphabetic). Role strings are embedded verbatim — no .lower(), no .strip(). A regression test in plugin/tests/ test_qr_sign.py exercises this for mixed-case and non-ASCII role values so a future refactor can't silently divergence the canonical form.

Pin store: CertPinStore continues to key by host:port, unchanged. Role doesn't participate in the pin key — if the operator points two roles at the same hostname they'll share a pin (which is correct: same cert, same pin). The role → hostname mapping lives in the per-device endpoint record in DataStore, read by the Paired Devices screen to render one row per (device, endpoint).

Backward compatibility:

hermes: 1 and hermes: 2 QRs continue to parse. When endpoints is absent, the phone synthesizes a single priority-0 candidate of role: lan (or role: tailscale if the top-level host matches the Tailscale CGNAT pattern 100.64.0.0/10 or a .ts.net suffix — same heuristics TailscaleDetector already uses).
hermes: 3 is the first version that requires endpoints. The bump is bookkeeping only; the parser is version-liberal (ignoreUnknownKeys = true already, plus a nullable endpoints field for the intermediate period when the server emits v3 but some phones haven't updated yet).

Alternatives rejected:

Layer Noise / libsignal over WSS. The operator owns both endpoints and the transport is already TLS-terminated by the operator's chosen path (Tailscale / reverse-proxy / WireGuard). A second crypto layer adds complexity without defending against any threat in our model.
Auto-promote reachability over priority ("LAN is flaky, switch to Tailscale even though priority-0 works"). Breaks operator intent; the operator might have a reason for preferring LAN (zero egress billing, lower latency, local-network-only policy). Strict priority keeps the operator in charge.
Close-vocabulary role enum. Would force a release every time an operator spun up a new mesh VPN. Open-string + known-values-display map gives us the UI polish for common cases and zero friction for unusual ones.

Key Files:

plugin/pair.py — build_payload() emits endpoints; --mode / --public-url flags; LAN-IP + Tailscale status auto-detection
plugin/relay/qr_sign.py — canonical form docstring now covers arrays
plugin/relay/server.py — handle_pairing_mint / handle_pairing_register accept optional endpoints in the body
plugin/tests/test_pairing_mint_schema.py + test_qr_sign.py
app/src/main/kotlin/.../data/Endpoint.kt — new EndpointCandidate data class
app/src/main/kotlin/.../ui/components/QrPairingScanner.kt — payload extended; v1/v2 synthesizer
app/src/main/kotlin/.../data/PairingPreferences.kt — per-device endpoint list store
app/src/main/kotlin/.../network/ConnectionManager.kt — resolveBestEndpoint() + NetworkCallback plumbing
app/src/main/kotlin/.../viewmodel/RelayUiState.kt — activeEndpointRole field
skills/devops/hermes-relay-pair/SKILL.md — new flags documented

25. First-class Tailscale helper as optional hermes enhancement (2026-04-19)

Problem: the one piece of first-party upstream remote-access work (PR #9295 — Tailscale Serve integration) is open but unmerged. Our current TailscaleDetector is informational-only. We want Tailscale to be a first-class supported mode today, without forking upstream.

Decision: Ship a thin Tailscale helper in plugin/relay/tailscale.py that mirrors PR #9295's contract — keep the relay loopback-bound, let tailscale serve --bg --https=<port> http://127.0.0.1:<port> terminate TLS + identity. The helper is optional (no-op when the tailscale binary is absent) and auto-retires when upstream lands the canonical hermes gateway run --tailscale flag (detection via a one-shot capability probe; helper prints an [info] pointing at the canonical path and exits 0). Same pattern hermes_relay_bootstrap/ uses for the session-API endpoints.

Surface:

plugin/relay/tailscale.py — thin module: status(), enable(port=8767), disable(port=8767), canonical_upstream_present(). All shell out to tailscale CLI and return structured dicts; no new daemon, no new state.
scripts/hermes-relay-tailscale — shell shim mirroring hermes-pair pattern for scriptability and older Hermes builds. Current upstream supports generic plugin CLI command dispatch, so native hermes <subcommand> should be preferred when available.
install.sh gets an optional step [7/7]: detect tailscale binary; if present and the operator hasn't declined, offer to run tailscale serve --bg --https=8767 http://127.0.0.1:8767. Skipped silently if binary absent, TS_DECLINE=1 set, or non-interactive shell without TS_AUTO=1.
plugin/pair.py --mode auto calls tailscale.status(). If a .ts.net hostname is available, emit a role: tailscale endpoint at the next available priority after any role: lan entries.

Why this is the right shape:

Loopback-bound stays loopback-bound. The relay keeps listening on 127.0.0.1:8767; Tailscale handles the TLS + external reachability. No change to the relay's security posture.
tailscale serve already handles auth via tailnet ACLs + device identity. We don't layer a second auth on top — the relay's existing bearer-token session auth covers the application layer.
Pins work unchanged. The .ts.net hostname's cert is issued by Tailscale's internal CA; TOFU pin captures it on first connect, same mechanism as any other wss cert.
Graceful absence. Operator without Tailscale sees zero noise. All failure modes exit non-fatal.

Upstream-merge retirement: when PR #9295 lands, canonical_upstream_present() returns True (detected via hermes gateway run --help | grep tailscale or a hermes.version >= X.Y.Z gate — the exact probe is TODO until the PR lands and we see what its contract looks like). Helper then no-ops with a log line pointing at hermes gateway run --tailscale. install.sh gains an # TODO(upstream-merge #9295): comment marking the exit criteria. Same removal pattern as other compatibility overlays: keep the helper until a released core build exposes the native path, then delete the local shim instead of layering another abstraction on top.

Alternatives rejected:

Bundle our own tailscaled daemon. Replaces Tailscale's existing user-facing install / login flow with a worse one. Operators already have Tailscale installed or don't — meeting them where they are is strictly better.
Wait for upstream PR #9295. Unclear merge ETA; we'd ship v0.4.x with the first-class-Tailscale UX paper-thin while waiting.

Key Files:

plugin/relay/tailscale.py (new)
scripts/hermes-relay-tailscale (new) — shell shim
plugin/tests/test_tailscale_helper.py (new)
install.sh — optional step [7/7]
plugin/pair.py — --mode auto integration
docs/remote-access.md (new) — operator-facing setup guide

ADR 26 — Rich cards via inline `CARD:{json}` line markers

Status: Accepted (v0.7.x) — phone-side Phase A. Upstream adapter parity is a separate follow-up (Phase B).

Context. The chat feed needed a way to surface structured content — skill results, approval prompts, link previews, calendar entries, weather — as discrete Material 3 cards instead of drowning them in markdown. Upstream hermes-agent has zero rich-content abstraction in its platform adapters (gateway/platforms/base.py exposes send() + send_image() + edit_message() with no blocks or embeds). Discord uses no discord.Embed() at all, Slack uses Block Kit only for the exec-approval dialog, and the base class only hints at rich surfaces via the REQUIRES_EDIT_FINALIZE attribute for DingTalk AI Cards.

We already had a working precedent: the MEDIA: marker in assistant text gives the LLM a lightweight way to attach files that works identically across every streaming endpoint we support (/v1/runs + /api/sessions/{id}/chat/stream + /v1/chat/completions). Cards follow the same recipe.

Decision.

Wire format — CARD:{json} on its own line. Parser in network/handlers/ChatHandler.kt mirrors the media-marker path: dedicated line buffer, dedupe set, single-line regex, finalize pass on turn/stream complete, reload pass in loadMessageHistory. Multi-line JSON is disallowed so the line-buffer strategy stays trivial; escape newlines in string fields as \n.
Data model (data/HermesCard.kt). @Serializable with ignoreUnknownKeys = true so newer agents emitting fields the phone build doesn't know about never crash the parser. Unknown type values render via a generic fallback (title + body + fields + actions), so the surface degrades gracefully when we add new built-ins server-side.
Built-in types (Phase A). skill_result, approval_request, link_preview, calendar_event, weather. approval_request intentionally mirrors the Slack exec-approval 4-button pattern (Allow / Allow Session / Always Allow / Deny → map to actions with style: "primary" / "secondary" / "danger") so future upstream parity (Phase B) is just adapter-side translation, not a data-model rethink.
Accent vocabulary. info / success / warning / danger — semantic, not raw hex, so the renderer pulls from colorScheme and dark/light themes stay coherent.
Action dispatch. mode is one of send_text (default), slash_command, open_url. All three route through ChatViewModel.dispatchCardAction, which records a HermesCardDispatch stamp on the owning message before forwarding — so the card collapses into a "Chose: X" confirmation even if the side effect (browser launch, server round-trip) throws. open_url resolves at the UI layer (needs Context); the other two reuse sendMessage because slash commands are plain text to the server.

Why markers and not structured SSE events. A card.available event alongside tool.completed would be cleaner on the wire but would require a server patch upfront and would NOT work on /v1/chat/completions — which is the one endpoint every upstream deployment supports. The marker approach ships today across every endpoint with zero server dependency. If/when we decide structured events are worth the bootstrap cost, the parser can fan out; the HermesCard data model stays.

Why CARD: not <hermes-card>...</hermes-card>. Matches the existing MEDIA: shape exactly — LLMs have an easier time producing consistent markers when every rich-content construct uses the same grammar. A future delimiter swap (e.g. fenced code blocks) is easy if the flat marker turns out to be fragile in practice.

Tradeoffs accepted.

Marker leakage: if the LLM gets confused mid-turn the raw CARD:{...} line can appear in a bubble. We mitigate by leaving unparseable lines in the content (visible artifact beats silent drop) and by including the marker contract in the same section of prompt_builder.py as MEDIA:.

Server-side session sync (shipped alongside Phase A).

Each [com.hermesandroid.relay.data.HermesCardDispatch] carries a syncedToServer flag. On the next chat send, CardDispatchSyncBuilder.buildSyntheticMessages materializes every unsynced dispatch into an OpenAI-format assistant (with tool_calls) + tool (with tool_call_id) pair under a synthetic tool name hermes_card_action — a namespaced name the upstream dispatcher will never try to execute, it's a historical audit record only. The arguments object carries card_key / action_value / card_type / card_title / action_label / action_mode / action_style so the LLM has enough context to describe the interaction even if the card itself gets trimmed from rolling window memory. Pairs are spliced into the same request-body slot as voice-intent synthetic messages (voiceIntentMessages param — name is historical, the param accepts any synthetic-message JsonArray); once the API client accepts the request, ChatHandler.markCardDispatchesSynced flips every dispatch's flag so subsequent turns don't re-emit. Commit-timing matches the voice-intent path exactly (post-handoff) so a thrown request-building exception leaves dispatches unsynced for the next try.

Update (2026-07, HRUI-001): the top-level messages array these pairs originally rode on the sessions/runs SSE payloads was never parsed by native upstream — the context was silently dropped on those transports. The payload builders (HermesChatPayloads.kt) now deliver synthetic history through channels upstream actually consumes: tool-call pairs render as a plain-text digest folded into the per-turn ephemeral system prompt (system_message on sessions, instructions on runs, the system message on completions), and plain realtime-voice turns ride a real history channel where one exists (completions messages splice, runs conversation_history). The digest is per-turn context, not persisted server-side session history.

Phase B (deferred — not v0.7.x).

Contribute a gateway/rich_cards.py helper upstream + Discord/Slack adapter translations. Discord gains its first real embed usage; Slack reuses the existing Block Kit path. Plain-text platforms (Signal, SMS) fall back to a markdown render of the same card. Same playbook as the current compatibility-overlay model: ship locally while the shape is proving out, then retire the local marker path once a released core build exposes the native card surface. Held until real phone-side card usage surfaces concrete fidelity issues worth translating for.

Key Files:

app/src/main/kotlin/com/hermesandroid/relay/data/HermesCard.kt (new)
app/src/main/kotlin/com/hermesandroid/relay/ui/components/HermesCardBubble.kt (new)
app/src/main/kotlin/com/hermesandroid/relay/viewmodel/CardDispatchSyncBuilder.kt (new)
app/src/test/kotlin/com/hermesandroid/relay/viewmodel/CardDispatchSyncBuilderTest.kt (new)
app/src/main/kotlin/com/hermesandroid/relay/network/handlers/ChatHandler.kt (+scanForCardMarkers / tryDispatchCardMarker / finalizeCardMarkers / extractCardsFromContent / recordCardDispatch / markCardDispatchesSynced)
app/src/main/kotlin/com/hermesandroid/relay/ui/components/MessageBubble.kt (+onCardAction plumb-through)
app/src/main/kotlin/com/hermesandroid/relay/viewmodel/ChatViewModel.kt (+dispatchCardAction, sync splice in startStream)

ADR 27 — Desktop control native shell uses Tauri v2 with CLI fallback

Status: Superseded by ADR 37 (2026-07-13). Retained as historical context.

Context. The desktop computer-use surface needs more than a terminal prompt once it graduates from experimental CLI use. Safe control needs a tray icon, always-visible observing/control chip, local grant prompt, task log, settings, and an emergency stop that is available even when no terminal window is open. At the same time, the existing TypeScript CLI and daemon are already the durable thin-client surface and must keep working for operators, headless boxes, and scripting.

Decision. Use Tauri v2 (Rust + static web UI) for the native tray/overlay shell. Keep the core desktop tool router, pairing state, grant model, and policy logic in the existing TypeScript CLI first. Rust stays thin and native-specific: tray, overlay window, hotkey, installer/sidecar integration, and later OS-level screenshot/input helpers.

Overlay UX refinement (2026-05-17). The desktop overlay should behave like a compact Conjure-style pill, not a mini management card. It is a small transparent always-on-top status window anchored bottom-center to Tauri's monitor work area so it sits just above the taskbar; the full management UI remains in the tray/dashboard. The pill dynamically sizes to its current label (Observing, Paused, Offline, or Unavailable) instead of reserving a fixed dashboard-width capsule. On Windows the tray shell reasserts topmost with SetWindowPos(HWND_TOPMOST | SWP_NOACTIVATE), strips native chrome styles, and marks the overlay click-through so the pill cannot steal focus or expose a titlebar.

UX tiers.

Easy: pair once, tray runs, compact visible pill shows Observing / Control Active / Paused / Disconnected, and pause or emergency stop is always reachable from the tray, dashboard, or hotkey.
Standard: full tray menu with Devices, Revoke, Task Log, Settings, grant history, and a settings panel.
Advanced: CLI + hermes-relay daemon + JSON policy and scripts remain first-class forever.

Seamless pairing requirement. The Tauri app must reuse the current hermes-relay pair flow, QR/code payloads, ~/.hermes/remote-sessions.json, and relay /sessions device management. A successful Easy-tier pair should leave the tray connected, the overlay chip visible, and Devices / Revoke / Task Log / Settings / Emergency Stop reachable from the tray. Local revoke must also clear active computer-use grants.

Tray click behavior (2026-05-17). Tauri's tray menu can appear on left click by default, so the app disables left-click menu display and reserves left click for opening the dashboard. The native management menu remains on right click and through explicit menu events, preventing a single tray click from trying to show both a menu and the main window.

Control refresh wiring (2026-05-17). Dashboard controls, tray menu actions, and overlay status all read the same Rust daemon state. Control commands must release the daemon mutex before asking for the resulting daemon status; otherwise pause/stop can deadlock by trying to lock the same mutex twice. Mutating commands emit dashboard://refresh, the dashboard guards against stale overlapping refreshes, and the overlay listens for the same event in addition to its fallback poll.

Desktop GUI refinement (2026-05-17). The desktop dashboard should reference the Android app's visual discipline and shared Hermes branding, not copy its mobile navigation. The Tauri shell uses the Chevron Compass logo from the repo brand assets, keeps the sidebar navigation-only, and places Start / Pause / Emergency Stop in a sticky topbar so critical daemon controls stay reachable at short window heights. The Android reference remains useful for graphite surfaces, dense operational spacing, short labels, and visible state-first controls.

Default blocklist baseline. Computer-use policy starts with password managers, credential vaults, MFA/passkey/OS credential prompts, banking/brokerage/payment apps, crypto wallets, OS security/admin settings, and private-key or token surfaces blocked by default. Advanced users can narrow or extend that baseline through ~/.hermes/desktop-control.json; the Easy tier keeps it on.

Why Tauri.

Small binaries and system webview match the thin-client posture better than Electron.
Tray, always-on-top windows, native dialogs, hotkeys, signing, and Rust interop cover the missing safety UX.
React keeps UI implementation close to the existing TypeScript stack.

Alternatives rejected.

Electron is too heavy for a background "desktop hand" helper.
Pure WinUI3 / SwiftUI / platform-native stacks increase maintenance cost across platforms.
CLI-only plus node tray does not give a strong visible overlay and grant-prompt story.

Compatibility rule. Phases 1-4 of desktop computer-use must keep working fully in the CLI and daemon with no Tauri dependency. The Tauri app is the Windows Easy/Standard wrapper and primary installer surface, while hermes-relay daemon remains the durable advanced/headless backend. The tray installer bundles the compiled CLI as a Tauri sidecar so normal pair/daemon/devices/task-log workflows do not depend on a separate PATH install. When the tray starts the daemon, it also provides a local grant bridge so assist/control requests can open the Grant Requests view for visible approval instead of silently granting host input.

Key Files:

docs/plans/desktop-control-computer-use-enhanced.md
desktop/src/
desktop/tray/
desktop/README.md
plugin/tools/desktop_tool.py

ADR 28 - Desktop Android pairing parity and one active tray relay

Status: Superseded by ADR 37 (2026-07-13). CLI pairing behavior remains.

Context. The first Windows tray pass made the desktop surface testable: real tray icon, click-through overlay pill, sticky daemon controls, dashboard refresh wiring, and shared Hermes branding. The next gap is product parity with the Android pairing/settings experience. Android already has the right model: pairing is a first-class route, endpoint candidates make LAN/Tailscale/public routes visible, auth failures separate API health from relay session auth, and advanced settings are tucked behind expandable sections instead of crowding the default screen.

Decision. Track the next desktop implementation in docs/plans/2026-05-17-desktop-android-pairing-parity.md. The Windows Tauri tray app should default to dark mode, present one active paired relay instance for now, support manual pairing and pasted pairing invites, expose LAN/Tailscale/manual endpoint choices when available, and require explicit confirmation before replacing the active desktop pairing. Visible "Tier" settings and "Easy tier" copy should be removed from the tray UI; CLI/daemon/JSON configuration remain the advanced operator path without being represented as a user-selectable app tier.

Terminal and diagnostics refinement (2026-05-17). After pairing parity, the tray dashboard adds task-based Terminal / CLI and Diagnostics routes. Terminal / CLI opens the remote TUI in a real terminal and converts the active relay into copyable standard hermes-relay shell, chat, daemon, status, tools, and doctor commands so users can move between GUI and terminal workflows without hunting through docs. The CLI resolver reads the tray-selected active relay from ~/.hermes/desktop-control.json, so copied commands omit --remote after pairing and reserve it for explicit one-off overrides. If the app is only using its bundled sidecar, the tab nudges the user to install the CLI shim instead of copying full sidecar paths into normal workflows. Diagnostics keeps local install/session checks visible in the tray and runs hermes-relay doctor --json through the bundled sidecar, preserving the CLI as the source of truth for install triage.

Pairing invite URL and desktop consent repair (2026-05-17). Host-side pairing now prints a paste-friendly hermes-relay://pair?payload=... invite URL alongside the QR payload, and /pairing/mint returns the same value as pairing_url. Desktop accepts raw JSON, base64 JSON, or the invite URL in the Paste invite flow. The desktop CLI now correctly uses relay.code as the one-shot relay pairing code; the top-level key remains the Hermes API bearer for direct HTTP chat. The tray dashboard also exposes an explicit active-relay desktop-tool consent grant/revoke control so users do not need to drop into a TTY just to allow the daemon.

Optional hermes alias (2026-05-17). The desktop CLI installer no longer creates hermes / hermes.cmd by default. That alias is useful for Orca and upstream-style hermes workflows because it opens the paired remote Hermes session, but it can shadow a real local hermes-agent install. Alias management is therefore explicit through desktop/scripts/hermes-alias.ps1 and desktop/scripts/hermes-alias.sh; both scripts enable, disable, or report status and only touch aliases that point to hermes-relay.

Implementation constraints.

Do not change Android as part of the desktop pass.
Keep CLI and daemon behavior intact.
Keep Rust config deserialization backward-compatible with existing tier values while removing the visible selector from the tray UI.
Keep multiple stored sessions compatible with ~/.hermes/remote-sessions.json, but present exactly one active desktop relay in the tray UI until multi-instance desktop UX is explicitly approved.
Use a collapsed Advanced section for low-frequency controls such as computer-use flag, emergency hotkey, overlay tuning, raw relay URL override, and blocklist.

Key Files:

docs/plans/2026-05-17-desktop-android-pairing-parity.md
docs/android-ui-design-reference.md
desktop/tray/src-tauri/src/main.rs
desktop/tray/ui/index.html
desktop/tray/ui/app.js
desktop/tray/ui/styles.css

ADR 29 - Voice output uses streaming TTS as renderer, realtime as agent mode

Status: Implemented baseline (2026-05-18).

Context. The first relay-mediated realtime voice path made provider PCM, Android AudioTrack playback, waveform metrics, and barge-in testable. It also exposed a correctness issue: when normal chat speech is sent to a realtime voice-agent provider as a conversational prompt, the provider can answer differently from the Hermes chat response. We patched the current path with a render_mode="verbatim" bridge, but that is still instruction-following on a conversational provider rather than a true speech renderer.

Decision. Treat deterministic assistant narration and realtime voice agents as different output modes.

streaming_tts_renderer is the default target for normal assistant speech, pre-tool-call status lines, long-task filler, and final response narration. It receives final Hermes text and should emit exact audio without answering conversationally.
realtime_agent is reserved for speech-to-speech, provider turn-taking, provider tool-call event experiments, and comparison testing.
Hermes remains the owner of chat text, tool execution, approvals, and final answers. Provider-emitted tool-call events are scaffolds unless a later explicit integration changes that contract.

Provider direction. Grok streaming TTS (xai_tts) is the first renderer, with OpenAI streaming TTS (openai_tts) added as the next renderer. Existing Hermes /voice/synthesize remains the fallback path, and ElevenLabs remains a lab comparison adapter. Keep Grok/OpenAI Realtime providers in the lab and relay for agent-mode testing and comparison.

Shared broker behavior. The voice output broker owns verbatim rendering, PCM waveform levels, first-audio latency, chunk-gap metrics, playback, barge-in cancel, fallback selection, and short spoken tool/wait status lines. These behaviors should not be reimplemented separately in every provider adapter.

Key Files:

docs/realtime-voice-poc.md
docs/realtime-voice-lab-readme.html
plugin/relay/voice_output.py
plugin/relay/realtime_voice.py
plugin/voice_lab/providers/xai_tts.py
plugin/voice_lab/providers/openai_tts.py
plugin.voice_lab
app/src/main/kotlin/com/hermesandroid/relay/network/RelayVoiceClient.kt
app/src/main/kotlin/com/hermesandroid/relay/ui/screens/VoiceSettingsScreen.kt
app/src/main/kotlin/com/hermesandroid/relay/viewmodel/VoiceViewModel.kt

ADR 30 - Desktop surface plugins and built-in Herm launcher

Status: Amended by ADR 37 (2026-07-13). The CLI plugin commands remain; the tray plugin view and embedded PTY do not.

Context. The desktop tray already owns a local xterm/PTY host for the remote Hermes Relay TUI. Herm (liftaris/herm) is a separate OpenTUI dashboard for Hermes, published as herm-tui, and should be installable without folding its code into Hermes-Relay or replacing the default relay TUI path.

Decision. Add a small desktop surface plugin registry to the CLI and Tauri tray. The first built-in descriptor is Herm:

source: https://github.com/liftaris/herm
package: herm-tui
binary: herm
install/update: prefer bun add -g herm-tui, fall back to npm install -g herm-tui
launch: prefer installed herm, fall back to bunx herm-tui or npx --yes herm-tui
resume: use -c for installed and fallback launchers

The registry describes plugin tabs, status cards, keybindings, and session actions for the dashboard. The CLI exposes hermes-relay plugins; the tray adds a Plugins view with Install, Update, Open, Resume, and Embed actions. Embedding reuses the existing xterm/PTY host and records the running surface as plugin:<id> so the dashboard can distinguish Herm from the default Relay TUI.

Why built-in first. Herm is a known terminal surface with simple package manager semantics. A built-in descriptor gets install/update/launch UX working now while keeping dynamic third-party plugin loading out of the trusted desktop surface until signing, permissions, and manifest validation have an explicit design.

Compatibility rules.

Bare hermes-relay remains the default remote Relay TUI path.
Plugin installation is optional and does not mutate relay pairing or session token state.
Plugin fallback launchers are convenience paths only; a user can still manage herm-tui directly with Bun or npm.
The tray must keep external terminal launch available when embedded PTY focus, resize, or shortcut behavior needs a fallback.

Key Files:

desktop/src/surfacePlugins.ts
desktop/src/commands/plugins.ts
desktop/src/cli.ts
desktop/tray/src-tauri/src/main.rs
desktop/tray/ui/index.html
desktop/tray/ui/app.js
desktop/tray/ui/styles.css
desktop/README.md
user-docs/desktop/subcommands.md

ADR 31 - Desktop Chat tab and first-run chat route

Status: Superseded by ADR 37 (2026-07-13). Chat remains a CLI/TUI surface.

Context. fathah/hermes-desktop is useful as a product reference for a chat-first desktop surface and first-run setup, but Hermes-Relay's desktop app is a thin Tauri client that should not fork a separate Electron chat stack. The durable local source of truth is still the paired relay session in ~/.hermes/remote-sessions.json plus the existing hermes-relay chat CLI path. Some users also need chat-only access to a Hermes WebAPI gateway before or instead of pairing a relay.

Decision. Add a tray Chat tab with route selection:

Paired relay mode is the default whenever selected_url resolves to a stored session. It spawns the bundled CLI sidecar as hermes-relay chat --json with the active relay URL, so auth, session creation/resume, gateway events, and cancellation remain shared with the CLI.
Direct Gateway/API mode is available when no relay is paired or when the user selects it explicitly. The tray saves only chat_gateway_url in ~/.hermes/desktop-control.json; the optional API key is passed to the sidecar as an environment variable for the current chat turn and is not persisted.
The direct API worker probes /api/sessions/probe/chat/stream first, then /v1/runs, mirroring the Android WebAPI capability split.
The tab owns only chat UX: transcript, stop, retry, new chat, clear, setup diagnostics. Models, providers, memory, skills, schedules, and richer agent management remain future plugin panels rather than core tray navigation.

Compatibility rules.

Relay pairing remains required for daemon, terminal/TUI, devices, grants, and desktop tool routing.
Direct Gateway/API mode is chat-only and must not imply desktop-control consent or a paired relay session.
The default relay TUI and plugin terminal surfaces must remain available unchanged.

Key Files:

desktop/src/commands/chat.ts
desktop/src/commands/chatWorker.ts
desktop/src/cli.ts
desktop/tray/src-tauri/src/main.rs
desktop/tray/ui/index.html
desktop/tray/ui/app.js
desktop/tray/ui/styles.css
desktop/README.md
user-docs/desktop/index.md
user-docs/desktop/subcommands.md

ADR 32 - Realtime Agent uses provider preamble before relay-forced Hermes

Status: Accepted for Realtime Agent correction (2026-05-22).

Context. Realtime Agent is supposed to feel like a provider-native speech-to-speech session while Hermes remains the governed brain and tool authority. Recent forced-Hermes routing protected correctness by sending current data, tool, memory, and confirmation requests through Hermes, but it could regress the audible turn shape by cancelling the provider response and using local status TTS before Hermes ran. That makes the feature feel like the old STT -> Hermes -> TTS pipeline instead of a native realtime agent.

Decision. Relay-forced Hermes turns must use this sequence:

Provider owns speech recognition and yields the final user transcript.
Relay detects that the transcript must go to Hermes.
Relay asks the active realtime provider to speak one short acknowledgement, for example "I'll check Hermes.", without calling tools or adding detail.
After that provider audio drains, the relay starts the Hermes run and mirrors clean Hermes tool/progress/confirmation state into the UI.
Hermes returns a compact authoritative function result.
The same realtime provider speaks the final natural summary from that result.

Rules.

Hermes remains the only path for tools, memory, current data, research, side effects, durable context, and confirmations.
Android must not read raw Hermes tool output aloud. Raw output belongs in logs/debug trace; spoken output is a provider-generated summary after the compact Hermes result.
Local spoken status lines are fallback or long-wait affordances only. They should not replace provider-native pre-Hermes acknowledgement during a healthy Realtime Agent turn.
The relay must suppress provider tool calls during the preamble phase; the preamble is acknowledgement only, and Hermes starts after the preamble drains.

Key Files:

docs/realtime-voice-poc.md
docs/relay-protocol.md
plugin/relay/realtime_agent/broker.py
plugin/relay/realtime_agent/providers/xai.py
plugin/relay/realtime_agent/providers/openai.py

ADR 33 - Realtime Agent foregrounds short Hermes turns and promotes long ones to background tasks

Status: Accepted, phased (2026-05-24). Default-on at feature GA, gated by a prerequisite provider-idle-tolerance spike (Phase 0). Supersedes the blocking-broker assumption inside ADR 32's sequence; ADR 32's preamble -> forced-Hermes -> provider-summary shape is preserved for the foreground tier.

Context. Today a Realtime Agent turn runs Hermes synchronously inside the provider event pump: _pump_provider_events awaits _handle_provider_tool_call -> _run_brokered_tool, which streams the entire Hermes SSE run to completion before the pump consumes the next provider event (broker.py). This is correct and lowest-latency for short Q&A, but it has two costs that grow with task length:

The provider realtime socket sits attached-but-idle for the whole run (a live, billed, audio-clocked WebSocket parked for tens of seconds during research, multi-tool, or desktop/build tasks).
The tool surface already advertises a background vocabulary (hermes_run_task, hermes_get_status, hermes_cancel, hermes_confirm) and a hermes_run_status state machine, but hermes_get_status / hermes_cancel as provider tool calls are unreachable mid-run because the pump is parked. Only the client->relay response.cancel path can interrupt.

The blocking await is also an implicit mutex: it serializes the three audio sources that can produce voice.output_audio.delta (see "Who speaks" below) so they never overlap. Removing it requires replacing that mutex with an explicit floor owner.

Who speaks (confirmed against both supported providers). Up to three mouths exist; only one is active per phase today because of the blocking await:

Realtime provider - xAI grok-voice-latest, OpenAI gpt-realtime-2. Both run with turn_detection: None (relay owns turn boundaries; no server VAD) and audio output modality. Speaks the pre-Hermes acknowledgement and the final post-result summary.
Relay TTS render - xai_tts/openai_tts via _render_provider_audio. A separate, non-realtime synthesizer used as the forced-summary fallback and the legacy render path. Emits the same voice.output_audio.delta wire event as the provider, so Android cannot distinguish them.
Android local TTS - the should_speak long-wait filler, driven by hermes.run.progress. Client-side, not the provider.

Because all three converge on one Android AudioTrack, floor arbitration must happen relay-side, before bytes reach the socket.

Decision. Keep three turn classes; make promotion automatic and default-on, with the relay as the single explicit floor owner.

Tier A - Foreground (short Q&A). Unchanged from ADR 32: provider preamble -> relay-forced Hermes (still awaited) -> provider summary. Lowest latency, trivial floor. This remains the path for any turn that completes inside the promotion grace window.
Tier B - Promoted (long task detected late). Start in Tier A. If the Hermes run has not produced a final result within a grace window (promote_after_ms, default ~6000ms, tunable), the relay detaches the run from the pump: the run continues as a tracked asyncio.Task, the pump resumes consuming provider events, and the provider speaks a short "I've started that - I'll let you know" handoff. Progress continues via hermes.run.progress. When the background run completes, the relay injects the result as a tool result and requests a provider summary at the next floor-idle moment.
Tier C - Explicitly durable. hermes_run_task(mode="background") returns a run handle immediately (no grace window). For tasks the model/profile knows up front are long (research, builds via desktop tools, multi-step). Same completion-injection path as Tier B.

Promotion is the default behavior, not a flag. Grace-period promotion preserves Tier A latency for the common case and only forks when a run actually proves long, so the user never has to pick a mode.

The relay is the single floor owner. A per-session floor state (idle | provider_speaking | hermes_filler | result_pending) gates every audio source:

Only one mouth may hold the floor. The provider holds it by default.
A completed background result does not barge in. It is queued as result_pending and spoken only when the floor returns to idle (provider finished, user not mid-utterance). Provider VAD being off means the relay controls response.create, so it can withhold the summary until the floor is clear.
Android local filler (should_speak) is suppressed whenever the provider holds the floor; it is a Tier B/C long-wait affordance only.
Relay TTS render (mouth 2) may only fire when it owns the floor and the provider has drained, exactly as the forced-summary fallback does today.

Settings (ample, per ADR intent). Surface in Voice Settings -> Realtime Agent, with relay-side realtime_voice config as source of truth and per-profile override:

promotion_enabled (default true) - master switch; false pins Tier A blocking.
promote_after_ms (default ~6000) - grace window before Tier B handoff.
background_default_mode - whether ambiguous long turns prefer promote vs. stay-foreground.
spoken_handoff (default true) - speak the "I've started that" line on promotion vs. silent + visual only.
progress_spoken_after_ms / progress_repeat_ms - reuse existing _HERMES_SPOKEN_PROGRESS_* knobs, now configurable.
result_delivery - speak_verbatim (default; the realtime provider reads the authoritative answer word for word, with relay TTS as the validator's fallback) vs. speak_when_idle (provider/model summary), notify_then_speak (chime/visual, speak on user re-engage), or visual_only.
max_background_runs - concurrent background runs per session (default 1 for the MVP; the existing single-hermes_task field assumes 1).

Protocol additions (relay <-> Android, additive).

hermes.run.promoted - run moved to background; carries run_id, promote_after_ms, spoken_handoff. A false spoken_handoff is also the foreground turn boundary; when true, the following voice.response.done is the boundary. In both cases the background socket, task card, cancellation, progress, and result delivery remain active.
hermes.run.background_completed - background run finished; precedes the provider/relay summary.
Extend hermes.run.progress with tier and floor so the client can render background state distinctly (e.g. a persistent "working on: ..." chip).
hermes_get_status / hermes_cancel become genuinely reachable as provider tool calls in Tier B/C because the pump is no longer parked; no schema change.

Prerequisite (Phase 0 spike) — RESOLVED 2026-05-24. The spike asked how xAI and OpenAI realtime sessions behave when held open and idle. Verdicts (see docs/realtime-voice-poc.md): OpenAI hold-floor-ok (empirical — survived 10/20/30s idle with clean post-idle audio); xAI hold-floor-ok (shipping Realtime Agent already holds xai_realtime sessions open across between-turn idle with turn_detection: None + resume TTL; relay-host probe retained as a regression check, not a precondition). The premise was also superseded in implementation: Tier B closes the pending provider call with an interim ack rather than holding an open response, so the socket only sees the normal between-turns idle gap. This unblocked default-on at the short-window scale. See the 2026-07-08 revision below for xAI's later 900s idle-expiry behavior.

Phase 0 revision (2026-07-08). The xAI verdict was scoped to between-turn idle and broke at the 15-minute scale: a live event log showed xAI closing a quiet conversation with "timed out after 900.0 seconds due to inactivity" (~896s of zero events after a background-run summary finished speaking). Follow-up probe runs proved no keepalive works: neither uncommitted silent PCM nor acknowledged session.update pings reset xAI's 900s timer. The broker now treats an idle timeout as routine provider-session expiry: it logs the expiry, closes the attached Android websocket cleanly while idle, emits no voice.error, and lets the next user turn open a fresh provider conversation seeded from the durable Hermes session. Full findings live in docs/realtime-voice-poc.md → "Idle tolerance" → "Revision (2026-07-08)".

Rules.

Hermes remains the only path for tools, memory, current data, research, side effects, durable context, and confirmations (unchanged from ADR 29/32).
Exactly one audio source may hold the floor at a time; the relay enforces this before audio reaches Android. Background results never barge in.
Android must not read raw Hermes output aloud; spoken output is always a provider (or relay-fallback) summary of the compact Hermes result.
Promotion must be cancel-safe: a promoted/background run is cancelable via both the provider hermes_cancel tool and the client response.cancel path, reusing _cancel_active_hermes.
A turn must never strand: if a background run completes while the WS is detached, the result is replayed through the existing event-ring/resume path on reattach.

Open questions / risks.

Floor arbitration is the hard part and the existing native_forced_* / _should_forward_provider_response_event suppression machinery is already the most fragile code in broker.py. Concurrency stresses it most; the floor-owner state machine should replace ad-hoc suppression, not stack on top of it.
Concurrent background runs (max_background_runs > 1) are out of scope for the MVP; the session model assumes a single hermes_task.
"Notify then speak" result delivery needs an Android affordance (chime + chip + tap-to-hear) that does not yet exist.

Phased rollout.

Phase 0 - provider-idle-tolerance spike (above). Gate for default-on.
Phase 1 - introduce the relay floor-owner state machine under the current blocking behavior (no functional change), with tests that prove single-floor invariants.
Phase 2 - Tier B grace-period promotion behind promotion_enabled, default off, validated on long-task transcripts.
Phase 3 - flip promotion_enabled default on; add Tier C mode="background"; ship the Voice Settings surface.

Key Files:

docs/decisions.md (this ADR; supersedes ADR 32's blocking assumption)
docs/plans/2026-05-24-realtime-background-hermes-runs.md (companion plan)
plugin/relay/realtime_agent/broker.py
plugin/relay/realtime_agent/hermes_tool_broker.py
plugin/relay/realtime_agent/models.py
plugin/relay/realtime_agent/providers/xai.py
plugin/relay/realtime_agent/providers/openai.py
plugin/relay/profile_voice.py
docs/relay-protocol.md
app/src/main/kotlin/com/hermesandroid/relay/viewmodel/VoiceViewModel.kt
app/src/main/kotlin/com/hermesandroid/relay/ui/screens/VoiceSettingsScreen.kt

34. v1.0.0 Vanilla-Hermes-First Relay Plugin Boundary (2026-06-16)

Status: Accepted.

Context: v1.0.0 made a plain upstream Hermes install sufficient for Android chat, Manage, and voice. The remaining Relay code should be treated as a third-party plugin surface: useful and powerful, but additive and cleanly manageable. The legacy installer still creates side effects outside the upstream plugin manager, including an editable/root package install, .pth bootstrap, systemd user unit, shell shims, and external skill-path entries.

Decision: Treat plugin/ as the plugin-manager-owned root for the current repo layout and document the canonical install identifier as Codename-11/hermes-relay/plugin. The plugin manifest carries the tool list, dashboard plugin metadata stays under plugin/dashboard/, and the plugin CLI now includes hermes relay doctor as the single read-only diagnostics surface for agents and operators. The optional legacy compatibility startup hook is managed by hermes relay compat status/install/remove rather than an undocumented installer side effect. New compat hooks load the bootstrap implementation from plugin/hermes_relay_bootstrap/; the repo-root hermes_relay_bootstrap/ package remains only as a backward-compatible import shim for old .pth files. The Python package, plugin manifest, dashboard manifest, and relay runtime all use version 1.0.0 for this stable line.

Runtime boundary:

Vanilla upstream Hermes owns chat, Manage, dashboard auth, dashboard voice, sessions, runs, responses, capabilities, skills, and toolset discovery.
Relay owns pairing, terminal, bridge/device control, relay voice extensions, remote access, media relay, notification companion, desktop tools, and the dashboard Relay tab.
plugin/hermes_relay_bootstrap is legacy compatibility only. It must not be a hidden prerequisite for the Vanilla Hermes path.

Why not add a repo-root plugin.yaml now: Current upstream imports directory plugins from the directory that contains both plugin.yaml and __init__.py. Adding a root manifest while the actual register(ctx) entry point remains in plugin/__init__.py would make the repository look installable as a root plugin but fail or mislead at load time. A future repo restructure can move the plugin entry point to root or split a dedicated plugin repository; until then the subdirectory install is explicit and correct.

Operational rule: hermes plugins remove hermes-relay removes only the plugin tree. It cannot clean every legacy external artifact today. hermes relay compat remove owns the bootstrap .pth hook, while uninstall.sh remains the cleanup surface for legacy service/shim/root-package installs until plugin lifecycle hooks or plugin-owned service commands replace those pieces. The legacy installer now calls hermes relay compat install for the hook, and the uninstaller delegates hook removal to hermes relay compat remove when that command is available, falling back to deleting only the Relay .pth file.

Key files:

plugin/plugin.yaml
plugin/__init__.py
plugin/cli.py
plugin/compat.py
plugin/doctor.py
plugin/hermes_relay_bootstrap/
plugin/after-install.md
docs/upstream-surface-matrix.md

ADR 34 — Structural fence between vanilla-upstream and Relay network surfaces

Status: Accepted (2026-06-17).

Context. The project's load-bearing invariant — the Vanilla Hermes (no-plugin) path must work against unmodified upstream hermes-agent — is enforced only by convention and CLAUDE.md prose, never by structure or test:

All network code lives in one flat network/ package. The client classes are cleanly named by surface (HermesApiClient, GatewayChatClient, DashboardApiClient → upstream; RelayHttpClient, RelayVoiceClient, ConnectionManager, ChannelMultiplexer → relay), but nothing prevents a Vanilla-Hermes-path file from importing a relay client. The discipline is a code-review rule, not a compiler/lint rule.
The Vanilla Hermes path has never been exercised against true vanilla upstream. The staging server runs a fork with relay routes compiled into api_server.py, so "vanilla-only" is an aspiration validated by review, not by CI. When upstream renames or drops a route the app depends on, nothing surfaces it until a user hits it on a server we don't control.

Decision. Three net-additive, behavior-preserving changes:

Package fence. Split app/.../network/ into network/upstream/, network/relay/, and network/shared/. Upstream/relay client and transport code are physically separated; genuinely-neutral utilities and the routing-seam abstractions live in shared/.
Enforced import rule (Konsist). A JUnit-level architecture test asserts upstream imports nothing from relay and vice-versa, and that shared imports neither concrete client. The boundary becomes a failing test, not a convention.
Vanilla-upstream contract test. A CI job clones a pinned, unmodified upstream, boots it with the bootstrap compat hook absent, and asserts the Vanilla-Hermes-path route surface exists (404 = fail; 401/400/405 = pass — existence, not agent behavior). A scheduled run against upstream main acts as a drift siren.

Placement calls (decided by reading, not the plan's guess):

ChatHandler → upstream/ (not shared/). Per ADR 3, chat connects directly to the API server / gateway and the relay carries only bridge + terminal. The handler is fed solely by upstream transports, imports only upstream models, and has zero relay imports.
VoiceAudioClient.kt is split three ways: the VoiceAudioClient interface and the AutoVoiceAudioClient router (interface-only deps) → shared/; StandardHermesVoiceClient → upstream/; RelayVoiceAudioClientAdapter (imports RelayVoiceClient) → relay/. Keeping them co-located would force one file to import both worlds.

Rejected alternatives.

Full ConnectionViewModel transport-strategy split now. The largest clarity win (it's the one true god-object leak — it instantiates both worlds and branches on auth state inline), but also the riskiest change. Deferred until the fence makes the seam obvious; tracked as a follow-up.
Custom ktlint/detekt lint rule to enforce the boundary at the lint gate. Deferred in favor of a Konsist test: the repo's lint step is a ktlint-or-android-lint fallback chain with no architecture tooling, and a custom rule needs a separate Gradle-plugin / lint-AAR module. The Konsist test reuses the existing JVM unit-test infra for a fraction of the setup. Promote to a lint rule later if lint-gate enforcement is wanted too.

Consequences.

A future file cannot quietly pull a relay client into the Vanilla Hermes path — CI fails.
shared/ stays genuinely neutral (ConnectivityObserver, EndpointResolver, HermesLanDiscovery, ProfileApiUrlResolver, voice routing seam).
The contract test converts the standing "upstream-only path never actually validated" caveat into an automated signal, on our pinning cadence.

Key files:

app/src/main/kotlin/com/hermesandroid/relay/network/{upstream,relay,shared}/
app/src/test/kotlin/com/hermesandroid/relay/network/ArchitectureBoundaryTest.kt
gradle/libs.versions.toml (Konsist test dependency)
.github/workflows/ci-android.yml (boundary test in the explicit --tests list)
.github/workflows/ci-contract.yml (vanilla-upstream route-contract job)
docs/plans/upstream-relay-isolation.md

ADR 35 — In-flight Chat recovery is session-centric and client-checkpointed

Status: Accepted (2026-07-10).

Context. A session-backed agent turn can outlive Android's Activity, process, or WebSocket. Current upstream Hermes can reattach an exact live Gateway session and reports whether it is still running plus its user/partial-assistant text, but that server snapshot does not contain Android presentation state such as live reasoning, tool/subagent card phases, lifecycle captions, pending ask cards, or a client-owned background-task chip. Persisted history becomes authoritative only after the turn settles and therefore cannot restore the in-between UI.

Decision. Treat Chat recovery as session-centric instead of request-centric:

Persist active session-backed turns as a bounded set in the shared Android DataStore, keyed by connection/profile context plus durable session id, with a 24-hour expiry. Store each visible assistant state and server-issued ask, including clarify multi-select semantics, but never an entered password/secret or approval response.
On reopen, restore that UI immediately, then recover in this order: exact session.activate using the saved live id; session.resume using the durable id; bounded, positionally anchored history reconciliation.
Separate detach from cancel. Lifecycle teardown and Gateway session/profile/draft/Thread navigation release local callbacks without session.interrupt; selecting a running sibling reclaims its exact live id. Explicit Stop and connection switches still interrupt, and SSE navigation remains exclusive because that transport cannot multiplex live sessions.
Apply the same history fallback to sessions-SSE transport drops and route handoffs. A final persisted transcript replaces the checkpoint and clears it.
When session.resume.inflight includes user corrections, restore the bounded corrections in server order exactly once between the original user turn and the partial assistant response.

Consequences. Reopening Chat or moving between running Gateway chats can continue each assistant bubble with its last-known reasoning and tool state instead of inventing a second prompt or empty spinner. Detached siblings keep the shared Gateway event socket alive and reconnect it after route loss. Tool state that changed while no client was attached remains explicitly last-known until a new event or authoritative history reconcile arrives. Older Hermes builds without live activation degrade to durable history recovery.

Key files:

app/src/main/kotlin/com/hermesandroid/relay/data/ChatTurnCheckpointStore.kt
app/src/main/kotlin/com/hermesandroid/relay/network/upstream/GatewayChatClient.kt
app/src/main/kotlin/com/hermesandroid/relay/network/upstream/ChatHandler.kt
app/src/main/kotlin/com/hermesandroid/relay/viewmodel/ChatViewModel.kt

ADR 36 — Play preflight precedes public Android distribution

Status: Accepted (2026-07-12).

Context. The Android tag workflow historically created the public GitHub Release—including the installable sideload APK—before uploading a Production draft to Play. Play can surface compatibility and pre-launch failures only after an artifact is uploaded, so a Play-detected release blocker could arrive after the project had already committed publicly to the same version on GitHub. A manual Play rollout button did not protect the earlier sideload publication.

Decision. Stable Android releases use two explicit phases:

Private Play preflight. A manual workflow builds and signs final artifacts from dev or untagged main, runs source and minified-DEX compatibility checks, and uploads the Google Play AAB as a Production draft. It records a short-lived proof keyed by version and Git tree hash; no GitHub Release or public sideload APK is created.
Public approval. After Play's pre-review and pre-launch results are reviewed, a separate workflow on main requires explicit confirmation and the exact preflighted tree. It creates the stable tag. The release workflow changes the existing Production draft to completed first and creates the public GitHub Release only after Play accepts that submission.

The proof uses the Git tree rather than commit SHA so the required dev → main no-ff merge may create a different commit without invalidating identical release contents. Any content change invalidates approval and requires another private preflight.

Consequences.

Play-detected blockers can stop a release before the sideload APK is public.
Stable tags cannot bypass missing preflight proof, changed contents, missing Play credentials, or a failed Play production submission.
Pushing a stable tag remains a recovery path, but the tag workflow enforces the same tree proof.
Play reports remain an additional signal, not a complete correctness proof; local/CI tests and final DEX scanning remain mandatory.

Key files:

.github/workflows/play-preflight-android.yml
.github/workflows/approve-release-android.yml
.github/workflows/release-android.yml
scripts/check-android-collection-apis.py
RELEASE.md

ADR 37 — Desktop is CLI/TUI plus an optional Windows management tray

Status: Amended (2026-08-11; originally accepted 2026-07-13).

Context. The Windows Tauri shell grew from a tray convenience into a second desktop client with Chat, embedded PTYs, sessions, plugins, voice, diagnostics, settings, an overlay, and its own daemon ownership. That duplicated the CLI/TUI contract, obscured which surface was authoritative, and made a background tray helper carry a full WebView application architecture.

Decision. Hermes-Relay desktop has exactly two deliverables on the desktop-v* production track (historical cli-v* tags remain immutable):

hermes-relay, the primary cross-platform CLI and terminal TUI. Interactive behavior, pairing, sessions, daemon management, grants, audit, diagnostics, plugins, chat, and voice remain CLI-owned.
hermes-relay-tray, an optional Windows-only management tray. It may display a single compact popup for connection state, paired Hermes hosts, the selected host's access policy, pending grants, daemon controls, authorized clients, activity, and settings. It invokes the installed CLI rather than duplicating its state or protocol logic. Daemon state includes PID liveness and the current User/Administrator privilege level; elevation is an explicit UAC-confirmed daemon action, never a permanently elevated tray. Desktop computer use is a per-host CLI-owned policy, while the tray displays and invokes that contract rather than maintaining private settings.

The management popup is deliberately not a second Hermes desktop client. It must not contain chat, an embedded terminal or PTY, agent sessions, plugins, voice, an always-on overlay, or a general diagnostics dashboard. Pairing may open the real CLI until an equally narrow native pairing dialog exists. The Windows installer places one CLI binary and the tray binary beside each other under ~/.hermes/bin; there is no private bundled sidecar.

2026-08-11 access and host amendment. Stored relay URLs represent distinct Hermes hosts, not peer devices on one relay. The compact host selector lists those local pairings and places Pair another host... inside the selector; when none exist, the selector becomes the single Pair action. Peer sessions for the selected relay appear only under Settings as authorized clients and may be revoked there.

Host access is persisted locally, keyed by canonical relay URL, and defaults to ask. trusted permits the regular command and file tool surface while keeping task-scoped screen/input approval. full_access also permits screen, keyboard, and mouse use without expiring task grants for that host. Full Access never bypasses relay authentication, local audit, revocation, emergency stop, or UAC boundaries. The daemon may connect in a locked zero-tool state under ask, so starting connectivity does not itself require or imply a tool grant.

Consequences.

A narrowly scoped Tauri WebView is allowed for the popup; xterm, tray-owned PTY/process state, and full desktop-client assets remain excluded.
Pending computer-use requests retain a CLI command and are also resolved in a focused in-window dialog, so normal tray operation never requires opening a terminal merely to approve or reject a request.
Normal-user operation remains the default. Administrator desktop-tool access is available through an explicit UAC prompt, and elevated daemon lifecycle actions retain that privilege boundary.
The tray shows host access, active grant and expiry, alerts on pending local approval, supports immediate cancellation, and warns when an Administrator input grant is active.
Local desktop-tool audit entries use a backward-compatible structured event envelope (tool.completed, category, duration, request ID, and optional process exit code). The tray derives its live, filterable Activity view from that local log and treats non-zero exits as attention-worthy even when the tool handler itself returned normally. Successful tray management actions append compatible management.completed events, keeping daemon, host-access, grant, client-revocation, startup, pairing, and update changes in the same local timeline without creating a second state store.
Settings shows only a short Activity preview. A dedicated detail view owns filters, event expansion, and confirmed local-history clearing. Non-zero subprocess exits are warnings rather than user-actionable Issues; only handler failures and aborts contribute to the Issues count.
Host cards navigate to local detail rather than implicitly switching the daemon. Host selection is an explicit detail action, and an optional local alias is stored beside the selected relay URL without mutating the paired session or remote server identity.
The tray is a small Rust/Tauri process; NSIS packages it with the same compiled CLI released separately.
2026-08-14 process-containment amendment. Treat every external executable launch as a bounded resource. Periodic UI refreshes must be single-flight and coalesced across windows, must not start another refresh while one is pending, and must apply a timeout, termination, and retry backoff to every child. Grant-card discovery reads the local bridge directly rather than polling the complete management snapshot. Static probes such as the CLI version, registry settings, and optional ADB availability are cached or refreshed only when the owning setting changes. The tray must remain useful when a probe fails and must never amplify that failure into an unbounded subprocess queue.
Tray lifecycle, refresh, and child-process diagnostics are recorded in the bounded local ~/.hermes/tray.log; daemon authentication, transport, and tool- router lifecycle remain in ~/.hermes/daemon.log. Operational logs exclude credentials and sensitive command arguments. Management failures are reported as failures rather than silently dropped or appended as successful activity.
Rich full-window desktop chat and management remain upstream desktop-product concerns, not a Hermes-Relay surface.
The CLI package version remains canonical for both binaries and the installer.

Key files:

desktop/src/cli.ts
desktop/src/commands/grants.ts
desktop/tray/Cargo.toml
desktop/tray/src/main.rs
desktop/tray/installer/hermes-relay.nsi
desktop/README.md
RELEASE.md

ADR 38 — Dashboard/Gateway is the primary Android connection surface

Status: Accepted (2026-07-18).

Context. Android originally treated the API server URL and bearer key as the identity and prerequisite for every saved connection. The app later gained the upstream Dashboard/Gateway /api/ws transport, dashboard authentication, native Manage, session control, and dashboard voice. That route now supplies the complete standard experience, but onboarding and persistence still framed it as an optional service derived from an API URL. Users with a healthy dashboard were therefore asked for an API server and sometimes entered fake credentials.

Decision. A saved connection represents one Hermes installation and has a stable identity independent of endpoint URLs.

Dashboard/Gateway is standard. It owns primary chat, dashboard auth, sessions, Manage, and Vanilla Hermes voice against unmodified upstream Hermes.
API server is optional. When discovered or explicitly configured, it is an automatic chat fallback and an advanced headless compatibility surface. Its bearer is requested and validated only when that endpoint is configured.
Relay is optional. It adds pairing, terminal, bridge/device control, media, notification companion, enhanced voice, and desktop tooling. It never becomes a prerequisite for standard chat or management.
Relay compatibility shims are narrow and read-only. When upstream Gateway suppresses image_generate lifecycle frames with tool progress disabled, a paired Android client may poll Relay's /chat/image-activity route during that active turn. Relay derives start/completion state from the selected profile's authoritative Hermes session database without proxying chat. Native Gateway lifecycle events take precedence, and absence of the optional route silently restores the vanilla behavior.
Readiness is capability-based. Chat, Manage, Voice, API fallback, and Relay extensions report their own state. A missing optional endpoint does not mark the whole connection unhealthy.
Routing is automatic. Chat prefers Dashboard/Gateway and falls back to the API server only when configured and usable. Users choose a transport only in advanced diagnostics or compatibility settings, not during normal setup. Endpoint discovery may advertise a conventional API route, but does not enable that optional fallback unless the connection has persisted API configuration; cold-start state remains unconfigured until that persisted value is hydrated.

Product flow. Normal onboarding asks for one Hermes address, discovers the Dashboard/Gateway, authenticates through its supported provider, and finishes with a capability summary. Manual endpoint fields live under Advanced. The Connections list leads with server identity and available outcomes rather than ports, keys, or transport names. Existing API-only records and API-first pairing payloads migrate without behavior loss and remain valid headless configurations. An API endpoint or Relay can be added later without recreating the connection.

Consequences.

Dashboard-only Hermes connections can chat, manage, use sessions, and use Vanilla Hermes voice without fake API credentials.
API outages do not degrade a healthy Gateway session; they remove only the fallback capability.
Connection storage, diagnostics, backup/restore, route discovery, pairing, and profile/session scoping must tolerate independently absent endpoints.
Legacy API-only users keep working, but public documentation no longer teaches that compatibility configuration as the normal path.

Key files:

app/src/main/kotlin/com/hermesandroid/relay/data/ConnectionData.kt
app/src/main/kotlin/com/hermesandroid/relay/data/ConnectionStore.kt
app/src/main/kotlin/com/hermesandroid/relay/viewmodel/ConnectionViewModel.kt
app/src/main/kotlin/com/hermesandroid/relay/ui/components/ConnectionWizard.kt
app/src/main/kotlin/com/hermesandroid/relay/network/upstream/GatewayChatClient.kt
app/src/main/kotlin/com/hermesandroid/relay/network/upstream/HermesApiClient.kt
docs/upstream-surface-matrix.md

ADR 39 — Android dashboard redirect auth uses native PKCE

Status: Superseded by ADR 40 (2026-07-27).

Context. Android originally completed redirect-provider dashboard sign-in inside a WebView and imported cookies. Current upstream Gateway can advertise a native authorization-code flow with PKCE, bearer refresh, and WebSocket ticket support. That contract allows the provider to use the user's browser session without exposing browser cookies to the app.

Decision. When /api/status.auth_flows contains native_pkce, Android uses an AndroidX Custom Tab and a lifecycle-owned callback bound to literal 127.0.0.1 on an OS-assigned port. The selected provider, S256 challenge, redirect URI, and CSRF state are sent to upstream. Verifier/state remain only in the sign-in coroutine; callback input is bounded and state-validated before errors or codes are accepted. Tokens are encrypted per connection and attached only to the exact trusted dashboard base. Native bearer exchange requires HTTPS, except literal loopback development. Missing capability selects the legacy cookie/WebView flow; native failures do not silently downgrade.

All dashboard consumers share the same authenticated client policy: Gateway chat and tickets, Manage and cold prewarm, standard voice, and voice config. Local sign-out clears cookies and native tokens and closes the cached Gateway socket.

Consequences.

Redirect-provider sign-in remains in the app task while using the system browser's provider session and security posture.
Process death or cancellation discards the ephemeral authorization and simply requires a new attempt.
Plain-LAN HTTP dashboards must be upgraded to HTTPS before native bearer auth is offered.
Older upstream versions remain usable through the explicitly identified WebView compatibility path.

ADR 40 — Android dashboard redirect auth is provider-compatible

Status: Amended (2026-07-28).

Context. Upstream advertises native_pkce in /api/status.auth_flows for its desktop client. The corresponding /auth/native/* broker is explicitly a desktop system-browser flow: it redirects to a loopback listener owned by the desktop process and returns bearer tokens rather than dashboard cookies. Android incorrectly treated that server-wide capability as a platform-neutral mode selector, so redirect providers such as self-hosted OIDC were sent through the desktop loopback contract.

Decision. Android redirect-provider sign-in uses the upstream dashboard cookie flow by default:

open /auth/login?provider=...&next=... in a full-screen embedded sign-in destination with a normal app bar rather than a modal WebView;
allow the provider to return through the dashboard's public /auth/callback;
import only cookies observed on the configured dashboard origin;
verify the imported session through /api/auth/me;
reject a foreign http://127.0.0.1, localhost, or [::1] /callback navigation instead of following or importing it.

Android does not select /auth/native/authorize merely because it appears in auth_flows. Self-hosted OIDC remains on the cookie contract above. Nous Portal is the narrow exception: its Cloudflare Turnstile challenge rejects embedded Android WebViews, so Android uses the gateway-brokered native PKCE route for that provider when advertised and opens it in a system Custom Tab. As in the official Desktop client, Android does not hardcode the UI provider identifier into the hosted native authorization request; the gateway selects its single native-eligible provider. The callback retains upstream's five-minute window, and failures are classified without recording codes, state, tokens, provider responses, or other authentication material. The ephemeral loopback listener, S256 verifier, state validation, encrypted bearer store, and exact-origin attachment remain app-owned. Public cleartext dashboards are rejected; explicitly configured RFC 1918 and Tailscale-IP dashboard routes retain the same HTTP allowance as their existing cookie sessions. If the provider redirect from a private route declares a canonical HTTPS dashboard callback, Android begins browser authorization on that canonical origin so the temporary PKCE cookie and callback remain same-origin; the one-time code exchange and resulting exact-origin bearer stay bound to the active private route.

Consequences.

Self-hosted OIDC uses the same public callback registered for the dashboard.
Android Manage, Chat, Voice, and onboarding continue to share one verified dashboard cookie session.
A server-wide desktop capability can no longer switch Android into a loopback callback flow.
Android retains a full-screen embedded WebView for compatible dashboard cookie providers, while providers that prohibit embedding use the explicit brokered native route.

ADR 41 — Android owns full-turn interruption and experimental wake detection

Status: Accepted (2026-07-29).

Context. Android barge-in previously armed only when speech playback began. That left agent generation non-interruptible and recreated the microphone/VAD pipeline at the Thinking-to-Speaking boundary. Upstream voice work established a safer full-turn lifecycle, quiet calibration before playback, and phase-aware bare stop behavior. Upstream wake listening is host-local, but enabling that server listener from Android would capture audio on the wrong machine and couple Standard voice to non-standard server behavior.

Decision.

Android owns one barge-in listener per active voice response, spanning Thinking, Speaking, and final audio drain on Standard and Realtime paths. A turn epoch fences callbacks, and teardown completes before replacement capture or another listener can acquire the microphone.
Optional AEC and noise suppression attach to the listener's AudioRecord capture session, matching Android's preprocessing contract. Playback session IDs are not effect attachment targets.
Quiet-room RMS calibration occurs before output and freezes at playback start. The gate follows upstream's 90th-percentile ambient floor, 3× default multiplier, generation/playback minimums, 4,000 RMS ceiling, 500 ms grace, and 80%-majority decision window. Calibration frames cannot trigger. The renderer drives playback phase, ambient drift resumes only in quiet gaps, and grace rearms after gaps of at least one second. Android exposes the multiplier and grace for device tuning while preserving upstream defaults.
Interruption uses the existing active-turn cancellation seam. Late Standard stream content and Realtime audio are suppressed. Silencing does not cancel a promoted Hermes task; explicit background-task cancellation remains the separate destructive intent.
Configurable stop phrases default to exact bare stop and end the active voice chat during generation or playback; an empty list disables the feature. The phrase remains ordinary agent input outside voice chat, and longer requests remain agent input. Existing exact pause/resume controls remain phase-gated to Continuous mode.
A playback interruption arms the upstream-compatible one-shot note for the next model-bound Standard turn, expires after 120 seconds, and is carried in API-local interface context rather than visible or persisted user text. Generation or pre-audio synthesis interruption does not claim that spoken output was cut off, and Realtime relies on its persistent provider session context.
Wake-word detection is Android-local, experimental, and off by default. A user-started microphone foreground service runs sherpa-onnx for the single validated “Hey Hermes” phrase, with an ongoing notification and Stop action. No pre-activation PCM leaves the phone.
Wake and voice share a process-wide microphone lease. Detection releases its recorder before entering the existing voice flow and resumes only after voice exits. There is no boot receiver or server wake-listener control.
The KWS model is downloaded and SHA-256 verified on first enable. Preferences store enabled state, fixed phrase, strictness, decoder confirmation, start-new-session behavior, and a future-safe profile-routing shape. Only active-profile preservation is implemented; profile-specific phrases are intentionally not claimed.
sherpa owns temporal confirmation through numTrailingBlanks. Android treats each non-empty keyword result as a completed event, resets the native stream immediately, reuses its equal-sized normalization buffer, and does not require the same completed result to recur. Voice settings can arm a bounded real-microphone test; test detections report success without entering voice or acquiring a second microphone owner.

Consequences.

Standard voice remains Dashboard/Gateway-backed and works against unmodified upstream Hermes. Local detection is an Android input affordance, not a Relay server dependency.
The sherpa runtime increases Android artifacts for each packaged ABI, while the approximately 6 MB model is device storage rather than APK payload.
Continuous wake listening has visible microphone and battery cost and requires explicit device/acoustic validation before the experimental label can be reconsidered.
The ordinary foreground-service mode may continue listening in the background, but it does not launch an activity from the background. It holds a detection behind an actionable notification until Hermes is visible. Default-assistant integration is a separate Android system role and lifecycle.

ADR 42 — Full assistant wake uses Android's selected VoiceInteractionService

Status: Accepted (2026-07-30).

Context. The microphone foreground-service preview can detect in the background, but Android correctly prevents an ordinary background app from presenting its Activity immediately. Queuing the activation behind a notification is therefore not equivalent to a default digital assistant. Accessibility, overlays, full-screen intents, or server-side microphones would either bypass platform policy, weaken privacy, or break the vanilla-Hermes boundary.

Decision.

Hermes declares a VoiceInteractionService and associated VoiceInteractionSessionService. Android activates it only after the user confirms Hermes as the Assistant role; the app never silently takes the role.
The always-running interaction service remains lightweight and owns only opt-in Android-local sherpa-onnx KWS. Session UI and lifecycle work run in a separate process. The system session opens the existing app voice flow with startVoiceActivity, including the keyguard-supported platform path.
Assistant KWS and the experimental microphone foreground service are separate, mutually exclusive modes. Both use the same model, tuning, privacy boundary, and one-microphone handoff. Voice capture, barge-in, and diagnostics retain their existing process-wide lease.
Session state crosses the process boundary through explicit, package-scoped broadcasts. Exit/cancel tears down the existing voice flow, finishes the system session, and retries local wake ownership only after the microphone is free. Process recreation creates a fresh activation rather than relying on an in-memory Activity reference.
The system session defaults to a compact bottom bar and can expand without changing turn lifecycle. Open full voice disables only the system-owned session UI and foregrounds the app-owned Voice surface. It does not create another voice session or reacquire the microphone. Back collapses an expanded surface first; Back from compact, hide, Stop, or cancel remains terminal, while the hidden session still observes the final Closed state and finishes without cancelling the completed turn.
Connection, chat, and voice runtime ownership is application-lifetime in the main process rather than Activity-owned. The assistant service may initialize that graph and start a turn while no Activity exists; the app UI later binds the same ViewModels, recorder, players, clients, and turn state. The isolated session process never creates voice collaborators.
Standard voice remains dashboard-backed and upstream-only. Assistant mode adds an Android invocation surface; it does not add or require a Relay/server wake endpoint.

Consequences.

Background and locked-screen invocation is mediated by Android's selected assistant UI/session rather than an ordinary background Activity launch.
Users can leave Hermes selected for gesture/power-button invocation while turning continuous KWS off, or remove Hermes through Android's Assistant settings.
Android does not grant third-party assistants Google's dedicated low-power hotword DSP integration. Local sherpa inference keeps pre-activation audio private but can consume materially more battery than the built-in assistant.

ADR 43 — App-owned voice overlay uses a microphone foreground service

Status: Accepted (2026-07-31).

Context. A TYPE_APPLICATION_OVERLAY remains visible over another app, but it does not make its owning process foreground for Android's while-in-use microphone app-op. The first capture could begin during the foreground grace window, while later captures opened successfully but received silenced PCM. Keeping AudioRecord in the existing voice runtime was still desirable: wake, voice capture, barge-in, and diagnostics already share one process-wide owner.

Decision.

Opening the system voice overlay while Hermes is visible starts a dedicated service with foregroundServiceType="microphone" before the app backgrounds. The required ongoing notification explains the microphone access and offers a terminal Stop voice action.
The service never creates an AudioRecord and never acquires a microphone lease. It supplies only the foreground execution capability; the existing VoiceViewModel and VoiceRecorder remain the sole capture owner.
Overlay Hide, Exit, Open Hermes, voice-mode shutdown, add-view failure, and app-task removal stop the service. Notification Stop closes the overlay and exits voice mode rather than leaving an unprotected capture surface visible.
The service is distinct from experimental wake-word listening. Wake remains paused during voice and cannot become a second microphone owner.

Consequences.

Repeated overlay turns can receive real microphone PCM after Hermes moves to the background instead of Android substituting silence.
Background overlay use has an explicit persistent privacy affordance and cannot silently retain microphone eligibility after the overlay session.
Android 14+ while-in-use rules require starting this service from the visible, user-initiated overlay action; an arbitrary background caller cannot create equivalent microphone privilege.

ADR 44 — Floating pets use one measured-rail behavior director

Status: Accepted (2026-08-01).

Context. An app-level pet is visually pleasant only when its movement reads as intentional and never competes with the interface. Treating the entire Compose or accessibility tree as walkable geometry would make text and controls accidental terrain, while independently launched animation effects can interrupt dragging, misrepresent active agent work, or snap to stale coordinates when a scroll changes measured bounds. Chat adds a special case: the pet should walk on top of a response bubble without ever covering its text or jumping through it.

Decision.

One root overlay owns position and arbitration. It consumes no transcript or control-bar layout space; only the pet-sized target handles pointer input.
Appearance persists one 60–120% scale, default 100%, that changes art, touch target, collision footprint, perch eligibility, and route clearance together. The 100% base equals the previous 125% physical size, and legacy stored values are rebased to preserve their rendered size; visual and planner geometry may never scale independently.
Screen owners opt in by publishing live-measured perches, obstacles, scrolling, and modal visibility. The supported terrain is Chat's composer and eligible visible settled message bubbles, Terminal's extra-keys toolbar, root Settings summary/category cards, Appearance section cards, and the persistent status strip on Settings/Appearance/About. No accessibility-tree scan or arbitrary-Composable discovery is permitted.
Registration does not make a surface rectangle walkable. Bubble interiors remain protected obstacles; only an explicitly derived edge rail or touchdown with exact collision-validated endpoints is autonomous terrain. A validation failure waits locally instead of projecting or snapping to invented geometry.
Direct interaction has priority over agent activity, followed by a pending response visit, autonomous roaming, and idle reactions. Activity clips remain truthful because locomotion is eligible only while Hermes is idle.
Drag/drop changes placement without changing the roaming preference. The held and falling states suspend autonomous movement until the placement and landing animation settle, then an enabled pet resumes from the valid landing surface.
Autonomous, recovery, and direct drag/drop routes are distinct motion/debug kinds. Recovery and direct manipulation cannot be reused as autonomous route eligibility. When a layout update has already placed the pet inside a measured obstacle, recovery is limited to the shortest bounded straight egress to a clear edge and stops there.
A response visit is a deterministic bounded terrain journey. Nearby responses use the direct composer/gutter/bubble excursion. Farther visible responses require a complete chain of settled message-top rails with no hop over 210 dp; sparse chats without that chain defer the visit. The newest response gets the full cross and greeting, then the pet may inspect one to three successively older visible rails according to temperament before retracing the same bounded journey. A measured bubble too narrow for walking may contribute a centered, zero-width touchdown when at least 35% of the pet width is supported; it is a transient route step, never an idle or patrol surface. Unsafe geometry and interactive response rows are skipped rather than crossed, and a sparse gap ends exploration instead of inventing a ledge. The active planner uses the same rail-overlap launch point as the debug graph, walking there before takeoff and backtracking past dead-end candidate edges.
Settled Chat uses one text-safe habitat order: the measured side pocket beside the newest settled bubble, then its raised top edge, then the outer composer corner. A changed habitat is reached from the live coordinate, never snapped.
Settings terrain may be published dynamically by reusable components, but it remains explicit opt-in measured geometry with owner scroll/modal state. The app never turns every card, heading, control, or semantics node into terrain. Root Settings and Appearance explicitly publish selected card tops. Their bounded planner may tour several connected levels in either direction and must retrace the exact selected legs; the same maximum transfer length still rejects any card without a pet-sized clear approach.
Temporary suspension preserves screen coordinates. Scrolling stops movement but does not reclassify the route, re-dock, teleport, or dim the pet. Settings, Appearance, and About hide it while a dialog owns the surface. Routes without an approved rail use the persisted logical-edge home.
Motion states correspond to physical movement: directional walking for horizontal travel, jump to the apex, fall through descent and manual-drop settling, and held during drag. Airborne duration grows with route length; anticipation, turn pauses, cycle-quantized walking, shadow height, and landing squash are presentation polish around those honest states.
Calm, Balanced, and Playful alter cadence and cap older-bubble exploration at one, two, or three stops. App animation settings, foreground/activity state, scrolling, dialogs, Android animator scale, and TalkBack touch exploration always take precedence.
Debug builds may expose a default-off Pet path inspector anchored below the app header and Android status-bar inset, leaving header navigation and actions accessible. It starts as a narrow collapsed bar, moves only from an explicit grip, snaps horizontally, persists normalized viewport placement, re-clamps after viewport or size changes, and exposes a reset action. A recoverable PASS mode collapses it and leaves only the unlock control interactive; non-control regions and the diagnostic Canvas remain click-through. It expands on demand. Terrain is the default balanced view; Plan isolates selected and active travel, and Full adds protected bounds, footprint, raw labels, gate, and locomotion state. The views distinguish measured terrain, narrow-bubble touchdown points, collision-checked candidate routes, the selected out-and-back route with directional arrows and numbered stops, and the solid currently active route. The planner keeps an event-driven lookahead ready while behavior pacing is idle, refreshes it on terrain or supported-waypoint changes, and treats an in-flight transfer as atomic. Freeze snapshots only the visualization while planning continues. The expanded actions can disable the persisted overlay request entirely, and diagnostics are cleared when Developer Options are locked.
Installed Petdex pets use the same renderer and planner without manifest edits or asset conversion by the user. The catalog preview resolves through the live renderer and names the exact direct, mirrored, fallback, or mirrored-fallback row for each supported action.

Consequences.

Adding another roam surface requires an explicit measured-rail registration and ownership of its scroll/modal state; it is not automatically inferred.
Bubble visits remain fun but deterministic and text-safe. When geometry is not provably safe, no visit is preferable to a partially obscured message.
Position, activity truth, accessibility behavior, and Petdex fallback semantics share one source of runtime truth instead of drifting across route-local effects.

ADR 45 — Provider/model reasoning levels use an optional capability overlay

Context. Upstream Hermes owns model discovery and selection. Its model.options contract supplies provider/model identities and may report a reasoning boolean, but not every provider exposes an authoritative list of selectable effort values. A client-side provider table would drift, while making Relay mandatory would break the Vanilla Hermes path.

Decision.

Upstream model.options remains the source of provider/model identity. Relay never invents models or gates model selection or chat.
Android normalizes that inventory before publication: repeated rows with the same case-insensitive provider slug are merged, repeated exact model IDs within that provider are collapsed, and different provider slugs remain separate choices even when their labels and model IDs match. Compose keys use the provider slug plus exact model ID, never a display label or list position.
When available, the optional Relay POST /relay/model-capabilities overlay resolves capability metadata for the exact, profile-scoped {provider, model} pairs supplied by the client. A remote caller needs a paired session with an active chat grant; loopback operator calls may omit it.
Resolution uses this precedence: an exact upstream effort list, then an exact Relay effort list, then an explicit upstream reasoning: false, then the canonical advisory set none, minimal, low, medium, high, xhigh, max, ultra.
Only a coherent provider/model identity may produce an exact result. API model aliases may contribute only when their provider resolves uniquely. Missing, malformed, unsupported, unauthenticated, or failed overlay calls keep the advisory set instead of disabling the control.
Exact lists are selectable truth for the next request. Advisory lists are compatibility choices, not a claim that every value is provider-supported. An effort confirmed by the active session may remain visible as current state even when it is no longer selectable for a new request.
Requests are schema-versioned and bounded to 64 exact pairs. Relay may perform bounded provider discovery using host-owned credentials, but returns only capability metadata. Credentials, credential fingerprints, probe details, and internal cache keys never cross the route.
Primary UI copy describes available levels for exact results and standard levels for advisory results. Provider source and overlay details belong in diagnostics, not in the normal composer.

Consequences. Vanilla Hermes remains complete without Relay. A connected Relay can narrow choices after capability discovery without changing the model inventory. Old Relay versions, a missing chat grant, network failures, and unsupported providers fail soft to the same stable advisory behavior. Cached, dynamic, and refreshed catalogs are idempotent at the Android publication boundary, while provider-specific aliases and reasoning capabilities retain their exact route identity.

ADR 46 — Session pin and archive state follows upstream profile storage

Status: Accepted (2026-08-07).

Context. Android's session drawer originally kept pin and archive sets in Compose remember state. The controls appeared to work until the drawer or app was recreated, at which point neither set had a durable owner. Current upstream Hermes exposes both fields on stored session rows, returns them from the Dashboard session list, and accepts pinned or archived through the existing profile-aware session PATCH route. The official client treats the server row as durable truth. A separate Android registry would drift across clients and could collide when two connections or profiles contain the same session id.

Decision.

Android maps upstream pinned and archived fields into ChatSession; the drawer is a stateless renderer of those fields.
Gateway-mode lists request archived=include through the active connection and effective profile's Dashboard route so archived rows survive recreation and remain available for restore.
Pin/unpin and archive/restore use the same profile-scoped Dashboard PATCH contract as rename. API-server mode uses its official session PATCH for pins. Its current list cannot include archived rows, so Android does not offer an archive action there that would become unrestorable after restart. Relay endpoints are never required.
Mutations update the visible row optimistically, roll back on a failed write, and refresh from server truth after success. Revision and scope-generation fences prevent older completions or a profile/connection transition from changing the newly selected namespace.
Deleting a session needs no metadata cleanup on Android because there is no local flag index; the next server list simply omits the deleted row.

Consequences. Pin state, and archive state on the standard Dashboard path, now survive Android process and drawer recreation and agree with other official clients. Identical session ids in different profiles or installations remain isolated by their server database. Hosts predating the upstream fields fail the mutation visibly and retain their previous row rather than reporting local-only success.

ADR 47 — Android follow-up queues are destination- and run-owned

Status: Accepted (2026-08-08).

Context. Android previously represented queued composer submissions as one ViewModel-wide list of strings. Queue presentation and completion dispatch both consulted the mutable visible session, so switching between two running Gateway sessions could display and submit a follow-up in the wrong server conversation. Composer attachments and voice context were also read at dispatch time rather than captured with the queued text.

Decision.

Each queued item snapshots its connection/profile context key, stored session, configured transport, originating local run id, attachments, voice context, and stable queue id at submission.
The queue tray projects only items owned by the visible context and session. Session/profile switching does not rebind or globally expose queued items.
A run must complete before its own items become eligible. Detached Gateway completion carries the exact live session generation and reconciles only the checkpoint with that same durable and live identity, preventing a late older completion from releasing a newer same-session queue.
Dispatch occurs only while the exact destination is visible and idle. A route change, deleted session, connection replacement, cancellation, or unavailable destination removes only the affected queue and reports the loss where user action is required.
In-flight checkpoints persist bounded queued text and voice context. Attachment bytes are deliberately excluded from Preferences DataStore; restoration drops such an item with a visible review-and-resend notice rather than sending an incomplete prompt.
Durable session metadata and live tool-card identity remain independent. Queue ownership consumes their existing connection/profile and checkpoint seams but does not move metadata or Compose expansion state into the queue.

Consequences. Concurrent Gateway sessions retain independent queues and no completion can redirect a follow-up through the currently visible chat. Queue editing restores only the selected item's composer payload. Process restoration is deterministic for bounded text queues, while attachment restoration fails closed instead of silently changing the submitted turn.

ADR 48 — Android profile switching lives in a context shelf

Status: Accepted (2026-08-08).

Context. Agent Passport had become both an inspection/configuration surface and the primary profile switcher, while the Session Drawer already had the separate responsibility of listing the active profile's conversations. Profile switching was therefore buried, duplicated across picker implementations, and the old default alias collapsed two different identities: following the server's sticky default and explicitly selecting the root profile named default.

Decision. Chat owns a compact collapsible Profile Shelf directly below its top app bar. The header toggles it; the active capsule opens Agent Passport; inactive avatars switch context; and the fixed overflow plus Passport's Switch agent control open one canonical full switcher. Presentation order and hiding remain ProfilePresentationStore state owned by ProfileController, including the selected-hidden exception. Local icons remain connection/profile scoped. Compose renders state and invokes ViewModel actions; it never writes a store.

Server default is represented only by SERVER_DEFAULT_PROFILE_KEY/a null selection and uses a home glyph. A profile literally named default retains its own request, lock, icon, presentation, and session keys. Selecting either does not call the upstream sticky-default mutation.

Profile changes preserve the existing activateGatewayProfile and switchProfileContext lifecycle. A live Gateway turn detaches and reconciles to its original session; SSE switching remains disabled while streaming. The destination restores only its connection/profile/transport-compatible last session or starts a draft, never hot-swaps a live session. Model/provider, personality, reasoning, approval, Fast, and YOLO state reset at the ViewModel context boundary before destination session truth can repopulate them.

Consequences. The hamburger remains exclusively the Session Drawer. Agent Passport stays focused on inspection and configuration. The drawer may widen its read-only browse scope to all profiles and organize that combined set by recency, project, status, or profile; opening a cross-profile row performs the same explicit profile-context switch as the shelf. Named-profile ownership uses a deterministic identity accent with an optional per-connection local override, while Server default remains a neutral home identity; color never communicates activity or health. The shelf adds no fake activity indicator, hides for one visible identity, retains 48 dp touch targets, and exposes meaningful TalkBack actions in compact, large-font, light, and dark layouts.

ADR 49 — Android text shares become reviewed chat drafts

Status: Accepted (2026-08-08).

Context. Android users can share selected text between apps, but Hermes Relay did not advertise a sharesheet target. Treating an external intent as a message send would bypass composer review, and handling draft/session state in MainActivity or Compose would split ownership from ChatViewModel and risk writing into whichever session happened to be visible during startup.

Decision.

The shared manifest advertises ACTION_SEND with text/*, so Google Play and sideload builds expose the same user-mediated entry point.
MainActivity accepts only non-blank EXTRA_TEXT from that contract. A process-local identity-fenced handoff retains one pending share across cold Compose initialization and prevents an older completion from clearing a newer intent.
RelayApp waits for onboarding and initial chat-context settlement, then asks ChatViewModel to create a new chat under the already active connection, effective profile, and configured transport before navigating to Chat.
ChatViewModel reuses the existing new-chat lifecycle. Gateway turns retain their detached background reconciliation; SSE keeps its existing exclusive cancellation behavior. No profile, sticky default, or presentation store is changed.
Composer prefill is a buffered one-consumer event. It may wait for Chat to compose, is delivered once, and never calls sendMessage; submission always requires an explicit user action.

Consequences. Shared text lands in a fresh, reviewable draft without crossing profile/session namespaces or creating a new server-side integration. Non-text attachments and multi-item shares remain outside this contract.

ADR 50 — Android transcript reads are explicitly bounded and intent-specific

Status: Accepted (2026-08-09).

Context. Hermes now defaults omitted message limits to the latest 500 rows. Android previously omitted pagination on both the API-server and profile-scoped Dashboard routes, which silently dropped older history and shifted transcript- derived edit ordinals in long sessions.

Decision. Both transports share one 500-row pagination contract. Complete user-visible reads page oldest-first and accept legacy envelopes with no pagination metadata. Reads stop at 50,000 messages or 32 MB of decoded response payload. Recovery explicitly requests one latest page only while the already known transcript fits that window; otherwise it uses complete paging to retain the positional user anchor. Gateway row ids are retained as durable rewind addresses and rebound from the server after a rewrite. A transcript with no row ids keeps the older ordinal contract, while a mixed transcript refuses to edit a row whose durable address is missing. Cancellation propagates rather than becoming an empty transcript.

Consequences. Long session history, sharing, retry, and edit/regenerate no longer operate on a silent latest-500 subset. Recovery stays cheap for ordinary sessions without allowing a bounded window to replace the complete visible history. Durable histories cannot silently downgrade to destructive ordinal guessing, while older Hermes releases remain compatible.

ADR 51 — Android coding-session context is optional upstream metadata

Status: Accepted (2026-08-12).

Context. Current upstream Hermes records a session's working directory, Git branch, and repository root in its session database and returns those fields on Dashboard session-list routes. It also exposes a read-only profile-wide endpoint that recovers the pull request a session created from a narrowly validated tool result. Android previously discarded all of that context, making coding chats indistinguishable in the session drawer.

Decision. Android maps optional cwd, git_branch, and git_repo_root fields from existing Dashboard list responses. For rows with workspace context, it asks POST /api/profiles/sessions/pull-requests and accepts only a positive PR number with a non-blank URL. Unresolved active sessions retry on a bounded cadence and receive one final scan after ending; resolved associations and terminal misses remain cached. Cache ownership is profile plus session id, and ambiguous duplicate ids in an all-profile response are never assigned a transcript result. Android then uses the repository- scoped POST /api/git/review/pr-list view to refresh the matching branch or known PR number's lifecycle state. The drawer displays only the repository basename, exact branch, PR number, and lifecycle state; it does not expose host paths. Both reads are best-effort. A missing route, unavailable GitHub CLI, malformed response, or API-server-only connection leaves ordinary session behavior intact and is not promoted to a session-list failure.

Consequences. Coding sessions become recognizable without introducing a Relay dependency or a GitHub credential path. Repository and branch state can still appear when PR recovery is unsupported, while legacy hosts show no new badges and active sessions can surface a PR created after their first drawer refresh without allowing cross-profile cache collisions.

ADR 52 — Desktop RPC is explicitly device-targeted and capability-honest

Status: Accepted (2026-08-12).

Context. A Relay can retain several authorized desktop sessions, but the desktop channel historically latched one latest WebSocket. Pairing two PCs could therefore make an agent command change targets implicitly. The desktop surface also mixes typed file/process/screen operations with unrestricted terminal and PowerShell execution. Camera, microphone, and attached-device toggles cannot be claimed as security boundaries while unrestricted code still runs as the same Windows user.

Decision. Every desktop advertises a stable installation device_id and display name. The Relay keeps all connected desktop WebSockets, accepts a device selector on every client-routed tool, binds pending responses to that WebSocket, and fails closed when more than one desktop is connected without an explicit target. Health reports enumerate valid targets, and successful RPCs identify the resolved target.

Typed tools are the preferred automation surface and declare a capability such as files.read, files.write, process.manage, screen.observe, or input.control. desktop_terminal, desktop_powershell, detached processes, and background command jobs declare system.execute. This capability is an escape hatch: at user privilege it can transitively reach files, processes, USB devices, camera, or microphone through operating-system APIs. The UI must not present independent hardware-deny toggles as enforceable while system.execute remains enabled.

Existing local policy remains authoritative on each target PC: Ask does not start the headless tool router without consent; Structured withholds desktop_terminal, desktop_powershell, detached process launch, and command job start; Trusted permits typed and execution tools but retains task grants for screen/input; Full Access bypasses those task grants for that Relay host.

Hardware policy is separate and per host. USB defaults Disabled and exposes only typed, serial-bound ADB list, shell, push, pull, install, and bounded logcat operations. Ask raises the dedicated local approval card for every operation; Allow requires explicit confirmation. Full Access does not override the USB policy. Disabled or unavailable backends are omitted from advertised tools. Microphone and camera remain visibly unavailable until bounded brokers, active-use indication, and cancellation exist.

The management UI names the legacy no-tools Ask state Restricted. A separate Ask Every Time preset advertises each available command, file, screen/input, and USB operation behind a per-operation local approval; unavailable brokers remain disabled. New pairings receive Ask Every Time explicitly, while missing or existing legacy policy records retain their previous fail-closed meaning.

Consequences. Agents cannot accidentally execute against whichever PC most recently sent a heartbeat, and a response from another PC cannot satisfy a targeted request. Common operations remain typed and auditable while raw shell power is labeled honestly. Structured mode makes the USB policy enforceable, and device operations appear as their own local Activity category. Microphone and camera controls are not shipped as cosmetic switches.

ADR 53 — Desktop management separates host scope from local-PC scope

Status: Accepted (2026-08-12).

Context. The compact Windows tray had accumulated per-host connection and authorization information alongside local daemon, update, and application controls. That made Settings difficult to scan and left common operator tasks— opening the CLI, viewing the daemon log, running diagnostics, or returning an elevated daemon to normal user privileges—dependent on external instructions. The tray must remain a small management utility rather than regrow into an embedded terminal or general desktop client.

Decision. The three top-level destinations remain Overview, Hosts, and Settings, with scope determining ownership:

A Host detail page is the hub for one paired Relay instance. It owns the local display name, connection state, Relay address/version, pairing and session details, access preset, capabilities, authorized clients, explicit connect, re-pair, client deauthorization, and guarded Forget host actions. Opening the page never silently selects or connects the host.
Settings owns this Windows PC and the installed application. It contains daemon lifecycle and sign-in behavior, CLI launchers, logs and diagnostics, bundle updates, and Help & About. Start UI at sign-in controls only the per-user tray startup entry. Start daemon with UI is a separate opt-in, defaults off for existing installs, and never implies elevation. Per-host access and client lists do not appear there.
Open terminal starts a normal terminal with hermes-relay available. Open Hermes CLI starts the paired remote Hermes TUI in a real terminal; neither action embeds a terminal emulator in the tray.
View daemon log opens the daemon connection/tool-router log. Tray startup, refresh, and child-process failures use the separate local tray log so a UI failure remains observable even when no daemon is running. Run diagnostics delegates to the CLI diagnostic contract rather than creating a second health model and reports both log locations when relevant.
Help & About reports UI, CLI, and connected Relay versions and links to the documentation, troubleshooting guide, and release notes through the default browser. Log and diagnostic shortcuts remain available there as well.

The tray always remains unelevated. Restart as Administrator... is an explicit UAC-mediated action that elevates only the daemon. Return to user mode performs one elevated-daemon stop followed by a normal daemon start. Elevation is not stored as a toggle or sign-in preference because every approved command and input action inherits the daemon's privilege.

Consequences. Relay-specific actions are discoverable from the corresponding host without making global Settings wider. Routine CLI and support workflows no longer require users to find paths or commands manually. Administrator state is visible and reversible, while the management UI and automatic startup retain normal user privilege. The product stays a thin CLI/TUI plus compact Windows management surface; chat, plugins, voice, and terminal rendering remain outside the tray.

ADR 54 — Hermes Secure Link is pinned and service-isolated

Status: Accepted (2026-08-12).

Context. Tailscale Serve is the primary supported secure remote path today, but some operators need encrypted Relay access without adding an external reverse proxy or tailnet. Expanding a plugin-owned listener across API, Dashboard, and Relay would collapse independent trust domains and expose loopback-authorized management routes.

Decision. Keep Tailscale Serve WSS/HTTPS as the primary documented path. Add an opt-in Relay plugin ingress, named Hermes Secure Link, on port 9443 with fixed /relay, /api, and /dashboard namespaces. Pairing carries its exact HTTPS authority and SPKI SHA-256 pin in the signed, operator-reviewed QR. Relay sessions, API bearer authentication, and Dashboard cookie/native bearer authentication remain independent. Dashboard forwarding fails closed unless its upstream OAuth/password gate is active, so loopback-token HTML is never re-exported.

Clients require the QR-provided authority and pin before the first proxy request, retain hostname verification, and fail closed on malformed, missing, or changed trust. A rotation requires explicit re-pairing. Secure candidates are ordered before LAN by default, while LAN remains an independently configured fallback with its existing plaintext acknowledgement.

The three namespaces share one pinned TLS authority but not credentials: Relay keeps pairing/session authentication, API keeps its bearer, and Dashboard keeps its cookie/native bearer. Direct and Tailscale HTTPS routes remain independent fallback candidates. Secure Link verifies continuity with the paired endpoint and protects transport, but it does not establish the physical host's identity or make the listener reachable across NAT or a firewall. Full invariants and acceptance checks live in security-native-proxy.md.

Consequences. Hermes Secure Link adds a narrow pinned-TLS option without becoming a general host proxy or a second authentication system. Failure to initialize it does not make the ordinary Relay unavailable or advertise a candidate. Certificate rotation is explicit, and the vanilla upstream path remains independent of the optional Relay plugin.

ADR 55 — Outbound broker transport carries opaque Secure Link streams

Status: Experimental (2026-08-13).

Context. Hermes Secure Link encrypts an already reachable connection but does not traverse NAT or firewalls. A native zero-configuration remote route needs both host and client to connect outbound without giving a rendezvous service access to Hermes credentials or payloads. It must also preserve multi-device targeting and the direct, Tailscale, and LAN routes.

Decision. Add an optional outbound rendezvous candidate named Hermes Reach. The client and host connector use system-trusted WSS to reach the broker, which authenticates their route credentials and matches an explicit opaque host identifier. The resulting stream carries a second, inner TLS 1.3 connection to the host's existing Hermes Secure Link listener. The client validates the exact authority and SPKI pin from the operator-reviewed pairing QR. The broker and connector route bytes but do not terminate inner TLS, parse Hermes protocols, or receive Relay/API/Dashboard credentials.

Host and client broker credentials are separate. Initial client bootstrap is short-lived and one-time; durable route access is scoped and independently revocable. A host connection supports isolated simultaneous streams with explicit framing, quotas, replay handling, and device targeting. Broker or pin failure never downgrades to plaintext. Secure Link certificate rotation remains an explicit re-pair. The detailed wire, threat, rotation, and acceptance contract is security-broker-transport.md.

Consequences. Hermes Reach remains an explicitly enabled experimental fallback and must be ordered after Tailscale, public TLS, Direct Secure Link, and other supported routes. It is not promoted in normal setup or marketing. Hermes Reach can provide reachability while its broker can observe connection metadata, but a malicious or compromised broker can only deny, delay, replay, or misroute ciphertext; inner TLS detects modification and wrong-host routing. Hermes authorization remains local and the vanilla upstream path remains independent. Implementing the client requires a TLS-over-broker byte-stream adapter; advertising the candidate is forbidden until that adapter and the cross-platform acceptance gates are complete.

ADR 56 — Structured desktop control may use CUA Driver behind Hermes policy

Status: Accepted for phased implementation (2026-08-13).

Implementation note (2026-08-13). The first Windows integration keeps the public desktop_computer_* contract and adds canonical runtime discovery, manifest/version/tool/permission/health checks, an allowlisted CUA adapter, server-owned control-session envelopes, per-session grants, sensitive-target preflight, single-use snapshot tokens, pre/action/post snapshot flow, and local CLI/UI engine status. CUA actions are background-only in this phase; foreground escalation remains reserved and reports disabled. The optional driver is not bundled or updated by Hermes-Relay, and Hermes forces driver telemetry off for every child process it starts. The legacy engine remains the default and the fail-closed fallback while the live Windows acceptance and remaining scope, redaction, and grant-bridge hardening gates tracked in TODO.md stay open.

Second-phase refinement (2026-08-13). CUA is the preferred/default setting for new structured-control sessions; the original Windows input path is named legacy_compat and remains available only as an explicit compatibility choice or a fail-closed pre-session fallback when CUA is unavailable. The backend is selected once per authenticated control session and cannot change mid-session. Window-scoped snapshots/actions use CUA, while the existing read-only full- display screenshot remains a separate system_capture path. Local audit and the UI activity timeline expose bounded high-level fields—backend, dispatch, control session, target app/window identifiers, action, phase, and verification state—while omitting accessibility text, screenshots, entered values, and raw driver responses.

Hermes now owns an explicit Windows lifecycle surface without bundling the driver: computer-use cua status|health|install|check-update|update, with --yes required for install/update. Mutations accept only supported upstream releases (>=0.19.3 <0.20.0), verify the trycua/cua product/version manifest and installer SHA-256 before running a temporary installer under a sanitized environment, then verify the canonical packages/current binary, manifest, version, path, and permission mode. Accessibility health is an explicit diagnostic. There is no automatic install or update.

Context. The first Windows input backend uses PowerShell, SetCursorPos, mouse_event, and SendKeys. It can move the operator's physical pointer and depends on foreground focus. CUA Driver provides target-process/window UI Automation, accessibility snapshots, background input, and session-scoped animated agent cursors without moving the physical pointer. Its full local tool surface also includes foreground activation, arbitrary application control, recording, configuration, and update operations that exceed Hermes-Relay's screen/input capability.

Decision. Adopt CUA Driver only as an optional, version-pinned structured computer-control backend behind the existing desktop_computer_* contract. Hermes remains the outer authority for Relay authentication, target-device routing, per-host capability policy, local grants, emergency stop, and audit. The daemon translates an allowlisted Hermes schema to CUA operations; it never forwards arbitrary CUA tool names or JSON.

Background dispatch is the default. A background_unavailable result returns to the caller instead of silently foregrounding another application. Foreground dispatch, application launch/termination, JavaScript execution, recording, replay, configuration, and driver updates require separate, explicit local authority. Full Access may bypass ordinary task prompts, but never authenticated targeting, sensitive-surface blocks, UAC/session boundaries, audit, emergency stop, runtime validation, or per-action failures.

Every semantic element action requires a fresh pre-action window snapshot, an opaque element token bound to its snapshot generation, authenticated principal, grant, PID, and window, followed by a post-action snapshot that records whether the expected state changed. Grant scope and sensitive-surface policy apply to both accessibility text and pixels.

Multiple animated pointers are virtual agent overlays, not additional Windows hardware cursors. Relay must first attach a server-owned authenticated control session identity—at minimum the Relay session, requester device, chat/run, target device, and request—to each command. The desktop derives cursor identity locally and owns one bounded CUA session per active control session. Model arguments cannot choose or share cursor IDs. Grant expiry, cancellation, disconnect, re-pair, policy downgrade, emergency stop, desktop lock/session change, or daemon shutdown ends the corresponding driver session immediately.

CUA runs in the interactive user's logon session, never Windows Session 0. Initial packaging remains optional and resolves the canonical installed package rather than an untrusted PATH entry. Readiness uses the driver's live manifest, schema, tool surface, daemon status, and permission mode and fails closed on an absent or incompatible backend. Telemetry and driver updates remain explicit operator choices.

Temporary Windows implementation note (2026-08-14). Until trycua/cua#3103 is fixed in the supported driver range, the whole-desktop health_report is not a session-start gate: its fixed UIA timeout can report a false degradation and leave the driver temporarily busy. Operators can re-run that diagnostic from the CLI or UI. Structured actions retain their existing target, grant, snapshot, timeout, and fail-closed checks. Remove this exception when the upstream probe is bounded and cannot poison later actions.

Consequences. Hermes can gain background, element-aware control and clean per-agent animated cursors without replacing its Relay protocol or permission model. The existing PowerShell/User32 backend remains a compatibility fallback until authenticated control-session identity, scoped grants, sensitive-surface enforcement, and lifecycle tests ship. Raw command execution remains an honest escape hatch: strong computer-control isolation is meaningful only when command execution is disabled or the host is already fully trusted.

ADR 57 — Official Desktop Relay UI is a lazy, unified-package runtime plugin

Status: Accepted (2026-08-14).

Context. Official Hermes Desktop now discovers a regular agent plugin's desktop/plugin.js and loads it through @hermes/plugin-sdk. The SDK provides profile-aware ctx.rest() access to the same plugin_api.py namespace used by the web Dashboard, plus native pane, sidebar, status-bar, palette, close, reveal, drag, dock, i18n, query, and unload lifecycles. Registering a pane also adopts it into the live layout, which would violate Hermes-Relay's requirement that plugin load and application lifecycle events never open or focus Relay.

Decision. Ship plugin/desktop/plugin.js beside plugin.yaml and the existing Dashboard half. Keep the Desktop half opt-in and use only public SDK imports. At registration time contribute three labeled open actions, but no pane and no network work. The first explicit action registers one dismissible management pane and calls the plugin-scoped ctx.panes.reveal() method. Subsequent actions reveal the same pane. The host owns close, focus, drag, docking, enable state, hot reload, and disposer execution.

The pane reads and mutates the existing Relay Dashboard backend through ctx.rest() only. It maintains no server cache, starts no timer/socket/polling loop, sends no notification, and makes no request until the user opens a view. Every query key contains the active Desktop profile, while the SDK binds the request to that profile's authenticated backend namespace. Pairing, revocation, and remote-access actions stay user initiated; destructive or host-changing actions require an additional in-pane confirmation.

Security boundary. Runtime Desktop plugins are trusted local ESM with full renderer authority; upstream provides error isolation, not a sandbox. Relay therefore ships fixed reviewed UI code only. It does not load generated ESM, Kotlin, Python, remote modules, or arbitrary action schemas, and it never stores tokens, keys, QR secrets, media paths, or duplicated Relay state. The existing Relay-gated declarative Android Plugin Studio remains a separate host-rendered, digest-approved capability model.

Consequences. One plugin install now exposes the Relay-specific management role in both the web Dashboard and official Desktop without patching Hermes or replacing the standalone Relay CLI/tray. The SDK has no public programmatic pane-coordinate API and its agent focus_pane surface excludes contributed IDs, so movement remains native drag/dock and agent-driven reveal remains unsupported. Physical Desktop certification remains required for multi-window, named-profile, remote/SSH-mapped profile, close/reopen, drag/dock, hot-reload, and renderer-log behavior.

ADR 58 — Android owns bounded cron creation and local reset evidence; connector prompts stay separate

Status: Accepted (2026-08-15).

Context. Three upstream-impact opportunities overlapped Android automation, continuous-session diagnostics, and interactive cards. Current upstream source provides cron.manage over the authenticated Dashboard Gateway and forwards an optional positive repeat value to the existing cron store. It also provides session-bound clarify.respond, approval.respond, sudo.respond, and secret.respond RPCs with request identity and expiry. Separately, the NeMo Relay connector defines gateway-to-gateway prompt, prompt_response, and react operations whose clicking-user authorization and resolver ownership belong to messaging connectors, not Dashboard clients. Upstream telemetry session segmentation is opt-in server configuration and exposes no client mutation contract.

Decision. Android Manage may create jobs directly with cron.manage. The editor sends only name, schedule, task instructions, selected profile, and an optional finite repeat count. Blank repeat preserves upstream schedule-kind defaults. Explicit counts are limited to 1–999 and invalid input is rejected locally because upstream normalizes zero or negative values to unlimited, which would contradict the user's choice. Existing Dashboard HTTP routes remain the list, runs, pause, resume, trigger, and delete surface; Hermes-Relay does not add a scheduler or new Relay endpoint.

Before Android replaces a visible chat context for New chat or Thread entry, it writes one bounded local checkpoint. The allowlist contains only the reset reason, transport kind, structural counts, and boolean lifecycle state. It contains no prompt/message text, IDs, profile names, URLs, paths, media, tool arguments/results, credentials, or telemetry upload. The checkpoint uses the existing app-private reliability ring and appears only in the user-reviewed Diagnostics support bundle.

Android continues to resolve live Gateway asks through their native *.respond RPCs. Connector prompt operations are not implemented in the app or Relay plugin, and generic CARD:{json} actions remain display/send affordances rather than approval resolvers. A future generic Dashboard prompt RPC can be evaluated only if upstream publishes one with request ownership, authorization, expiry, one-answer, reconnect, and replay semantics. Until then, copying connector credentials, option IDs, or reaction routing into the phone would create a second and weaker interaction protocol.

Consequences. Users can create recurring work that stops after a reviewed number of runs on a current Gateway, while older or disconnected gateways fail visibly without a fallback mutation. Reset diagnostics survive an app process loss without retaining conversation content. Existing clarify multi-select, free-text, expiry, reconnect, and one-answer behavior remains canonical, and the connector-specific interactive-card evaluation is closed with no client migration.

ADR 59 — Android attachments use the upstream Gateway upload contract and fail closed

Status: Accepted (2026-08-15).

Context. Hermes exposes two unrelated media planes. Dashboard clients upload user-authored images, PDFs, and files into one live Gateway session through image.attach_bytes, pdf.attach, and file.attach before prompt.submit. Separately, upstream gateway connectors exchange platform media by reference through an authenticated /relay/media service and a capability-gated send_media operation. Connector credentials and expiring references belong to that gateway-to-connector boundary; they are not a mobile-client API.

Android previously treated any Gateway preflight failure alike. A document upload rejected by an older host could therefore fall through to an API-server SSE request that has no document channel. The local bubble still showed its file card even though the agent never received the file. The picker also read an entire provider stream before checking its configured size limit.

Decision. Android keeps outbound attachments on the authenticated upstream Gateway contract. It establishes or resumes the exact stored session, uploads each attachment in order, uses the server-returned @file: reference for generic files, and submits only after every upload succeeds. The legacy dotted image RPC remains the bounded compatibility fallback for older hosts. PDF or generic-file method absence, upload interruption, missing file reference, or an unconfirmed attachment-bearing submit fails the optimistic turn visibly; it never changes transport and silently drops the file. Image/PDF paths staged before a later attachment failure are detached best-effort. Queued messages use the same immutable destination and upload sequence.

Picked content is base64-encoded through a bounded stream. The configured client limit is enforced while reading, and the Gateway client independently checks the decoded size against the upstream 25 MB image and 50 MB PDF limits before creating a WebSocket frame.

The optional Hermes-Relay plugin continues to own only its existing bearer-authenticated agent-to-phone /media/* surface. It does not proxy outbound composer uploads, call upstream connector /relay/media, reuse connector credentials, or duplicate connector-side processing.

Consequences. The transcript cannot claim a file was delivered when the selected route omitted it, oversized providers cannot bypass the app's memory bound, cold sessions upload only after their authoritative live identity is known, and current hosts retain native image/PDF/file behavior. Older hosts still accept images through the established compatibility RPC; unsupported document capabilities produce an actionable retry/update failure instead of a text-only turn.

ADR 60 — Android resource and model-risk warnings follow upstream truth

Status: Accepted (2026-08-15).

Context. Current Hermes Dashboard status reports coarse memory and disk pressure, including a suspected out-of-memory restart, while Gateway model selection can require confirmation for unusually expensive or data-training tiers. Android previously discarded the resource blocks and treated a confirm_required model response as if the switch had succeeded. Recreating either policy in the client would drift from Hermes and would misrepresent older hosts.

Decision. Android parses only the optional public-safe /api/status.memory and /api/status.disk fields supplied by Hermes. It renders a persistent warning for the server classifications elevated or critical, and for the server's last_boot_suspected_oom signal. It does not calculate thresholds, sample the host, retain resource history, or emit telemetry. Missing, malformed, unknown, or unreachable status fails soft to no resource warning.

Gateway model picks continue through the upstream profile/session-scoped config.set {key:"model"} round trip. When its response carries confirm_required:true, Android restores the prior picker state and displays the exact confirm_message. Continue resends the same model/provider request with confirm_expensive_model:true; Cancel performs no mutation. The pending confirmation is bound to the exact profile and session and is discarded after a context change. Older gateways that omit these fields keep the established single-request behavior. API-fallback aliases remain unchanged because their current upstream contract exposes no equivalent confirmation preflight.

This path includes Server default and a pick made on a sessionless draft. A draft pick first materializes a profile-bound session using no raw model or provider create fields, then performs the guarded config.set; failure to obtain that session restores the prior selection. Ordinary session.create also omits model/provider, and a new Gateway chat clears the prior session's model pick, so no later send can reintroduce the unguarded create path. For a named profile, the draft is retained only after the create result confirms the exact info.profile_name; absent or different ownership fails before any model mutation. The same confirmed draft can be reused by a superseding picker revision, avoiding a second empty session or a wedged selection.

Consequences. Android warns before host pressure turns into lost chat or failed persistence and before a confirmed Gateway selection changes cost or data-use posture. Hermes remains the policy authority, Relay remains optional, and no private host details, provider credentials, or client-maintained risk table are introduced.

ADR 61 — Android uses Hermes-owned profiles, static avatars, and animated pets

Status: Accepted (2026-08-15).

Context. Hermes Gateway now owns a bounded profile roster, explicit profile creation semantics, compact ui_meta, and validated static avatar assets. Android previously merged Relay/Dashboard rosters, created through Dashboard HTTP, and displayed only phone-local icons or images copied from a Relay host. Gateway has no profile-delete method.

Decision. On a current host Android capability-gates profiles.list, profiles.create, profiles.get_asset, and profiles.set_asset independently. The list call excludes session previews and becomes authoritative only after a successful response; method-not-found stays sticky for that socket and leaves the established Relay/Dashboard roster intact. Avatar fetches validate decoded base64, declared MIME and size, PNG/JPEG/WebP magic, and the 2,000,000-byte cap. They publish only while both connection identity and refresh generation still match, so a late positive fetch cannot beat a newer has_avatar:false list.

Hermes-owned static avatars win at render time by default, but their cache and the device-local ProfileIconStore use separate keys. A persisted, explicit per-connection/profile phone override can make the local image win without mutating Hermes. Phone selection and Relay host import populate that local side; the shared picker and shared clear act directly on Hermes and never erase the phone image. Animated GIF/WebP profile-icon decoding remains a local override. Upstream animated mascots are instead consumed through profile-scoped pet.info, pet.gallery, pet.select, and pet.disable; Android honors the returned sprite geometry and revision cache contract. Pet archives, image base64, and Sphere skins are forbidden from ui_meta; outbound metadata is bounded to small preferences/references.

Profile creation serializes one reviewed auth choice: shared sign-in sends mirror_credentials:true, share_auth:true; a copied snapshot sends true/false; isolated sends false/false. Best-effort SOUL, model, environment, auth, and voice results remain partial results in the UI. Dashboard create is an explicitly enabled older-host fallback only for the legacy shared/default shape, never for an explicit isolated request. Profile deletion remains authenticated Dashboard DELETE /api/profiles/{name} until upstream publishes a Gateway contract.

Consequences. Static identity follows profiles across clients without silently destroying existing phone or Relay-host choices. Older Gateways keep their prior behavior, credentials never return to or enter Android logs, and phone-local pet packs remain local. Physical two-client/avatar/pet and shared-versus-isolated first-turn/voice certification remains required.

ADR 62 — Android composer drafts are durable and large pastes stay reviewable

Status: Accepted (2026-08-17).

Context. Session-scoped drafts previously lived only in the process-owned ViewModel. They survived navigation and Activity recreation, but process death discarded them. Profile and connection switches could also clear the shared pending-attachment list before the composer's key-change effect saved it. The 4,096-character default message limit rejected larger clipboard inserts before the user could review or send them as files.

Decision. The composer remains keyed by stable connection, owning profile, session, and draft slot. Its production store writes small JSON metadata plus content-addressed attachment blobs under Android's app-private no-backup directory. Writes run on IO, use replace-safe temporary files, retain at most 64 drafts and 128 MB of blobs outside the active draft, and collect unreferenced blobs after send, removal, or pruning. Chat debounces ordinary edits, flushes the latest snapshot on lifecycle stop, removes the active draft immediately after a successful dispatch, and saves the previous owner before restoring a session/profile/connection destination. Pending attachments are no longer cleared by the profile-switch owner before that handoff.

A device-wide Chat setting, on by default, treats one insertion of at least 5,000 characters as pasted text. Android immediately replaces that insertion with a loading pasted-text.txt card, prepares UTF-8/Base64 content off the UI thread, and disables only submission until the attachment is ready. The surrounding composer text remains editable. Gateway delivers the file through upstream file.attach. Because vanilla API-server SSE and proactive Thread transports have no generic-file channel, Android materializes this client-made text attachment back into the outgoing text for those paths while retaining ordinary supported attachments and their existing failure behavior.

Consequences. Closing or recreating the app no longer discards reviewed composer work, navigation cannot rebind a draft or attachment to another agent, and large structured pastes remain visible before send without making the default path depend on Relay. Draft content stays local, is excluded from cloud backup, is cleared on uninstall, and remains subject to the app's attachment size limit.

ADR 63 — Android Bridge authority is connection-scoped, grouped, and fail-closed

Context. The original Bridge safety contract had one persisted master switch and one idle timer. Every command refreshed that timer, so a harmless contacts or clipboard read kept screen-driving authority armed, while timer expiry disabled harmless reads too. Command aliases, method-split clipboard operations, raw intents, and Python-side composite tools also made a flat list of UI command toggles easy to drift away from the actual executor surface.

Threat model. A valid Relay session or agent tool call may be mistaken, prompt-injected, replayed after reconnect, routed to the wrong configured Hermes connection, or upgraded while either endpoint runs an older version. Android OS permission, AccessibilityService, MediaProjection, overlay confirmation, target-package blocklist, and Relay channel grants are necessary independent gates; none is evidence that the user granted a Bridge capability.

Decision. Android owns a closed (HTTP method, path) -> capability registry at the first command boundary. Unknown routes and wrong methods return 403 before event reads, wake locks, confirmations, or executor calls. The master switch overrides every capability. Policy is persisted by stable Android Connection ID, and removing a connection removes its policy.

Capability	Lifetime	Routes
Device/app info	Always/Never	current app, launcher apps (including `/apps` alias)
Contacts	Always/Never	contact search
Location	Always/Never	last-known location
Clipboard read	Always/Never	`GET /clipboard`
Clipboard write	Always/Never	`POST /clipboard`
Media control	Always/Never	play/pause/toggle/next/previous
Communications	Always/Never + per-action confirmation	call, send SMS
Outbound sharing	Always/Never + per-action confirmation	share media, compose MMS
Screen/UI inspection	Explicit lease	tree, nodes, hashes/diffs, events, screenshot
Screen/device control	Explicit lease	tap/type/gesture, keys, app navigation, raw intents/broadcasts

ping, host-only setup, and bounded wait are operational primitives rather than data authority. android_navigate and android_macro receive no composite grant; each primitive route is checked. Notification history and shell remain separate Notification Companion and terminal-channel contracts. There is no read-SMS Bridge command in the audited inventory; adding one later requires an explicit registry entry, capability decision, Android permission review, status/docs update, and tests rather than inheriting Communications.

Screen leases offer 5 minutes, 30 minutes, or 2 hours of idle time. Active screen commands refresh that timer, so there is no fixed wall-clock session limit. A separately warned Until turned off choice has no idle expiry and is intended for a dedicated/dummy device. It survives inactivity and reconnect but still ends on End now, Master off, connection removal, or policy change. Status reports this under capabilities.unlimited, not a fake expiry. Restart, reconnect, master disable, manual revoke, and finite expiry cannot revive revoked authority. Calls, SMS, sharing, and MMS retain their on-device confirmation even when their capability is Always. Android runtime permissions and MediaProjection consent are checked separately at use time; the grant UI never claims to grant those OS authorities.

Migration and compatibility. Missing, malformed, or future policy schemas mean no grants. Existing installs migrate with the master off and every capability denied. The master moves to a v2 DataStore key while writes pin the legacy key false, so downgrading to an APK that does not understand granular policy fails closed. Capability policy is bound to a random install ID stored under Android's no-backup directory, so a cloud or exported restore cannot transfer authority to a destination install; restored connections must be re-authorized. bridge.status adds only capability IDs and expiry timestamps; older Relays cache and pass through the additive payload without interpreting it.

UX rationale. The main Bridge page is a cockpit, not the complete editor: it keeps permanent and screen-access policy directly below Master, followed by one authoritative Unattended Access card rather than a duplicate summary and switch. It guides first setup through a preset or Custom and summarizes only the Android prerequisites required by selected capabilities. Expanding Android access preserves the full permission matrix and Test/Settings actions; Safety preserves all granular toggles, blocklists, confirmation vocabulary/timeouts, overlay setting, trusted actions, and audit history. Screen access distinguishes renewable idle limits from Until turned off, includes End now, and explains the dedicated-device risk. Ending it clears unattended while permanent grants remain available. This follows Android's guidance to request access in context and degrade gracefully, minimize permission scope, and require fresh consent for each MediaProjection session. It also mirrors MCP authorization's least-privilege scope selection without presenting Android app policy as OAuth scope.

ADR 64 — Android settles missing Gateway terminal frames from authoritative session state

Status: Accepted (2026-08-20).

Context. A Gateway turn ordinarily ends with message.complete, but a replacement WebSocket does not replay frames emitted while the prior socket was detached. Current upstream still closes every accepted turn with a scoped session.info carrying running=false, and session.activate returns the same authoritative running state for the exact live runtime. Official Desktop uses that idle state as its settle backstop when message.complete is absent. The TUI owns one live session directly and also restores the returned running state; API-server SSE has its own explicit assistant.completed, run.completed, and done boundaries.

Android previously reactivated the exact live runtime after a mid-turn socket loss but continued waiting only for the missing message.complete. The turn watchdog, streaming placeholder, send queue, and Compose state therefore stayed live even when upstream had already persisted the final answer. The earlier visible-chat Idle reattach fix covered a socket that closed after foreground prewarm; it did not cover this already-active turn state.

Decision. A Gateway turn may settle from session.activate or exact-session session.info only when the turn has already received turn-scoped activity and upstream reports running=false. Pre-start idle snapshots are ignored because they can race prompt admission. This backstop is a successful server-owned settle, not cancellation or transport failure: Android completes the local stream, keeps the durable session identity, performs bounded identity-fenced history reconciliation, and never resubmits through API fallback. Cold open continues to use session.resume; an authoritative resume rejection remains visible and cannot create or switch to a replacement context.

The recovery writes one bounded content-free diagnostic containing only route, missing-terminal phase, and reconciliation action. It records no prompt or message text, credentials, hostnames, URLs, profile names, session identifiers, or filesystem paths.

Consequences. Foreground-open and reconnected chats recover without leaving the session, duplicate submission, wrong-session events, or an arbitrary timer. Normal terminal delivery is unchanged, queued turns retain their existing ownership, Relay remains optional, and unmodified upstream compatibility is preserved. Physical certification across the reported device/network matrix remains tracked in TODO.md.

ADR 65 — Gateway client contracts use reusable on-demand scenario fixtures

Status: Accepted (2026-08-21).

Context. Android, official Desktop, the TUI, and API fallback expose related but non-identical chat lifecycles. Existing tests grew around individual event mappers, client harnesses, and source markers. They proved many local behaviors but did not provide one reusable scenario for a real socket gap, exact-session activation, authoritative history, lifecycle-aware rendering, and physical device evidence. Issue #365 crossed all of those boundaries.

Decision. Hermes-Relay keeps a client-neutral vanilla Gateway fixture with real HTTP/WebSocket JSON-RPC and declarative scenarios. Scenario manifests may declare current-upstream requirements; a separate non-provider source check validates those requirements against a clean unmodified upstream checkout. Android uses both a standalone real-socket instrumentation regression and an external-fixture adapter built from production Gateway, ViewModel, handler, main-looper, lifecycle, and Compose collection paths. A separate opt-in ADB runner owns APK identity, port reversal, instrumentation execution, optional app lifecycle smoke, and bounded privacy-safe evidence.

The lanes are manual and on demand. No cron, scheduled workflow, nightly run, device farm, provider call, real conversation, or automatic radio mutation is created. Device-wide radio changes require an explicit dry-run receipt and a second exact confirmation. Relay-only routes, OAuth browser flows, Desktop/TUI client adapters, hosted devices, and scheduled execution remain future work.

Consequences. Protocol regressions gain a deterministic cross-client vocabulary while client-specific assertions remain with each client. Current upstream drift, Android state-machine defects, rendered lifecycle failures, and physical-device failures are reported as distinct evidence lanes. A physical pass is never inferred from JVM or source checks, and scheduled execution can be considered later without being silently introduced now.

FilesExpand file tree

decisions.md

Latest commit

History

decisions.md

File metadata and controls

Hermes-Relay — Decisions & Implementation Guide

Framework Decision: Kotlin + Jetpack Compose

Architecture Decisions

1. Single WSS Connection with Channel Multiplexing

2. Relay Server as Separate Service (Port 8767)

3. Chat via Direct API, Not Relay Proxy

4. xterm.js in WebView for Terminal

5. tmux for Terminal Session Management

6. Auth: Pairing Code → Session Token

6a. QR Carries Both API and Relay Credentials (updated 2026-05-03)

7. Biometric Gate for Terminal Only

8. Dynamic Personalities over Hardcoded Profiles

8b. Dynamic Command Palette over Hardcoded Slash Commands

9. In-App Analytics (Stats for Nerds)

10. Command Palette + Inline Autocomplete

11. Animated Splash Screen

Deferrals

12. Android as a Hermes Platform/Channel — "Threads" (Shipped 2026-06-28; unified-session model 2026-06-29)

Upstream Platform Adapter Architecture (Researched 2026-04-07)

Implementation Approaches

13. Skill Distribution via external_dirs, not Hub Publish or Hand-Copy (2026-04-11)

14. Inbound Media via Plugin-Owned Relay Endpoint, Not Upstream Gateway (2026-04-11)

15. Pairing + Security Architecture: grants, user-chosen TTL, Keystore, TOFU, Paired Devices (2026-04-11)

CI/CD Patterns (from ARC)

16. Runtime API Server Patch via .pth Bootstrap (2026-04-12)

17. Wake-lock wrapping for gesture dispatch + multi-window ScreenReader + three-tier tapText cascade (2026-04-13)

18. Unattended-access visibility: in-app banner vs. system-overlay chip (2026-04-17)

19. Multi-Connection Support — one app, many Hermes servers (2026-04-18)

20. Dashboard plugin: single plugin with internal tabs + pre-built IIFE bundle (2026-04-18)

21. Agent Profile picker — Hermes profile API routing with overlay fallback (2026-04-18, updated 2026-05-18)

22. Profile-scoped read-only config + skills API (2026-04-18)

Voice Mode — Architecture

1. Voice endpoints live on the relay, not upstream hermes-agent

2. Buffer-not-stream TTS responses (sentence-level client chunking)

3. .m4a (MPEG-4/AAC), not .webm, for recorder output

4. ChatViewModel integration via observation, not callbacks

Alternatives explicitly rejected

23. Branch policy: main + dev (2026-04-19)

24. Multi-endpoint pairing payload + network-aware switching (2026-04-19)

25. First-class Tailscale helper as optional hermes enhancement (2026-04-19)

ADR 26 — Rich cards via inline CARD:{json} line markers

ADR 27 — Desktop control native shell uses Tauri v2 with CLI fallback

ADR 28 - Desktop Android pairing parity and one active tray relay

ADR 29 - Voice output uses streaming TTS as renderer, realtime as agent mode

ADR 30 - Desktop surface plugins and built-in Herm launcher

ADR 31 - Desktop Chat tab and first-run chat route

ADR 32 - Realtime Agent uses provider preamble before relay-forced Hermes

ADR 33 - Realtime Agent foregrounds short Hermes turns and promotes long ones to background tasks

34. v1.0.0 Vanilla-Hermes-First Relay Plugin Boundary (2026-06-16)

ADR 34 — Structural fence between vanilla-upstream and Relay network surfaces

ADR 35 — In-flight Chat recovery is session-centric and client-checkpointed

ADR 36 — Play preflight precedes public Android distribution

ADR 37 — Desktop is CLI/TUI plus an optional Windows management tray

ADR 38 — Dashboard/Gateway is the primary Android connection surface

ADR 39 — Android dashboard redirect auth uses native PKCE

ADR 40 — Android dashboard redirect auth is provider-compatible

ADR 41 — Android owns full-turn interruption and experimental wake detection

ADR 42 — Full assistant wake uses Android's selected VoiceInteractionService

ADR 43 — App-owned voice overlay uses a microphone foreground service

ADR 44 — Floating pets use one measured-rail behavior director

ADR 45 — Provider/model reasoning levels use an optional capability overlay

ADR 46 — Session pin and archive state follows upstream profile storage

ADR 47 — Android follow-up queues are destination- and run-owned

ADR 48 — Android profile switching lives in a context shelf

ADR 49 — Android text shares become reviewed chat drafts

ADR 50 — Android transcript reads are explicitly bounded and intent-specific

ADR 51 — Android coding-session context is optional upstream metadata

ADR 52 — Desktop RPC is explicitly device-targeted and capability-honest

ADR 53 — Desktop management separates host scope from local-PC scope

ADR 54 — Hermes Secure Link is pinned and service-isolated

ADR 55 — Outbound broker transport carries opaque Secure Link streams

ADR 56 — Structured desktop control may use CUA Driver behind Hermes policy

ADR 57 — Official Desktop Relay UI is a lazy, unified-package runtime plugin

ADR 58 — Android owns bounded cron creation and local reset evidence; connector prompts stay separate

13. Skill Distribution via `external_dirs`, not Hub Publish or Hand-Copy (2026-04-11)

3. `.m4a` (MPEG-4/AAC), not `.webm`, for recorder output

23. Branch policy: `main` + `dev` (2026-04-19)

ADR 26 — Rich cards via inline `CARD:{json}` line markers