Summary
Expand the realtime voice tool set so a user can discover existing GooeyPi agent sessions, inspect an agent that needs attention, and send a message to an exact existing session.
This is a client-owned capability, independent of the self-hosted OpenAI Realtime-compatible connection tracked by #135 and implemented in #130. The first implementation and automated live validation should use native OpenAI Realtime. Self-hosted gateway allowlisting and Qwen validation are separate work and do not block this issue.
Proposed tools
list_agents
Arguments:
filter: optional enum active | running | needs_attention | all; defaults to active
project_id: optional exact explicitly granted project ID
query: optional title search
Semantics:
running contains sessions whose durable status is running
needs_attention contains sessions whose durable status is waiting or failed
active is the union of running and needs_attention
- results are top-level, non-archived sessions in the voice session selected harness
- results use stable session UUIDs, never transient runtime IDs
- each result contains bounded presentation data: session ID, title, project, status, live state, updated time, and a short preview
get_agent
Arguments:
session_id: required exact stable session UUID
Return the session state and a small bounded recent conversational snapshot. Omit tool calls, tool results, and internal compaction records. Report truncation explicitly.
send_agent_message
Arguments:
session_id: required exact stable session UUID
message: required bounded message text
Deliver through the existing runtime manager. Queue a follow-up when the target is busy and safely wake an authorized saved session when it is idle. Return delivered, awakened, and queued. This tool sends a message; it does not approve permission prompts or complete extension UI forms.
Design constraints
- Keep one filtered discovery tool instead of separate running and attention tools.
- Use the existing session collaboration catalog, bounded transcript reader, wake logic, and delivery behavior; do not create a second session messaging implementation.
- Validate all arguments in the Electron main process.
- Limit access to the selected harness and explicitly granted projects.
- Keep tool execution in GooeyPi. Realtime providers receive schemas and return function calls, but do not gain filesystem or runtime access.
- Make the contract usable by native OpenAI Realtime and future compatible providers through the same executor.
Acceptance criteria
- Native OpenAI Realtime sessions receive all three tool contracts.
- GooeyPi can list and filter authorized sessions with stable IDs.
- GooeyPi can read a bounded snapshot for an exact listed session.
- GooeyPi can message an exact listed session, including busy queueing and safe idle wake behavior.
- Ambiguous or unauthorized IDs fail closed.
- Unit tests cover schemas, filters, bounds, authorization, and send behavior.
- An opt-in automated native OpenAI Realtime test forces a function call, executes it through the same client tool path, returns the tool output, and verifies the completed response.
- api.server and Qwen changes are explicitly out of scope for this work.
Summary
Expand the realtime voice tool set so a user can discover existing GooeyPi agent sessions, inspect an agent that needs attention, and send a message to an exact existing session.
This is a client-owned capability, independent of the self-hosted OpenAI Realtime-compatible connection tracked by #135 and implemented in #130. The first implementation and automated live validation should use native OpenAI Realtime. Self-hosted gateway allowlisting and Qwen validation are separate work and do not block this issue.
Proposed tools
list_agentsArguments:
filter: optional enumactive | running | needs_attention | all; defaults toactiveproject_id: optional exact explicitly granted project IDquery: optional title searchSemantics:
runningcontains sessions whose durable status isrunningneeds_attentioncontains sessions whose durable status iswaitingorfailedactiveis the union ofrunningandneeds_attentionget_agentArguments:
session_id: required exact stable session UUIDReturn the session state and a small bounded recent conversational snapshot. Omit tool calls, tool results, and internal compaction records. Report truncation explicitly.
send_agent_messageArguments:
session_id: required exact stable session UUIDmessage: required bounded message textDeliver through the existing runtime manager. Queue a follow-up when the target is busy and safely wake an authorized saved session when it is idle. Return
delivered,awakened, andqueued. This tool sends a message; it does not approve permission prompts or complete extension UI forms.Design constraints
Acceptance criteria