Skip to content

[Bug]: Idle agent readiness and orchestration mail notification gate disagree, stranding automatic delivery #19025

Description

@oyishyi

Operating system

Windows

Orca version

1.4.197 (installed desktop runtime)

Details

Summary

An idle coordinator that is not running orchestration check --wait can have pending Run mail without receiving the automatic You have N orchestration messages prompt. The mailbox notification entry uses OSC-title-derived lastAgentStatus plus lastAgentStatusObservedLive, while other native status/input paths can already recognize the same live agent as idle.

The pending message is retained, but a successful send does not cause the recipient to check its mailbox. This also affects an ordinary reply to a sender that has already ended its turn. It makes long-lived project coordinators dependent on a human sending another prompt or on keeping an explicit wait active.

Observed native status disagreement

On the same connected Claude Code session, queried on September 6:

{
  "worktreePs": { "state": "working", "workingMode": "monitoring" },
  "terminalAgentStatus": { "isRunningAgent": true, "status": "idle" }
}

The foreground response had finished; an Artifact comment listener remained in the background. This snapshot demonstrates the different state interpretations. It does not by itself expose the internal lastAgentStatusObservedLive flag, so I am distinguishing it from the gate verification below.

Gate verified in the shipped runtime

Inspection of the matching 1.4.197 installation shows the notification entry effectively doing:

if (leaf.lastAgentStatus !== 'idle' || !leaf.lastAgentStatusObservedLive) return;
// resolve the Run mailbox and attempt notification

This entry does not consult workingMode or terminal.agentStatus. A restored quiet agent may not emit another idle OSC title until it receives new work, leaving the live-observation gate unsatisfied indefinitely. #13203 describes this same restored-idle dead end.

I executed the installed notification functions in isolated fixtures with stubbed runtime dependencies (no fake hooks/title writes were sent to a real agent). Results:

Fixture Notification entry
title status idle, observedLive false blocked
title status idle, observedLive true entered
structured status done, title status unknown blocked
structured working + monitoring, title status working blocked

These are branch-level reproductions, not a claim that every restore/background-task combination has been reproduced end to end.

Related submit/retry gap still present

There is a second stage after the pointer text is written: the runtime records the messages as notified, then schedules Enter about 500 ms later. In the installed delayed-submit function, losing live status evidence or receiving false from the Enter write deactivates the watermark without marking those messages undelivered again. The same isolated fixtures reproduced both branches. Message bodies remain stored, but the notification attempt can lose its redrive opportunity.

#17470 is already represented in this installed code: delayed Enter accepts idle or working. It does not remove the live-observation requirement or repair every failed/unknown submit branch. This report is not asking to revert that fix.

Reproduction / acceptance scenario

  1. Bind a coordinator Run to a live Claude Code terminal. Let the foreground turn complete, then let the terminal sit idle without a check --wait invocation.
  2. Exercise a restore/reconnect where the terminal does not emit a fresh idle OSC title. Also cover a completed foreground turn with a background-only listener.
  3. From another coordinator, run orca orchestration send --to run:<recipient_run> --subject test --body test --json.
  4. Compare worktree ps, the native terminal.agentStatus RPC, and read-only orchestration check --run <recipient_run> --terminal <recipient_handle> --peek --json. Verify the current Run coordinator handle first, so a stale recipient binding does not confound this test.
  5. The recipient should be prompted and consume/acknowledge the Delivery without manual input or a pre-existing wait. Repeat in the reverse direction using orchestration reply after the original sender has returned to idle.
  6. For deterministic regression coverage, clear title live evidence after staging the pointer, and separately reject the delayed Enter write. Verify that these outcomes remain recoverable and are not reported as successful notification.

Steps 1–5 describe the product scenario to cover; the deterministic gate/submit branches above are the verification performed for this report. No computer restart or inferred update condition is required as the claimed cause.

Expected behavior / suggested direction

  • Use a shared, current-session-aware input readiness decision for native prompts and mailbox notifications. A completed foreground turn with a passive background listener should be eligible; permission prompts, stale process identity and uncertain input ownership still need protection.
  • Re-drive pending notifications when the relevant live readiness evidence or binding changes, rather than requiring a new idle OSC title as the only useful way out.
  • Distinguish text staged, submit confirmed and submit unknown. Do not treat an accepted PTY write as proof that the model received the prompt, and do not blindly paste duplicate text after an unknown submit.
  • Keep the original Run mailbox and unacknowledged Delivery durable through recovery.

This is not a request to set lastAgentStatusObservedLive=true externally or to inject into every non-thinking terminal. The issue is that the current gate can stay closed after other native interfaces recognize an available receiver.

Related work: #13203 (restored idle), #16198 (shared input ownership), #15274 (submit verdict), #18731 (mail waking slept agents). A pending PR is not assumed to be part of 1.4.197.

The earlier version of this issue was withdrawn. This revision removes an unproven attribution to an installed-but-not-restarted CLI update and separates observed state, isolated verification and proposed acceptance coverage.

Activity

  1. changed the title [-][Bug]: orchestration mail is never delivered when a live pane stops emitting OSC title status (delivery path has none of the tui-idle fallbacks)[/-] [+][Bug]: Idle agent readiness and orchestration mail notification gate disagree, stranding automatic delivery[/+] on Sep 6, 2026
  2. reopened this on Sep 6, 2026
  3. added theissue type on Sep 19, 2026
  4. minsu-kwon-3dve commented on Oct 2, 2026

    @minsu-kwon-3dve

    Fresh end-to-end reproduction of this on 1.4.218, with a supervised Codex worker on an SSH remote host after a desktop app quit/relaunch. The process-incarnation re-find from #23583 works (the worker is reported live), but mail sent afterward is never pointed into the idle pane, so the dispatch stalls indefinitely.

    Environment

    • Orca desktop 1.4.218 (macOS), SSH remote host, relay 0.1.0+f6e4b640b122 (unchanged across the relaunch)
    • Worker: Codex CLI 0.157.1 launched with --no-daemon, started by orca orchestration worker-start --worktree new-top-level ... --agent codex
    • Coordinator: Claude Code in an Orca terminal

    Steps to reproduce

    1. Start a supervised Codex worker. Its task loops 5 times: sleep 60, then the preamble command orca orchestration check --run <run_id> --dispatch <dispatch_id>.
    2. While the 5th check is in flight, quit and relaunch the Orca desktop app (same version; relay keeps running).
    3. The worker's check fails with Request "orca.cli" timed out after 300000ms. Following its contract, the worker does not send worker_done and ends its turn. The Codex TUI now sits idle at its prompt.
    4. After reconnect, the coordinator runs orca orchestration worker-list --include-remote.
    5. The coordinator runs orca orchestration send --to dispatch:<dispatch_id> --subject <subject> --body <body>.

    Expected

    The idle worker pane receives You have 1 orchestration message. Run `orca orchestration check --run <run_id>`. plus Enter, as it does when the worker goes idle while the app stays connected.

    Actual

    • Step 4: worker liveness live (same PTY incarnation), stage.activity: done, agentWait: null.
    • Step 5: send returns "ok": true.
    • No pointer ever appears in the worker transcript (checked about 12 minutes later). The worker stays idle until a human types into it.

    We saw the same symptom on 1.4.209 in production after a Mac sleep/disconnect. That time the worker reported liveness unverifiable (stale_status), and neither reply nor send --thread-id re-delivery reached it.

    Suspected cause

    This matches the gate described above. I read the code at main @ 1fc24d3 (the files below have the same gate in v1.4.218):

    • src/main/runtime/orchestration/mailbox-pointer-delivery.ts L42: deliverForHandle returns early unless leaf.lastAgentStatus === 'idle' && leaf.lastAgentStatusObservedLive. Mail arrival reaches this through mailbox-notification-coordinator.ts notifyMessageArrived (L48, L69). The restart repoint path also reaches it, through scheduleRestoredMessageRepoints and deliverPendingMessagesForHandle in orca-runtime-get-pty-record-for-pane-key.ts (L188, L315).
    • src/main/runtime/orca-runtime-apply-tracked-pty-title.ts L115-116 and L144-150: in production code, lastAgentStatusObservedLive = true is set only here, when a live OSC title frame arrives.
    • src/main/runtime/orca-runtime-maybe-hydrate-headless-from-renderer.ts L113-152 (applySeededAgentStatus): after a relaunch the restored status is seeded on purpose without the live flag. The graph resync in orca-runtime-sync-window-graph.ts (L143-144) defaults it to false.
    • An idle Codex TUI paints its idle title once, on the working→idle edge, and then stays silent. That edge happened before the relaunch, so the gate never opens. tests/e2e/orchestration-idle-mail-restore.spec.ts covers the restart path only by re-emitting the idle title from the fake agent (L170). A real idle agent does not send that frame.

    From the user's point of view, worker-list says the worker is live, but the mailbox gate still treats it as unknown. That is the readiness disagreement this issue reports.

    Suggested direction

    • Land something along the lines of fix(orchestration): wake a restored idle pane that never speaks again #13203: when a pane has pending mail and a restored or unknown (or seeded-idle) status, open the gate on positive evidence. Examples are output quiescence across a window plus a recognized foreground agent, or the existing tui-idle screen evidence (fix(runtime): apply the tui-idle evidence ranking to mailbox delivery #20578). This should run on reconnect or graph resync, and when mail arrives, so it does not depend only on a fresh idle title.
    • Make sure this works for SSH/relay PTYs, where foreground-process reads may be unavailable or slower than for local PTYs.
    • Optional: after a relaunch, check workers that are re-found as live with stage.activity: done and unacknowledged mail. If they still cannot be pointed, report them as such (for example mail_pending_unpointed) so the coordinator sees why the worker is not waking, instead of a bare "ok": true from send.

    Separately, though secondary: a worker CLI call made while the desktop restarts blocks for the full 300 s before it fails. That makes it likely the worker ends its turn while the app is disconnected, which then hits the dead end above.

  5. Jinwoo-H commented on Oct 28, 2026

    @Jinwoo-H
    Contributor

    Found a new cause in this family and fixed it in #34706.

    A Claude Code coordinator with an agent-teams teammate alive keeps reporting working after its own turn ends. The mail gate only typed the You have N orchestration messages wake into a pane that looked idle, so the wake never came and the mail stayed undelivered (reproduced: 3.5+ minutes with no delivery).

    #34706 removes the idle requirement instead of guessing when the turn ended. Every agent we tested (Claude Code, Codex, OpenCode, Antigravity, Grok, pi) queues or steers text typed mid-turn without stopping the running tool, so a pane now takes the wake when it is idle or working. It is still held while a dialog or permission prompt is open, after the agent exits to a shell, and before the pane has finished starting. In the same repro the teams lead now gets the wake about 1 s after worker_done, and a busy coordinator reads it once its tool call finishes.

    This does not cover every disagreement described here (for example terminal wait --for tui-idle), so I am leaving the issue open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions