Skip to content

Coordinator prompt goes mute while a Delivery is unacknowledged (mailbox-pointer-delivery.ts:70) — proposal: ack prompt + nextCommand in check #21081

Open

Description

@Tauri-EPO

Symptom

The coordinator prompt injected into the coordinator terminal («You have N orchestration message(s). Run orca orchestration check --run …») stops arriving for hours while the run mailbox keeps receiving heartbeat, worker_done and question messages. Observed on Orca 1.4.203 (Windows 11) in two independent coordinators/runs on the same day: three windows of 45 min to 6 h with 4–48 messages accumulated. Every window ended only when a human ran check --ack by hand.

Root cause (mapped and reproduced)

Chain: send → notifyMessageArrived (orchestration/mailbox-notification-coordinator.ts:48-75) → mailbox-pointer-delivery.ts → mailbox-pointer-stage.ts → formatter.ts:112-121 (prompt text).

Gate 1: mailbox-pointer-delivery.ts:70-72 and mailbox-pointer-eligibility.ts:68 return without a prompt while the run mailbox has an outstanding Delivery (view outstanding_deliveries, db/schema/migrate-v41.ts:12-18). A Delivery is created by every check --run (rpc/methods/orchestration/messaging/check-run.ts:132 → db/messages/role-mailbox-delivery.ts:88-104) and is only released when its messages become read=1 through --ack. There is no lease or deadline.

The cycle: the prompt only says Run check --run R (no --ack) → the coordinator runs check → Delivery D is now outstanding → everything that arrives afterwards gets no prompt until someone acks D. check --wait --types … does not work as a wake-up either: getOrCreateMailboxDelivery returns D in replay before applying wakeTypes (role-mailbox-delivery.ts:45-58), and a background --wait creates outstanding Deliveries of its own.

Ruled out: busy terminal (the next idle edge delivers), run-use/consumer_generation (fences check, not the prompt), FIFO ordering (symptom: the first manual check returns only heartbeats in replay; worker_done comes in the next batch).

Reproduction

Real-runtime test (OrcaRuntimeService + SQLite + RPC orchestration.check, same harness as orchestration-mailbox-notification-consistency.test.ts). Happy to open a PR with it.

  • Mute: 1 heartbeat → 1 prompt → check without ack; then 11 heartbeats 5 min apart + 1 worker_done, with an agent turn (working → idle) between each: still 1 prompt.
  • Control: check + --ack after each heartbeat → 6 heartbeats, 6 prompts.
  • Causality: with the two hasOutstandingMailboxDelivery checks disabled, the mute scenario emits 13 prompts (expected 13 to be 1).

Why this is a design question, not a small bug

The suppression is intended behaviour from #16904 (orchestration-mailbox-notification-consistency.test.ts:835): one outstanding Delivery per mailbox, never replayed twice. The guarantee is sound; what is missing is that the coordinator never learns it is being silenced by its own unacknowledged Delivery.

Proposal (A + B)

A. Ack prompt instead of silence. When gate 1 would suppress and there are new messages above the watermark, emit once per new batch: «You have N new orchestration messages behind unacknowledged Delivery D. After processing, run orca orchestration check --run R --ack D.» Keeps “one Delivery per mailbox”, never replays the batch; only the emission condition and the text change.

B. check returns nextCommand. Optional field in the result (JSON and text) with the exact ack command: orca orchestration check --run R --ack <deliveryId>. Additive on the wire; removes the “forgot the ack” failure on the agent side.

Alternatives considered: C (prompt text always cites the ack — touches dozens of string assertions, does not help an agent that already forgot); D (lease / periodic re-prompt — per-mailbox timer interacting with the Enter reservation and replay); E (--wait --types ignores an outstanding Delivery without a matching type — changes documented semantics and opens out-of-order FIFO).

Related: #16822 (the prompt pushes the coordinator into polling).

Current workaround for coordinators

After every check --run R that returns a deliveryId, run check --run R --ack <deliveryId> and repeat until deliveryId: null; never leave check --wait running in the background; read-only fallbacks only with --peek (creates no Delivery). The canonical loop in the orchestration skill (check --ack <id> --wait --types … chained) is correct as long as the ack never slips to the next turn.

Activity

  1. AmethystLiang commented on Sep 16, 2026

    @AmethystLiang
    Contributor

    thanks for reporting. looking into it

  2. added theissue type on Sep 19, 2026
  3. yoyofreeman commented on Oct 8, 2026

    @yoyofreeman

    Corroborating evidence from Orca 1.4.222 (Linux), for the exact "prompt goes mute while a Delivery is unacknowledged" behaviour.

    A long-running consumer (a pi extension) polls with a consuming orca-ide orchestration check --terminal <handle> --wait. Each returned batch leaves its Delivery at status=outstanding in orchestration.db, and the consumer can never ack it, because the CLI gives it no way to obtain the delivery id:

    • check --json returns only result = { runId, messages, count, acknowledged } - no delivery id.
    • inbox --json returns { messages, count } - no delivery id.
    • --ack requires a value: --ack with none errors --ack requires a value; the only source of the id is deliveries.id in the sqlite DB.

    So a normal CLI consumer cannot call check --ack <delivery_id> at all. In one session we observed 8 consumed-not-acked deliveries (messages read=1, delivery status=outstanding) accumulated this way, plus 21 untouched orphan deliveries for Runs whose coordinator terminal is gone.

    Also: a one-shot orca-ide orchestration check --run <R> --wait used as a background waiter returns the batch and exits, nothing acks it, and exactly the mute-the-prompt result you describe follows (we lost a worker question and a worker_done this way).

    --peek composes with --wait (verified check --terminal <h> --peek --wait --timeout-ms 1), is non-consuming, and does not create an outstanding delivery - so it is a viable monitor-only pattern.

    Suggestions: (a) expose the delivery id in check --json / inbox --json; and/or (b) allow --ack with no value to acknowledge the prior batch; and/or (c) auto-release/downgrade unacked batches when a one-shot check --wait process exits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions