Repository navigation
Coordinator prompt goes mute while a Delivery is unacknowledged (mailbox-pointer-delivery.ts:70) — proposal: ack prompt + nextCommand in check #21081
Description
Symptom
The coordinator prompt injected into the coordinator terminal («You have N orchestration message(s). Run orca orchestration check --run …») stops arriving for hours while the run mailbox keeps receiving heartbeat, worker_done and question messages. Observed on Orca 1.4.203 (Windows 11) in two independent coordinators/runs on the same day: three windows of 45 min to 6 h with 4–48 messages accumulated. Every window ended only when a human ran check --ack by hand.
Root cause (mapped and reproduced)
Chain: send → notifyMessageArrived (orchestration/mailbox-notification-coordinator.ts:48-75) → mailbox-pointer-delivery.ts → mailbox-pointer-stage.ts → formatter.ts:112-121 (prompt text).
Gate 1: mailbox-pointer-delivery.ts:70-72 and mailbox-pointer-eligibility.ts:68 return without a prompt while the run mailbox has an outstanding Delivery (view outstanding_deliveries, db/schema/migrate-v41.ts:12-18). A Delivery is created by every check --run (rpc/methods/orchestration/messaging/check-run.ts:132 → db/messages/role-mailbox-delivery.ts:88-104) and is only released when its messages become read=1 through --ack. There is no lease or deadline.
The cycle: the prompt only says Run check --run R (no --ack) → the coordinator runs check → Delivery D is now outstanding → everything that arrives afterwards gets no prompt until someone acks D. check --wait --types … does not work as a wake-up either: getOrCreateMailboxDelivery returns D in replay before applying wakeTypes (role-mailbox-delivery.ts:45-58), and a background --wait creates outstanding Deliveries of its own.
Ruled out: busy terminal (the next idle edge delivers), run-use/consumer_generation (fences check, not the prompt), FIFO ordering (symptom: the first manual check returns only heartbeats in replay; worker_done comes in the next batch).
Reproduction
Real-runtime test (OrcaRuntimeService + SQLite + RPC orchestration.check, same harness as orchestration-mailbox-notification-consistency.test.ts). Happy to open a PR with it.
- Mute: 1 heartbeat → 1 prompt →
checkwithout ack; then 11 heartbeats 5 min apart + 1worker_done, with an agent turn (working → idle) between each: still 1 prompt. - Control:
check+--ackafter each heartbeat → 6 heartbeats, 6 prompts. - Causality: with the two
hasOutstandingMailboxDeliverychecks disabled, the mute scenario emits 13 prompts (expected 13 to be 1).
Why this is a design question, not a small bug
The suppression is intended behaviour from #16904 (orchestration-mailbox-notification-consistency.test.ts:835): one outstanding Delivery per mailbox, never replayed twice. The guarantee is sound; what is missing is that the coordinator never learns it is being silenced by its own unacknowledged Delivery.
Proposal (A + B)
A. Ack prompt instead of silence. When gate 1 would suppress and there are new messages above the watermark, emit once per new batch: «You have N new orchestration messages behind unacknowledged Delivery D. After processing, run orca orchestration check --run R --ack D.» Keeps “one Delivery per mailbox”, never replays the batch; only the emission condition and the text change.
B. check returns nextCommand. Optional field in the result (JSON and text) with the exact ack command: orca orchestration check --run R --ack <deliveryId>. Additive on the wire; removes the “forgot the ack” failure on the agent side.
Alternatives considered: C (prompt text always cites the ack — touches dozens of string assertions, does not help an agent that already forgot); D (lease / periodic re-prompt — per-mailbox timer interacting with the Enter reservation and replay); E (--wait --types ignores an outstanding Delivery without a matching type — changes documented semantics and opens out-of-order FIFO).
Related: #16822 (the prompt pushes the coordinator into polling).
Current workaround for coordinators
After every check --run R that returns a deliveryId, run check --run R --ack <deliveryId> and repeat until deliveryId: null; never leave check --wait running in the background; read-only fallbacks only with --peek (creates no Delivery). The canonical loop in the orchestration skill (check --ack <id> --wait --types … chained) is correct as long as the ack never slips to the next turn.
Activity
AmethystLiang commentedon Sep 16, 2026
thanks for reporting. looking into it
yoyofreeman commentedon Oct 8, 2026
Corroborating evidence from Orca 1.4.222 (Linux), for the exact "prompt goes mute while a Delivery is unacknowledged" behaviour.
A long-running consumer (a pi extension) polls with a consuming
orca-ide orchestration check --terminal <handle> --wait. Each returned batch leaves its Delivery atstatus=outstandinginorchestration.db, and the consumer can never ack it, because the CLI gives it no way to obtain the delivery id:check --jsonreturns onlyresult = { runId, messages, count, acknowledged }- no delivery id.inbox --jsonreturns{ messages, count }- no delivery id.--ackrequires a value:--ackwith none errors--ack requires a value; the only source of the id isdeliveries.idin the sqlite DB.So a normal CLI consumer cannot call
check --ack <delivery_id>at all. In one session we observed 8consumed-not-ackeddeliveries (messagesread=1, deliverystatus=outstanding) accumulated this way, plus 21 untouched orphan deliveries for Runs whose coordinator terminal is gone.Also: a one-shot
orca-ide orchestration check --run <R> --waitused as a background waiter returns the batch and exits, nothing acks it, and exactly the mute-the-prompt result you describe follows (we lost a workerquestionand aworker_donethis way).--peekcomposes with--wait(verifiedcheck --terminal <h> --peek --wait --timeout-ms 1), is non-consuming, and does not create an outstanding delivery - so it is a viable monitor-only pattern.Suggestions: (a) expose the delivery id in
check --json/inbox --json; and/or (b) allow--ackwith no value to acknowledge the prior batch; and/or (c) auto-release/downgrade unacked batches when a one-shotcheck --waitprocess exits.