Repository navigation
Keep orchestration notices flowing after an unacknowledged mail batch - #26351
brennanb2025 wants to merge 4 commits into
Conversation
|
A worker that fetched a mail batch without acknowledging it could silently stop hearing from its coordinator. The outstanding batch blocked notices for the entire mailbox, so later mail piled up while the worker sat idle. Both terminal and native-chat workers now receive the normal notice for later mail while the old batch remains outstanding. Notice ownership is derived for particular messages from existing deliveries and conversation history, preserving live work without blocking independent mail. Repeated checks explain the replay, waiting mail and exact acknowledgement command; explicit acknowledgement still advances to the next batch. Fixed: the mailbox-wide latch, duplicate native-chat turns after failed bookkeeping, lost terminal idle-edge recovery, and query argument growth. Deferred: no remaining findings in this lane. Two independent final reviews found no P0/P1; their remaining query-size finding was reproduced and fixed. Verified on final head Not manually verified: real provider execution, SSH or mixed-version pairing, and Linux/Windows UI. Compatibility was reviewed in source; platform smoke checks and the automated CI suites cover their reported environments. Ready for review; no merge or auto-merge was performed. |
📝 WalkthroughWalkthroughMailbox pointer delivery now selects and revalidates messages by mailbox attention eligibility. Structured-session delivery reconciles queued sends and journal submissions with message provenance. Eligible consuming replay responses include acknowledgement guidance and may include a waiting-message count; CLI output renders this recovery information. Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to Later orchestration mail now gets notices even when an earlier fetched batch has not been acknowledged. Repeated checks also show acknowledgement guidance. No concrete defect was found in the reviewed changes. The one open suggestion is optional hardening of a database query that is safe for its current callers. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/main/runtime/orchestration/db/messages/mailbox-pointer-attention.ts (1)
61-78: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueBatch the
id IN (...)list ingetMailboxPointerAttentionIds.The query binds one SQLite parameter for each ID.
getUndeliveredUnreadMessagesusesjson_each(?)for exclusions so it does not hit the parameter limit.areUnreadMessagesprocesses IDs in batches of 500. Pointer batches are capped byORCHESTRATION_DELIVERY_BATCH_LIMIT, so current callers are probably safe. However, this helper is a public DB method with no size cap. To keep it safe for any caller, usejson_each(?)here as well.♻️ Proposed change
- AND id IN (${ids.map(() => '?').join(',')}) AND ${OUTSIDE_FETCHED_BATCH_SQL}` + AND id IN (SELECT value FROM json_each(?)) AND ${OUTSIDE_FETCHED_BATCH_SQL}` ) - .all(mailboxHandle, ...ids) + .all(mailboxHandle, JSON.stringify(ids))Before you apply this change, check the
INDEXED BY idx_messages_idplan assertion inorchestration-mailbox-notification-consistency.test.ts. That test checks for theid INSQL text.
ℹ️ Review info
⚙️ Run configuration
- Configuration used: Repository UI
- Review profile: CHILL
- Plan: Advanced
- Run ID:
6dda97a9-bab3-4efa-a478-913a9e1807fc
⛔ Files ignored due to path filters (1)
src/main/runtime/orchestration/__snapshots__/preamble.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (39)
config/scripts/vitest-sqlite-database-runtime-files.mjssrc/cli/handlers/orchestration-check-replay-recovery.test.tssrc/cli/handlers/orchestration/message-check-handler.tssrc/main/native-chat/agent-session-wire/structured-agent-session-host.tssrc/main/runtime/orchestration-dispatch-mailbox-delivery.test.tssrc/main/runtime/orchestration-mailbox-notification-consistency.test.tssrc/main/runtime/orchestration-mailbox-unacked-notice.test.tssrc/main/runtime/orchestration/agent-facing-parity.test.tssrc/main/runtime/orchestration/db-messages.test.tssrc/main/runtime/orchestration/db/messages/mailbox-pointer-attention.tssrc/main/runtime/orchestration/db/messages/message-inbox.tssrc/main/runtime/orchestration/db/messages/replay-waiting-count.test.tssrc/main/runtime/orchestration/db/messages/role-mailbox-delivery.tssrc/main/runtime/orchestration/mailbox-pointer-delivery.tssrc/main/runtime/orchestration/mailbox-pointer-eligibility.test.tssrc/main/runtime/orchestration/mailbox-pointer-eligibility.tssrc/main/runtime/orchestration/mailbox-pointer-partial-fetch.test.tssrc/main/runtime/orchestration/mailbox-pointer-stage.tssrc/main/runtime/orchestration/mailbox-pointer-submit.tssrc/main/runtime/orchestration/preamble.test.tssrc/main/runtime/orchestration/preamble.tssrc/main/runtime/orchestration/structured-mailbox-pointer-delivery.test.tssrc/main/runtime/orchestration/structured-mailbox-pointer-delivery.tssrc/main/runtime/orchestration/structured-mailbox-pointer-host.test.tssrc/main/runtime/orchestration/structured-mailbox-pointer-host.tssrc/main/runtime/orchestration/structured-pointer-operation-id.test.tssrc/main/runtime/orchestration/structured-pointer-operation-id.tssrc/main/runtime/rpc/methods/orchestration/messaging/check-delivery-result.tssrc/main/runtime/rpc/methods/orchestration/messaging/check-replay-recovery.test.tssrc/main/runtime/rpc/methods/orchestration/messaging/check-replay-recovery.tssrc/main/runtime/rpc/methods/orchestration/messaging/check-run-pending-mail.tssrc/main/runtime/rpc/methods/orchestration/messaging/check-run.tssrc/main/runtime/rpc/methods/orchestration/messaging/check-worker.tssrc/main/runtime/structured-chat-coordinator-mail.test.tssrc/main/runtime/structured-chat-coordinator-observation-clock.test-fixture.tssrc/main/runtime/structured-chat-mail-queue-recovery.test.tssrc/main/runtime/structured-chat-worker-unacked-notice.test.tssrc/shared/orchestration-check-output.test.tssrc/shared/orchestration-check-output.ts
Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 3 remain after this review.
|
Closing: the unacknowledged-batch notice fix moved into #29006 (one lane for this mechanism), and this branch had gone stale against main. |
ELI5
A worker could silently stop hearing from its coordinator after reading one mail batch without acknowledging it. That batch locked notice delivery for the whole mailbox, so later messages piled up while the worker sat idle.
New mail now gets the normal notice even when an earlier fetched batch still needs acknowledgement. A repeated check explains which batch is being replayed, how much unread mail waits behind it, and the exact acknowledgement command that reaches the next batch.
What Changed
Why
Acknowledgement records that a particular batch was processed; it cannot safely decide whether the mailbox may announce unrelated new work. Deriving exact membership fixes both lanes without another stored flag, expiry timer, implicit acknowledgement or repeated idle reminders.
The common pattern separates waking a consumer from ownership of particular work. The alignment is partial. Intended differences: Orca keeps explicit whole-batch acknowledgement and its existing host queue instead of adding expiring execution claims; existing arrival, restart and idle events drive recovery rather than a periodic sweep. A legacy notice whose original message IDs are missing can be replaced only after explicit withdrawal or its own current turn ends, so unresolved work cannot start twice. A terminal notice already written retains its original count after partial fetching, preserving the remaining mail and composer ownership. Recovery guidance uses the existing check receipt because explicit acknowledgement needs an actionable command. These choices reuse authoritative state and add no stored obligation. There is no temporary deviation remaining.
Linked Issue
Maintainer fix for the verified idle-worker incident; no new issue created.
Visual Proof
BEFORE: the worker received one notice, fetched without acknowledging it, and finished its turn. After later mail arrived, it stayed idle; the later message remained undelivered. This is an actual hidden native-chat window with an isolated database and an offline agent stub.
AFTER: later mail receives a second normal notice while the first delivery is still outstanding. The worker completes the new turn. A repeated check reports one waiting message, and its acknowledgement command returns that later message in a new delivery.
Captured on this laptop through CDP with hidden windows, isolated home/profile/agent configuration, hooks disabled before first launch, and an absolute-path offline Codex stub. The actual runtime, database and native-chat host delivered the notices. BEFORE used
31a5b42; AFTER used3d0d2d2. The final main sync changed none of this fix's production files. During QA, the live screenshot driver also fetched a coordination batch and ended without acknowledging it, stranding the next instruction; its retained transcript reproduced the same failure on the current live app.Testing
The initial CI red was an unchanged main-branch compatibility manifest exceeding its line limit, plus the dependent required-job verification failure. Main fix #26342 resolved both; final-head CI passed with no failures.
macOS tests use real database, orchestration runtime and native-chat host, with only the provider process faked. Linux, Windows and SSH execution are not manually verified. Selection and recovery remain on the owning host; no new remote opcode or required field is introduced, and folder workspaces require no Git assumption.
Review
Two independent plan reviews converged before implementation. The first implementation review found queue-ownership and idle-edge recovery gaps; tests were written first and both are fixed. Two independent final reviewers found no P0/P1. Their remaining query-size finding was reproduced and fixed using one JSON parameter; 93 affected tests and Node typecheck pass, and all 22 static checks passed before its push. Hidden before/after QA and explicit acknowledgement recovery passed. Final local checks and final-head CI after the one main sync passed.
Agent skill upstream boundary
Checklist