Skip to content

[Bug]: Worker recovery errors omit the failed condition and applicable next step #16529

Open

Description

@Tauri-EPO

Current scope — 2026-09-04

The remaining problem is guidance at the failure site. worker-start --retry-of already exists, and the installed 1.4.197 orchestration guide documents both it and request-show; #16919 also improved recovery guidance. The original five/seven-invocation walk is historical, not a necessary current recovery sequence or evidence that retry is missing.

The current worker-start validation still collapses different retry eligibility checks into:

Task <task> cannot retry from Dispatch <dispatch>.
Task <task> is failed; only a ready Task can start.

The first can mean a missing/wrong Dispatch, a newer attempt, a missing or ineligible worker, or an ineligible Task. The caller needs the applicable reason, not a command that is safe only under one of those conditions.

Requested behavior

  1. Name which eligibility condition failed and provide a valid next inspection or recovery command. Include the current attempt when it is known and the caller may inspect it; [Other]: a rejected lifecycle message names the revoked Dispatch and never the current one, so a worker cannot self-correct after a retry-of #15972 covers that identifier request.
  2. Explain the worktree/terminal mismatch and unsupported setup/model flags in terms of the supplied selector or flag. Accepting redundant flags would be a separate compatibility decision, not required to improve the error.
  3. Do not equate agent_unconfigured with a dead process. Inspect known execution-host evidence and distinguish a recognized exit from an unrecognized or unverifiable agent before suggesting a new terminal.
  4. Preserve idempotency guidance after runtime_unavailable: recover the existing request before making another start. Do not recommend blind reinjection when delivery is uncertain.

Existing evidence

The original Windows/Orca 1.4.188/Codex session recovered a runtime_unavailable request with --retry-request, found a readiness failure and then encountered terminal_worktree_mismatch, agent_unconfigured, an invalid setup option and task_not_startable. I discovered --retry-of only after that walk and corrected the original request.

A later retry report encountered the generic retry refusal; its proposed newer-Dispatch cause was an inference. A macOS/Claude report records the same agent_unconfigured text for an exited session and a live unrecognized one. Those observations motivate better explanations but do not establish every proposed remedy as safe.

No new recovery sequence was run for this edit. Related: #16377 (the original start failure), #17291 (agent recognition), #15944 (residual resources).

Activity

  1. IsidroLOlguin commented on Aug 29, 2026

    @IsidroLOlguin

    Ran into the --retry-of half of this directly: a dispatch that failed at dispatch_input with agent_prompt_stalled (unrelated root cause, see #17291), then:

    worker-start --task <task_id> --retry-of <ctx_id_of_the_failed_dispatch> --worktree ... --terminal ... --json
    

    returned:

    {"ok": false, "error": {"code": "task_not_startable",
     "message": "Task <task_id> cannot retry from Dispatch <ctx_id>."}}

    even though that dispatch was never fenced with worker-stop/worker-abandon — it just failed at dispatch_input. Re-issuing worker-start without --retry-of on the same terminal worked immediately, so the documented "intended remedy" from point 4 (--retry-of) doesn't actually help you skip the manual walk when the failing condition is anything other than "task not ready" — the message doesn't say which of the eligibility checks failed, so there's no way to tell from the error whether --retry-of is even usable without trial and error.

    Traced it in src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-start.ts: the eligibility check for --retry-of collapses five distinct conditions into one generic message:

    if (
      !prior ||
      prior.task_id !== task.id ||
      latest?.id !== prior.id ||
      !priorWorker ||
      !['failed', 'stopped', 'abandoned'].includes(priorWorker.state) ||
      !['failed', 'blocked'].includes(task.status)
    ) {
      throw new OrchestrationError('task_not_startable', `Task ${task.id} cannot retry from Dispatch ${params.retryOf}.`)
    }

    Confirmed by reading the code (not guessing) that a dispatch failing at dispatch_input does leave worker.state = 'failed' and task.status = 'failed', so it should be a valid --retry-of target — the actual failure mode in my case was almost certainly latest?.id !== prior.id (something created a newer dispatch context for the same task between my failed attempt and the retry, so the ctx_id I had stopped being "the latest").

    This is exactly the "recovery errors should carry next steps" ask from this issue's point 1 — same-shaped fix, applied to --retry-of specifically:

    --- a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-start.ts
    +++ b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-start.ts
    @@
    -      if (
    -        !prior ||
    -        prior.task_id !== task.id ||
    -        latest?.id !== prior.id ||
    -        !priorWorker ||
    -        !['failed', 'stopped', 'abandoned'].includes(priorWorker.state) ||
    -        !['failed', 'blocked'].includes(task.status)
    -      ) {
    +      const retryOfReason =
    +        !prior || prior.task_id !== task.id
    +          ? 'dispatch not found or belongs to a different task'
    +          : latest?.id !== prior.id
    +            ? 'not the latest dispatch for this task'
    +            : !priorWorker || !['failed', 'stopped', 'abandoned'].includes(priorWorker.state)
    +              ? `worker state '${priorWorker?.state ?? 'unknown'}' is not retryable`
    +              : !['failed', 'blocked'].includes(task.status)
    +                ? `task status '${task.status}' is not retryable`
    +                : null
    +      if (retryOfReason) {
             throw new OrchestrationError(
               'task_not_startable',
    -          `Task ${task.id} cannot retry from Dispatch ${params.retryOf}.`
    +          `Task ${task.id} cannot retry from Dispatch ${params.retryOf}: ${retryOfReason}.`
             )
           }

    Same five conditions, no business-logic change — just tells the caller which one tripped so they know whether to retry with a fresher ctx_id, drop --retry-of entirely, or that the worker/task state is genuinely not retryable.

  2. arthur-sophiieAI commented on Sep 1, 2026

    @arthur-sophiieAI

    Same failure shape on macOS + --agent claude, so this is not Windows- or codex-specific. Adding it because the report's point 1 — every recovery error names the condition and never the next step — held identically here, and because our data separates two states that agent_unconfigured currently collapses into one message.

    Environment: macOS 26.5.1, Orca 1.4.18x (August 2026; exact patch not recorded per occurrence), --agent claude. Coordinator stack is now on 1.4.193; the re-attach path has not been re-measured since.

    What we hit

    Three occurrences, all re-attaching a task to a terminal that already existed in the target worktree.

    (a) --model with --terminal is rejected, and the message is the whole guidance.

    worker-start --task <task_id> --terminal <handle> --model <m> --json
    → --model cannot be applied when reusing terminal
    

    Fine as a rule, but it arrives as the first of a chain — you drop --model and immediately get (b).

    (b) agent_unconfigured, exactly as in step (4) of the report.

    {"code": "agent_unconfigured", "message": "Terminal <handle> is not running a recognized agent."}

    We saw this in two materially different situations, and the message is byte-identical in both:

    1. The agent process had exited. The previous worker's Claude session was gone; the tab was still listed by terminal list. Only remedy: a fresh agent terminal in that worktree.
    2. The agent was alive and sitting at its idle prompt. Claude Code was running and healthy, just not recognized as an agent by the dispatcher.

    Case 2 is recoverable without creating anything; case 1 is not. The error cannot tell them apart, so the operator has to go and read the raw screen to decide which walk they are on.

    (c) The remedy we found for case 2 is undiscoverable from the CLI.

    orca terminal send --terminal <handle> --enter --text '<prompt>'
    

    This types straight into the idle Claude session and keeps the existing dispatch live. Nothing in the agent_unconfigured text, in worker-start --help, or in the orchestration guide points at it — we found it by trial, which is the same walk this issue describes.

    Interaction with #17291

    #17291's workaround (re-run worker-start against the now-warm terminal; succeeds on the first retry) covers case 2 above — an agent that is running but not yet recognized. It does nothing for case 1, where no agent process exists at all. Since both surface as the same agent_unconfigured string, "just retry" is correct advice roughly half the time and a silent dead end the rest.

    Concrete ask, in this issue's own idiom

    Make agent_unconfigured name the state and the remedy, e.g.:

    That is one string split in two, and it removes the screen-reading step from both walks.

  3. changed the title [-][Bug]: recovering from a failed dispatch takes 5 invocations and each error names the condition but not the remedy[/-] [+][Bug]: Worker recovery errors omit the failed condition and applicable next step[/+] on Sep 5, 2026
  4. Tauri-EPO commented on Sep 7, 2026

    @Tauri-EPO
    ContributorAuthor

    Status 2026-09-07 on main e628090ad4, after #18902 merged (typed refusal codes) and #16904 moved the worker-start code.

    Partially addressed by #18902. dispatch / worker-start refusals now carry data.nextSteps; for --retry-of the step is concrete (src/shared/orchestration-dispatch-refusal-contract.ts:49-51: check dispatch-show --task and worker-show --dispatch). Point 4 (idempotent retry after runtime_unavailable) is covered by the mutation-recovery guidance in src/cli/orchestration-mutation-recovery.ts.

    Still open:

    • Point 1, naming the condition: src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-start.ts:109 still throws the single sentence Task X cannot retry from Dispatch Y. for five different preconditions, and the current attempt id is not in data.
    • Points 2 and 3: src/main/runtime/rpc/methods/orchestration/worker/explicit-worker-terminal-validation.ts:39 (terminal_worktree_mismatch) and :45 (agent_unconfigured) are still bare strings with no nextSteps, and agent_unconfigured is still decided without consulting process evidence first.

    A macOS + Claude report with the same shape is in the comment above, so this is not Windows- or Codex-specific.

  5. Tauri-EPO commented on Sep 18, 2026

    @Tauri-EPO
    ContributorAuthor

    Another occurrence of item 2 ("explain … unsupported setup/model flags in terms of the supplied selector or flag"), measured on Windows 11 (Orca 1.4.203–1.4.204) during a supervised fix round.

    worker-start --worktree id:<existing> … --setup run fails with

    Creation and setup options apply only to new-child or new-top-level worktrees.
    

    The message does not name the flag that triggered it and returns no data. The check at worker-start-validation.ts:73-78 lumps --name, --repo, --base-branch and --setup into one condition. The two federated variants at :31-36 and :108-117 do the same. The caller has to work out which flag to remove. Here the fix was to drop --setup, and the retry started normally.

    The plumbing for a better message already exists. OrchestrationError takes a third data argument (orchestration-error.ts:5), and the prompt-budget error in the same verb already returns nextSteps (worker-start-prompt-budget.ts:17). Naming the offending flags and adding a nextSteps entry such as "Omit --setup when reusing an existing worktree" would cover this case.

    For contrast, the CLI's unknown-flag error is already in good shape: worker-read --run … returns validFlags, suggestions and nextSteps: ["Did you mean: --json", "Valid flags: …"] (src/cli/args.ts:273-279).

  6. added theissue type on Sep 19, 2026
  7. karoliang commented on Oct 14, 2026

    @karoliang

    A concrete macOS / installed Orca 1.4.224 case where the returned next step does not identify the failed condition or a supported recovery path:

    1. The dispatched worker emitted valid worker_done --outcome succeeded. Its latest Dispatch is now completed, and its worker resource is settled as success.
    2. Independent coordinator source review then rejected the output and recorded the original Task as failed, with substantive correction evidence. The worker's success report was completion, not acceptance.
    3. The same active coordinator requested a retry of that original failed Task, naming its latest settled Dispatch, a verified ready terminal in the existing worktree, and no unmet dependencies.
    4. worker-start --task <original_task> --retry-of <latest_dispatch> --worktree <existing_worktree> --terminal <ready_handle> --json returned task_not_startable.

    Observed error data says status: failed, unmetDependencies: [], and retryOf names the current latest Dispatch. Its next step says that --retry-of must name the latest settled Dispatch of a failed or blocked Task, which those inspections already confirm. Re-reading dispatch-show and worker-show establishes latest completed Dispatch / succeeded worker, but does not explain that the guard rejects this settlement kind or give a supported way to register the failed independent review.

    This is distinct from a worker that natively settled failed: another original Task with native failed settlement retried successfully on the same installed runtime. No auth, quota, model or terminal mismatch is inferred from the rejected case.

    Please make the failed eligibility condition explicit and identify the supported same-Task review-failure recovery path, if one exists. If this state needs an API/code change, preserve the original failed Task, successful Dispatch report and authentic independent-review history while permitting authorized correction. Recovery must still reject stale retries, active duplicate editors, wrong Task/Dispatch/terminal, unauthorized review claims and unmet dependencies. A reset, cloned Task, rewritten report or blind repeat of the unchanged denied command would not prove recovery.

    The reproducible original Task/Dispatch receipts, reviewer findings and request ID are retained locally. Nothing has been reset, and no installed app/pin has been changed. Source-level candidate repair remains unaccepted until it covers the already-failed Task case, rather than only a future completed-to-failed transition. This case currently blocks independently reviewed site-wide noindex source rework in karoliang/bob.so#315.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions