Skip to content

[BUG] Coder outcome envelope never reaches escalation logic (ALREADY-RESOLVED/NEEDS-VERIFICATION no-escalation contracts broken end-to-end) #1077

Description

@Defilan

Bug Description

Machine-readable coder outcomes never reach the controller's escalation logic. The executor's result envelope nests the model's extra.outcome under extra.modelExtra.outcome and stamps the top level with a generic value, while internal/foreman/controller/workload_coder_escalation.go (isNeedsVerificationCoder, isAlreadyResolvedCoder, shouldEscalateCoder) reads only the top-level status.result.extra.outcome. Result: the no-escalation contracts for ALREADY-RESOLVED (#970) and NEEDS-VERIFICATION (#1033/#1062) have never worked end-to-end; the controller escalates exactly the verdicts those features exist to stand down on.

Two independent paths are affected:

  1. In-process path (pkg/foreman/agent/executor_native.go, modelDecidedResult): unconditionally sets top-level outcome: "MODEL-DECIDED" for every model-emitted non-GO and nests the real outcome under modelExtra.
  2. Job-mode path (pkg/foreman/agent/executor_coderjob.go, coderJobResultToResult): worse; hardcodes outcome: "MODEL-NO-GO" for every NO-GO and discards the model's extra entirely, so resolvedBy/unverified are lost, not just nested.

Steps to Reproduce

  1. Run any coder task where the model emits submit_result(verdict="NO-GO", extra={"outcome": "ALREADY-RESOLVED", "resolvedBy": "<sha>"}).
  2. Inspect AgenticTask.status.result.extra: top-level outcome is MODEL-DECIDED (in-process) or MODEL-NO-GO (Job mode); the real outcome is under modelExtra (in-process) or gone (Job mode).
  3. shouldEscalateCoder sees a NO-GO without a recognized non-escalating outcome and dispatches an escalation cycle.

Observed live: workload run-20260711-182721 task run-20260711-182721-code-631 shows top-level outcome: MODEL-DECIDED.

Actual vs Expected

Why no test caught it

TestNativeExecutor_ModelEmitsNoGo locks in the broken top-level MODEL-DECIDED; TestAlreadyResolvedEnvelopeShape and the controller tests construct synthetic envelopes by hand. No test drives the real executor envelope into the controller classifiers.

Proposed Fix

  • In-process path: promote the terminal, non-escalating outcomes (NEEDS-VERIFICATION, ALREADY-RESOLVED) plus their paired fields (unverified, resolvedBy) to the top level in modelDecidedResult, keep MODEL-DECIDED for everything else and the modelExtra nesting for observability. This half lands with the honest-verdict slice 1 PR (part of [FEATURE] Honest-verdict harness: declare-then-verify contract for repo-agnostic coder gates #1075), with executor-level tests driving the real path.
  • Job-mode path (coderJobResultToResult): still open after that PR; needs the same promotion plus preservation of the model's extra instead of discarding it. This issue tracks the Job-mode half and a seam test that drives executor output through the controller classifiers.

Environment

  • LLMKube version: 0.9.4
  • Component: Foreman executor result envelope + Workload coder escalation

Additional Context

Found during adversarial review of the honest-verdict harness slice 1 (#1075); refs #970, #1033, #1062.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/foremanForeman: the agentic fleet orchestrator add-onbugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions