You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Machine-readable coder outcomes never reach the controller's escalation logic. The executor's result envelope nests the model's extra.outcome under extra.modelExtra.outcome and stamps the top level with a generic value, while internal/foreman/controller/workload_coder_escalation.go (isNeedsVerificationCoder, isAlreadyResolvedCoder, shouldEscalateCoder) reads only the top-level status.result.extra.outcome. Result: the no-escalation contracts for ALREADY-RESOLVED (#970) and NEEDS-VERIFICATION (#1033/#1062) have never worked end-to-end; the controller escalates exactly the verdicts those features exist to stand down on.
Two independent paths are affected:
In-process path (pkg/foreman/agent/executor_native.go, modelDecidedResult): unconditionally sets top-level outcome: "MODEL-DECIDED" for every model-emitted non-GO and nests the real outcome under modelExtra.
Job-mode path (pkg/foreman/agent/executor_coderjob.go, coderJobResultToResult): worse; hardcodes outcome: "MODEL-NO-GO" for every NO-GO and discards the model's extra entirely, so resolvedBy/unverified are lost, not just nested.
Steps to Reproduce
Run any coder task where the model emits submit_result(verdict="NO-GO", extra={"outcome": "ALREADY-RESOLVED", "resolvedBy": "<sha>"}).
Inspect AgenticTask.status.result.extra: top-level outcome is MODEL-DECIDED (in-process) or MODEL-NO-GO (Job mode); the real outcome is under modelExtra (in-process) or gone (Job mode).
shouldEscalateCoder sees a NO-GO without a recognized non-escalating outcome and dispatches an escalation cycle.
Actual: escalation fires on ALREADY-RESOLVED and NEEDS-VERIFICATION coder bails; resolvedBy/unverified are not where operators and the controller look (Job mode loses them entirely).
TestNativeExecutor_ModelEmitsNoGo locks in the broken top-level MODEL-DECIDED; TestAlreadyResolvedEnvelopeShape and the controller tests construct synthetic envelopes by hand. No test drives the real executor envelope into the controller classifiers.
Proposed Fix
In-process path: promote the terminal, non-escalating outcomes (NEEDS-VERIFICATION, ALREADY-RESOLVED) plus their paired fields (unverified, resolvedBy) to the top level in modelDecidedResult, keep MODEL-DECIDED for everything else and the modelExtra nesting for observability. This half lands with the honest-verdict slice 1 PR (part of [FEATURE] Honest-verdict harness: declare-then-verify contract for repo-agnostic coder gates #1075), with executor-level tests driving the real path.
Job-mode path (coderJobResultToResult): still open after that PR; needs the same promotion plus preservation of the model's extra instead of discarding it. This issue tracks the Job-mode half and a seam test that drives executor output through the controller classifiers.
Environment
LLMKube version: 0.9.4
Component: Foreman executor result envelope + Workload coder escalation
Additional Context
Found during adversarial review of the honest-verdict harness slice 1 (#1075); refs #970, #1033, #1062.
Bug Description
Machine-readable coder outcomes never reach the controller's escalation logic. The executor's result envelope nests the model's
extra.outcomeunderextra.modelExtra.outcomeand stamps the top level with a generic value, whileinternal/foreman/controller/workload_coder_escalation.go(isNeedsVerificationCoder,isAlreadyResolvedCoder,shouldEscalateCoder) reads only the top-levelstatus.result.extra.outcome. Result: the no-escalation contracts forALREADY-RESOLVED(#970) andNEEDS-VERIFICATION(#1033/#1062) have never worked end-to-end; the controller escalates exactly the verdicts those features exist to stand down on.Two independent paths are affected:
pkg/foreman/agent/executor_native.go,modelDecidedResult): unconditionally sets top-leveloutcome: "MODEL-DECIDED"for every model-emitted non-GO and nests the real outcome undermodelExtra.pkg/foreman/agent/executor_coderjob.go,coderJobResultToResult): worse; hardcodesoutcome: "MODEL-NO-GO"for every NO-GO and discards the model'sextraentirely, soresolvedBy/unverifiedare lost, not just nested.Steps to Reproduce
submit_result(verdict="NO-GO", extra={"outcome": "ALREADY-RESOLVED", "resolvedBy": "<sha>"}).AgenticTask.status.result.extra: top-leveloutcomeisMODEL-DECIDED(in-process) orMODEL-NO-GO(Job mode); the real outcome is undermodelExtra(in-process) or gone (Job mode).shouldEscalateCodersees a NO-GO without a recognized non-escalating outcome and dispatches an escalation cycle.Observed live: workload run-20260711-182721 task run-20260711-182721-code-631 shows top-level
outcome: MODEL-DECIDED.Actual vs Expected
resolvedBy/unverifiedare not where operators and the controller look (Job mode loses them entirely).status.result.extraand suppress escalation, per [FEATURE] Foreman: distinct ALREADY-RESOLVED coder outcome (don't escalate the honest 'already done' bail) #970 and feat(foreman): flag unverifiable facts instead of inventing them (slicer anti-confabulation) #1062.Why no test caught it
TestNativeExecutor_ModelEmitsNoGolocks in the broken top-levelMODEL-DECIDED;TestAlreadyResolvedEnvelopeShapeand the controller tests construct synthetic envelopes by hand. No test drives the real executor envelope into the controller classifiers.Proposed Fix
NEEDS-VERIFICATION,ALREADY-RESOLVED) plus their paired fields (unverified,resolvedBy) to the top level inmodelDecidedResult, keepMODEL-DECIDEDfor everything else and themodelExtranesting for observability. This half lands with the honest-verdict slice 1 PR (part of [FEATURE] Honest-verdict harness: declare-then-verify contract for repo-agnostic coder gates #1075), with executor-level tests driving the real path.coderJobResultToResult): still open after that PR; needs the same promotion plus preservation of the model'sextrainstead of discarding it. This issue tracks the Job-mode half and a seam test that drives executor output through the controller classifiers.Environment
Additional Context
Found during adversarial review of the honest-verdict harness slice 1 (#1075); refs #970, #1033, #1062.