Skip to content

fix(harness): stage all six harnesses on messages.yml via shared resolve/install scripts - #789

Merged
aaronjmars merged 1 commit into
mainfrom
harness/shared-resolve-install
Jul 27, 2026
Merged

fix(harness): stage all six harnesses on messages.yml via shared resolve/install scripts#789
aaronjmars merged 1 commit into
mainfrom
harness/shared-resolve-install

Conversation

@aaronjmars

Copy link
Copy Markdown
Collaborator

Closes the last surface where "aeon supports six harnesses" wasn't true.

messages.yml staged only the claude and grok CLIs, so a repo configured for codex/pi/vibe/kimi had every inbound message answered on claude. It warned rather than swapping silently (#784), but it was still not the harness the operator chose — and on Anthropic credentials a single-provider fork may not even have.

Why it wasn't just a missing case

Both the harness/provider/model resolution (~100 lines) and the install recipes (~150 lines) lived inside aeon.yml steps, so no other workflow could reach them. messages.yml carried a second, weaker copy that knew claude and grok. Same shape as the two-path grok trap: a capability that exists on one surface and silently doesn't on another.

Extracted to two scripts, both called by both workflows:

  • scripts/resolve-harness.sh — decides HARNESS / AUTH_MODE / HARNESS_MODEL / MODEL_ARG, prints them as KEY=VALUE lines (summary to stderr). Takes an optional skill name: pass it for per-skill harness:/model: overrides, omit it for surfaces that aren't a per-skill run. That's what lets one script serve both.
  • scripts/install-harness.sh — stages that harness's CLI + provider auth. claude is a no-op so callers can invoke it unconditionally.

Each workflow still declares its own env: block (secrets.* only resolves per workflow), but the logic — the half that drifted — is shared.

Two latent bugs the extraction made reachable

A missing provider credential died mid-heredoc. Under set -u, $OPENROUTER_API_KEY unbound killed the script after cat > had created the file — leaving the harness with a 0-byte config.toml and an error naming a shell variable instead of the secret. It never fired inside a workflow, where the env: block always binds the name; it appears the moment the script is callable standalone, which is exactly what this change does. Now fails closed and names the missing secret.

The Anthropic gateway was gated on != grok. Correct while claude and grok were the only two options; with the other four reachable it would point their env at a gateway they never call. Now = claude, matching aeon.yml.

Equivalence

Refactors of live workflow logic are where silent behaviour changes hide, so I proved it rather than reasoned about it — replaying main's inline blocks against the new scripts:

  • 832 resolution cases (harness × model × skill × auth-secret × dispatch-override) — byte-identical KEY=VALUE output.
  • Every config-generating install case — byte-identical $HOME trees.

Tests

This logic had never been tested: inside a workflow step the only way to exercise it was to dispatch a live run.

  • test_resolve_harness.sh (28) — precedence, AUTH_MODE detection, per-harness MODEL_ARG shape, the stdout contract.
  • test_install_harness.sh (21) — stubs npm/pipx, asserts the generated codex/kimi/vibe config and the supply-chain pins.

Two fixture traps worth recording, both of which would have produced tests that passed while testing nothing:

  • aeon.yml's skills map uses inline flow-mappings on one line ( skill: { enabled: true, harness: "vibe" }). A block-style fixture is valid YAML but the per-skill greps match nothing, so every override case would silently test the fallback path.
  • The odd-one fixture pins vibe, which correctly forwards no --model — so the per-skill model is observable as HARNESS_MODEL, not MODEL_ARG. My first assertion had it backwards and failed against correct code.

Verified live

Dispatched a real inbound message on a live instance configured for codex (run 30298390190):

Resolve harness      Harness: codex | auth: openrouter | model: openai/gpt-5-mini
Install harness CLI  codex: OpenRouter
Install harness CLI  installed harness CLI: codex  (auth: openrouter)
Run                  Using harness: codex | auth: openrouter | model: openai/gpt-5-mini

The agent's own reply: "Running under the Aeon / Codex CLI harness." Job green, and zero has no CLI staged warnings — before this change that same dispatch was answered on claude.

actionlint, shellcheck, okf-validate clean; all shell suites pass.

messages.yml staged only the claude and grok CLIs, so a repo configured for
codex/pi/vibe/kimi had every inbound message answered on claude. It warned
rather than swapping silently, but it was still not the harness the operator
chose - and on Anthropic credentials a single-provider fork may not even have.

The cause was structural, not a missing case. Both the harness/provider/model
resolution (~100 lines) and the install recipes (~150 lines) lived inside
aeon.yml steps, so no other workflow could reach them. messages.yml carried a
second, weaker copy that knew claude and grok - the same shape as the two-path
grok trap in #784.

Extracted:

  * scripts/resolve-harness.sh - decides HARNESS / AUTH_MODE / HARNESS_MODEL /
    MODEL_ARG, prints them as KEY=VALUE lines on stdout (summary on stderr).
    Takes an OPTIONAL skill name: pass it for per-skill harness:/model:
    overrides, omit it for surfaces that are not a per-skill run.
  * scripts/install-harness.sh - stages the CLI + provider auth for that
    harness. claude is a no-op so callers can invoke it unconditionally.

Both workflows now call both scripts. Each keeps its own env: block, since
secrets.* only resolves inside a workflow, but the logic - the half that
drifted - is shared.

Two latent bugs became reachable during the extraction and are fixed:

  * A missing provider credential died MID-HEREDOC under set -u. bash had
    already created the file, so the harness was left with a 0-byte config.toml
    and an error naming a shell variable instead of the secret. It never fired
    inside a workflow, where the env: block always binds the name; it appears
    the moment the script is callable standalone. Now fails closed and names
    the missing secret.
  * messages.yml gated the Anthropic gateway on `!= grok`, correct only while
    claude and grok were the sole options. With the other four reachable it
    would point their env at a gateway they never call. Now `= claude`.

Tests - this logic had never been tested, because inside a workflow step the
only way to exercise it was to dispatch a live run:

  * scripts/tests/test_resolve_harness.sh (28 cases) - precedence, AUTH_MODE
    detection, per-harness MODEL_ARG shape, the stdout contract.
  * scripts/tests/test_install_harness.sh (21 cases) - stubs npm/pipx and
    asserts the generated codex/kimi/vibe config plus the supply-chain pins.

Equivalence verified by differential test against the inline versions:
832 resolution cases and every config-generating install case produce
byte-identical output.

actionlint clean, shellcheck clean, okf-validate clean.
@aaronjmars
aaronjmars merged commit e1630bd into main Jul 27, 2026
2 checks passed
@aaronjmars
aaronjmars deleted the harness/shared-resolve-install branch July 27, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant