Classify judge API failures for codex / pi / hermes, one recorded fixture per backend

[edonadei]

Follow-up to #75 / PR #76, which landed harness-boundary classification but wired it up for claude-code only. #76 deferred the other backends on purpose: "Codex/pi/hermes classification is intentionally out of scope until real CLI recordings exist (per issue design)." This issue is that remaining work.

Current state

PromptFailureKind and format_judge_failure (caliper/harness/prompt_failure.py) are backend-agnostic already. What varies is who populates PromptResult.failure:

• claude-code — _classify_claude_prompt_failure (caliper/harness/claude_code.py:453) maps api_error_status 404/401/429 onto the enum. Works.
• pi, hermes — no _prompt_output override, so they inherit CliHarness._prompt_output (caliper/harness/base.py:387), which sets PromptFailureKind.OTHER on any nonzero exit. format_judge_failure returns an OTHER message verbatim, so a retired pi/hermes judge model still produces an opaque error with no --judge-model hint.
• codex — _prompt_output (caliper/harness/codex.py:306) overrides the base and sets error but never sets failure at all. So codex is a step behind pi/hermes: it doesn't even reach the OTHER path. Classify judge model failures at the harness boundary, not in the judge #75's design said to map codex from _extract_codex_error (caliper/harness/codex.py:514), which already scrapes ERROR: lines and JSON error.message payloads; that mapping wasn't written.

The codex divergence is worth fixing regardless of whether real 404 fixtures ever get recorded — a judge failure on codex should at minimum carry OTHER, matching the base contract.

Record before matching

#72 was closed unmerged because its fixtures were fabricated. Same rule applies here: do not invent payloads. Record each backend's real output against a retired or nonexistent model, the way the claude envelope in tests/test_prompt_failure.py was recorded, and paste the raw returncode / stdout / stderr into this issue before writing any matching logic.

Rough recording commands, mirroring what each harness actually spawns (transcribed from each _prompt_command; adjust if the installed CLI disagrees):

codex exec --skip-git-repo-check --dangerously-bypass-approvals-and-sandbox \
  --color never --model <retired-model> - <<< "say ok"

pi --print --mode json --no-session --approve --model <retired-model> "say ok"

hermes -z "say ok" --model <retired-model> --ignore-rules

A backend with no recording gets no classification beyond OTHER. Partial delivery is fine and expected — one backend per recording.

Acceptance

• codex._prompt_output sets PromptResult.failure on every nonzero-exit path, at minimum OTHER, so it stops diverging from CliHarness._prompt_output.
• Any backend mapped to MODEL_UNAVAILABLE / AUTH / RATE_LIMITED has a real recorded fixture pasted in this issue and driven by a test in tests/test_prompt_failure.py, in the style of RETIRED_MODEL_ENVELOPE.
• A test asserts the judge surfaces the --judge-model hint for each newly classified backend (mirroring test_eval_judge_surfaces_classified_model_unavailable).
• No substring matching in caliper/judge/ — classification stays in the harness.
• If the user-visible message shape changes, update README Troubleshooting and both skill REFERENCEs per CLAUDE.md.

Non-goals

• Appending the --judge-model hint to every OTHER failure. That was the "generic hint" option weighed in Judge should map a "model unavailable" failure to a clear "pass --judge-model" error #71 and passed over; an unknown failure shouldn't suggest a model swap as the fix.
• Widening the claude-code gate to unclassified is_error: true envelopes. That partial gate is pinned by a test and needs its own recordings (max-turns, refusals) to move.