Skip to content

fix(codex): survive OpenAI's strict validator on --output-schema#109

Merged
AbirAbbas merged 1 commit into
mainfrom
fix/106-codex-strict-schema
Jul 22, 2026
Merged

fix(codex): survive OpenAI's strict validator on --output-schema#109
AbirAbbas merged 1 commit into
mainfrom
fix/106-codex-strict-schema

Conversation

@AbirAbbas

Copy link
Copy Markdown
Collaborator

Fixes #106

Summary

The server behind codex exec now validates --output-schema against OpenAI strict-mode rules and 400s non-strict schemas with invalid_json_schema — killing every schema-enforced codex role (PM, git init, …) in seconds, on both nodes.

Contrary to the issue's follow-up comment, this is not a blanket rejection of the flag: live probing on the same codex-cli 0.144.1 + ChatGPT-account auth shows a fully strict schema is ACCEPTED (the trivial repro schema was missing additionalProperties: false, which the validator now demands on every object node). The real gaps were in our strict rewrites:

  • free-form maps (dict[str, Any]{"type": "object"} with no properties) never get additionalProperties: false, and strict mode cannot express them at all without forcing {} output
  • Any-typed fields emit bare {} nodes — rejected with "schema must have a 'type' key" (and on the Go node, boolean true subschemas — the generic "Please ensure it is a valid JSON Schema" 400 from the issue report)
  • typed maps (additionalProperties: {schema}) are rejected outright

Changes Made (Python node)

Two layers in swe_af/runtime/codex_harness_patch.py:

  • Deterministic gate_codex_schema_strict_expressible checks the written strict schema against the live-probed validator rules before adding --output-schema. Inexpressible schemas run with --output-last-message only: codex still persists its final JSON and the harness runner's existing local validation enforces the schema (exactly the "prompt-embedded JSON + local validation" fallback proposed in the issue).
  • Reactive fallback — if the server refuses a schema anyway (its rules can tighten upstream at any time), the run is retried once without --output-schema, keeping --output-last-message.

Go node

The Go node's equivalent lives in the agentfield Go SDK's codex provider/runner — fixed by Agent-Field/agentfield#818 (same gate + fallback design, live-verified: the strict rewrites of the real GitInitResult/Architecture schemas are ACCEPTED by the validator, PRD is correctly gated). Once that merges, an SDK re-pin PR here (go.mod pseudo-version + AGENTFIELD_SDK_REF in ci.yml / go/Dockerfile) completes #106 for the Go node.

Testing

  • 6 new tests: expressibility accept/reject table (free-form map, typed map, bare Any, boolean subschema, poisoned $defs, object in a type list), rejection→rerun flow, inexpressible-schema flag skip, no-rerun on unrelated failure.
  • Full suite: 1135 passed, 1 skipped (pre-existing); compileall clean.
  • Validator rules established by live probes against codex exec --output-schema (10 schema shapes), not guesswork.

🤖 Generated with Claude Code

The server behind codex exec now validates --output-schema against OpenAI
strict-mode rules (probed live on codex-cli 0.144.1): every object node
needs additionalProperties:false plus a full required array, every node
needs a type (or $ref / anyOf), and free-form maps (dict[str, Any]),
typed maps, bare Any nodes, and boolean subschemas are rejected with
invalid_json_schema — which killed every schema-enforced codex role (PM,
git init, ...) in seconds.

Two-layer fix in the codex harness patch:

- Deterministic gate: before adding --output-schema, check the written
  strict schema against the probed validator rules
  (_codex_schema_strict_expressible). Inexpressible schemas run with
  --output-last-message only — codex still persists the final JSON and
  the harness runner's local validation enforces the schema.
- Reactive fallback: if the server refuses a schema anyway (its rules
  can tighten upstream at any time), rerun once without --output-schema
  and take the retry's output.

The Go node needs the paired agentfield Go SDK fix (same design in the
SDK's codex provider/runner) plus an SDK re-pin to pick it up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AbirAbbas
AbirAbbas merged commit e89c3d9 into main Jul 22, 2026
3 checks passed
@AbirAbbas
AbirAbbas deleted the fix/106-codex-strict-schema branch July 22, 2026 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(go): codex runtime — role output schemas rejected by OpenAI (invalid_json_schema for codex_output_schema)

1 participant