fix(codex): survive OpenAI's strict validator on --output-schema#109
Merged
Conversation
The server behind codex exec now validates --output-schema against OpenAI strict-mode rules (probed live on codex-cli 0.144.1): every object node needs additionalProperties:false plus a full required array, every node needs a type (or $ref / anyOf), and free-form maps (dict[str, Any]), typed maps, bare Any nodes, and boolean subschemas are rejected with invalid_json_schema — which killed every schema-enforced codex role (PM, git init, ...) in seconds. Two-layer fix in the codex harness patch: - Deterministic gate: before adding --output-schema, check the written strict schema against the probed validator rules (_codex_schema_strict_expressible). Inexpressible schemas run with --output-last-message only — codex still persists the final JSON and the harness runner's local validation enforces the schema. - Reactive fallback: if the server refuses a schema anyway (its rules can tighten upstream at any time), rerun once without --output-schema and take the retry's output. The Go node needs the paired agentfield Go SDK fix (same design in the SDK's codex provider/runner) plus an SDK re-pin to pick it up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #106
Summary
The server behind
codex execnow validates--output-schemaagainst OpenAI strict-mode rules and 400s non-strict schemas withinvalid_json_schema— killing every schema-enforced codex role (PM, git init, …) in seconds, on both nodes.Contrary to the issue's follow-up comment, this is not a blanket rejection of the flag: live probing on the same codex-cli 0.144.1 + ChatGPT-account auth shows a fully strict schema is ACCEPTED (the trivial repro schema was missing
additionalProperties: false, which the validator now demands on every object node). The real gaps were in our strict rewrites:dict[str, Any]→{"type": "object"}with noproperties) never getadditionalProperties: false, and strict mode cannot express them at all without forcing{}outputAny-typed fields emit bare{}nodes — rejected with "schema must have a 'type' key" (and on the Go node, booleantruesubschemas — the generic "Please ensure it is a valid JSON Schema" 400 from the issue report)additionalProperties: {schema}) are rejected outrightChanges Made (Python node)
Two layers in
swe_af/runtime/codex_harness_patch.py:_codex_schema_strict_expressiblechecks the written strict schema against the live-probed validator rules before adding--output-schema. Inexpressible schemas run with--output-last-messageonly: codex still persists its final JSON and the harness runner's existing local validation enforces the schema (exactly the "prompt-embedded JSON + local validation" fallback proposed in the issue).--output-schema, keeping--output-last-message.Go node
The Go node's equivalent lives in the agentfield Go SDK's codex provider/runner — fixed by Agent-Field/agentfield#818 (same gate + fallback design, live-verified: the strict rewrites of the real
GitInitResult/Architectureschemas are ACCEPTED by the validator,PRDis correctly gated). Once that merges, an SDK re-pin PR here (go.modpseudo-version +AGENTFIELD_SDK_REFin ci.yml / go/Dockerfile) completes #106 for the Go node.Testing
Any, boolean subschema, poisoned$defs,objectin a type list), rejection→rerun flow, inexpressible-schema flag skip, no-rerun on unrelated failure.compileallclean.codex exec --output-schema(10 schema shapes), not guesswork.🤖 Generated with Claude Code