Add structured output support for Ollama provider - #72
Conversation
Add a Format field to AgentConfig (TOML: format = "json" or a JSON Schema table), provider.Request, and ollamaRequest. When set, Format is passed as-is to Ollama's format parameter, enabling both JSON mode and schema-constrained structured output. Co-Authored-By: Regular <noreply@anthropic.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdded a Request-level structured output Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
cmd/run.go (1)
395-402:⚠️ Potential issue | 🟠 MajorFail fast if
formatis set on a provider that doesn’t support it.Line 401 forwards
cfg.Formatunconditionally. For non-Ollama providers this currently becomes a silent no-op, which is a sneaky UX footgun. Add an explicit guard and return a config error (exit code 2) with a fix message.💡 Suggested fix
diff --git a/cmd/run.go b/cmd/run.go @@ // Step 4-5: Parse model and validate provider provName, modelName, err := parseModel(cfg.Model) if err != nil { return &ExitError{Code: 1, Err: err} } + if cfg.Format != nil && provName != "ollama" { + return &ExitError{ + Code: 2, + Err: fmt.Errorf("format is only supported with provider %q; remove format or switch model provider", "ollama"), + } + }As per coding guidelines:
cmd/**/*.go: “Config validation errors must exit with code 2”.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cmd/run.go` around lines 395 - 402, When building the provider.Request you currently forward cfg.Format unconditionally; instead, detect when cfg.Format is set but the chosen provider implementation does not support formatting (i.e., not Ollama) and fail fast: check the active provider type/name (the code that selects the provider) and if cfg.Format != "" and provider does not support format, print a clear config error and exit with code 2. Update the validation before creating provider.Request so the guard uses cfg.Format and the provider identity/capability and returns a config error (exit code 2) with a suggested fix message rather than silently ignoring the field.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@internal/agent/agent.go`:
- Around line 94-97: The Format field is currently interface{} and isn't
validated early; update the agent config validation (the Validate method for the
agent config / AgentConfig.Validate) to check the Format value at load time:
allow either the string "json" or a JSON Schema table (e.g.,
map[string]interface{}), and return a clear validation error otherwise (message:
`format must be "json" or a JSON Schema table`). Implement a type switch on the
Format field (case string: verify == "json"; case map[string]interface{}:
accept; default: return error) so invalid types/typos fail fast with an
actionable error that callers (cmd/*.go) can exit with code 2. Ensure the error
originates from AgentConfig.Validate and includes that exact message.
In `@internal/provider/ollama.go`:
- Around line 236-238: The SendStream implementation is missing propagation of
the output format, causing structured output to be lost in streaming mode;
update the SendStream signature to accept the same format parameter (or access
req.Format) and set body.Format in SendStream the same way Send does (i.e., if
req.Format != nil { body.Format = req.Format }), then ensure the request uses
that body so streaming responses honor the format; locate SendStream and Send in
the ollama.go file and mirror how Send assigns body.Format to apply the fix.
---
Outside diff comments:
In `@cmd/run.go`:
- Around line 395-402: When building the provider.Request you currently forward
cfg.Format unconditionally; instead, detect when cfg.Format is set but the
chosen provider implementation does not support formatting (i.e., not Ollama)
and fail fast: check the active provider type/name (the code that selects the
provider) and if cfg.Format != "" and provider does not support format, print a
clear config error and exit with code 2. Update the validation before creating
provider.Request so the guard uses cfg.Format and the provider
identity/capability and returns a config error (exit code 2) with a suggested
fix message rather than silently ignoring the field.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7f67bd98-47d3-4004-9294-c76448556f2e
📒 Files selected for processing (4)
cmd/run.gointernal/agent/agent.gointernal/provider/ollama.gointernal/provider/provider.go
This commit addresses the actionable comments from the recent PR review: - Fail fast with exit code 2 if format is specified for a provider other than Ollama. - Validate Format in AgentConfig to ensure it is either "json" or a valid JSON Schema table. - Fix structured output support in Ollama.SendStream by propagating the Format field to the request body. - Add comprehensive test coverage for all new validation rules and streaming behavior.
This commit applies code style refactorings based on recently accepted PRs: - Refactored provider and format validation tests into a single table-driven TestRun_ProviderConfigValidation. - Added conditional display of Format to 'agents show' command, properly omitting it when not set. - Added Format to the agent TOML scaffold template and verified its presence via TestScaffold_IncludesFormat.
This commit updates the README.md, docs/src/configuration/agent-config.md, and docs/design/agent-config-schema.md to show how to configure the structured output format.
There was a problem hiding this comment.
🧹 Nitpick comments (1)
cmd/agents_test.go (1)
171-171: Consider one table-driven case for the JSON Schema branch too.You now cover
format = "json"nicely, but the non-string display path (Format: JSON Schema) incmd/agents.gois still untested. A tiny table-driven matrix (nil,"json", schema table) would close that loop.As per coding guidelines:
**/*_test.go: "Use table-driven tests with[]struct{name, input, want}pattern".Also applies to: 206-207
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cmd/agents_test.go` at line 171, Add a table-driven test in cmd/agents_test.go that covers the three display-format branches (nil/default, "json" string, and a non-string JSON Schema table) so the `Format: JSON Schema` path in cmd/agents.go is exercised; update the existing test cases to use the []struct{name, input, want} pattern and add a case where the format value is a schema object (not a string) to verify the non-string rendering branch, along with the existing `"json"` case and a nil/default case.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@cmd/agents_test.go`:
- Line 171: Add a table-driven test in cmd/agents_test.go that covers the three
display-format branches (nil/default, "json" string, and a non-string JSON
Schema table) so the `Format: JSON Schema` path in cmd/agents.go is exercised;
update the existing test cases to use the []struct{name, input, want} pattern
and add a case where the format value is a schema object (not a string) to
verify the non-string rendering branch, along with the existing `"json"` case
and a nil/default case.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: fe44536d-eda5-4371-939f-665c98bc49f4
📒 Files selected for processing (5)
cmd/agents.gocmd/agents_test.gocmd/run_test.gointernal/agent/agent.gointernal/agent/agent_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
- internal/agent/agent_test.go
- cmd/run_test.go
This commit refactors the structured output implementation to use an adapter model, enabling JSON mode and JSON Schema support across Ollama, OpenAI, Anthropic, and Gemini: - Defined a unified ResponseFormat internal representation and added capability checking to the Provider interface. - Implemented OpenAI adapter supporting JSON mode and Structured Outputs (json_schema). - Implemented Anthropic adapter supporting JSON mode via prompt injection and JSON Schema via forced tool calls. - Implemented Gemini adapter supporting JSON mode and response_schema. - Refactored CLI validation to be provider-agnostic, using the new SupportsFormat method. - Updated documentation and scaffolds to reflect multi-provider support. - Added and updated tests to verify behavior across different provider implementations.
9b6a80e to
1dcd386
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@internal/provider/anthropic.go`:
- Around line 247-263: The synthetic anthropic tool "print_output" (added via
anthropicToolDef with Name "print_output" and forced via body.ToolChoice)
prevents real tool invocation and causes streamed events to be emitted as
StreamEventTool* rather than JSON text, breaking consumers expecting Send;
update the handler to either reject requests that combine
req.Format.Type==FormatSchema with any real tool usage or special-case
"print_output" in the streaming/response path: if using the synthetic
print_output, translate StreamEventTool* events back into the expected JSON/text
stream (or emit a StreamEventText) and ensure body.ToolChoice is not forced when
real tools are present, or else validate and return an error early. Reference:
req.Format, anthropicToolDef{Name:"print_output"}, body.ToolChoice,
StreamEventTool*, and the stream parser/Send path.
In `@internal/provider/gemini.go`:
- Around line 105-109: Rename the geminiGenerationConfig struct field
ResponseSchema to ResponseJSONSchema and change its JSON tag from
"response_schema,omitempty" to "response_json_schema,omitempty"; then update all
places that construct/assign this field (the two assignment sites that set the
schema on geminiGenerationConfig) to use ResponseJSONSchema so the Gemini
structured-output API receives the correct "response_json_schema" field; also
update any references/serialization that rely on the old name (e.g., any use in
SupportsFormat logic or marshaling) to the new ResponseJSONSchema identifier.
In `@internal/provider/opencode.go`:
- Around line 313-333: The current code appends a synthetic ocAnthropicToolDef
named "print_output" and then forces selection by setting body.ToolChoice =
&ocAnthropicToolChoice{Type: "tool", Name: "print_output"}, which prevents real
tools from being chosen and causes structured outputs to surface as tool-stream
events; instead stop forcing the tool choice: keep adding the ocAnthropicToolDef
("print_output") and add a clear system/instruction hint to prefer using that
tool for structured output but remove or avoid setting body.ToolChoice to
"print_output" so real tools remain selectable; make the same change in the
other similar block (the later block around the 461-481 region) so both schema
and JSON structured-output flows don't suppress real tools.
- Around line 230-265: The current helpers convertToOCAnthropicJSONSchemaProps
and convertToOCAnthropicRequiredProps strip almost all JSON Schema constraints;
instead preserve and forward the original property schemas and required list.
Modify ocJSONSchemaProp (or replace it) to hold the raw property representation
(e.g., map[string]interface{} or json.RawMessage) so
convertToOCAnthropicJSONSchemaProps returns the original property objects
unchanged (not just "type" and "description"); ensure
convertToOCAnthropicRequiredProps builds a []string containing only valid string
entries from schema["required"] (no nils) and return nil only when there is no
required field. Update any callers that expect the old lightweight
ocJSONSchemaProp shape to consume the preserved/raw schema representation.
- Around line 606-614: The request structs (e.g., ocGPTStreamRequest and
ocGPTRequest) are setting a top-level response_format field which the
/v1/responses endpoint ignores; change the payload to use the text.format
mechanism instead of ResponseFormat/response_format: remove or stop populating
the response_format field and instead embed the structured output spec under the
model input using the text.format envelope (follow OpenAI Responses API
text.format schema) so JSON mode and schema enforcement are honored; update any
place that marshals ResponseFormat into requests to use the text.format payload
format.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e1bf79c0-ae01-46d7-9b60-06295073a7a6
⛔ Files ignored due to path filters (2)
docs/design/agent-config-schema.mdis excluded by!docs/**docs/src/configuration/agent-config.mdis excluded by!docs/**
📒 Files selected for processing (15)
README.mdcmd/run.gocmd/run_test.gointernal/agent/agent.gointernal/provider/anthropic.gointernal/provider/bedrock.gointernal/provider/gemini.gointernal/provider/ollama.gointernal/provider/ollama_test.gointernal/provider/openai.gointernal/provider/opencode.gointernal/provider/provider.gointernal/provider/provider_test.gointernal/provider/retry.gointernal/provider/retry_test.go
✅ Files skipped from review due to trivial changes (1)
- README.md
🚧 Files skipped from review as they are similar to previous changes (6)
- internal/provider/ollama_test.go
- internal/provider/ollama.go
- internal/agent/agent.go
- internal/provider/provider.go
- cmd/run.go
- cmd/run_test.go
| type geminiGenerationConfig struct { | ||
| Temperature *float64 `json:"temperature,omitempty"` | ||
| MaxOutputTokens *int `json:"maxOutputTokens,omitempty"` | ||
| Temperature *float64 `json:"temperature,omitempty"` | ||
| MaxOutputTokens *int `json:"maxOutputTokens,omitempty"` | ||
| ResponseMimeType string `json:"response_mime_type,omitempty"` | ||
| ResponseSchema map[string]interface{} `json:"response_schema,omitempty"` |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, find and examine the gemini.go file
find . -name "gemini.go" -type f | head -5Repository: jrswab/axe
Length of output: 83
🏁 Script executed:
# Get file structure and check the mentioned lines
wc -l internal/provider/gemini.goRepository: jrswab/axe
Length of output: 85
🏁 Script executed:
# Read the struct definition around lines 105-109
sed -n '100,115p' internal/provider/gemini.goRepository: jrswab/axe
Length of output: 708
🏁 Script executed:
# Check the Send method around lines 285-293
sed -n '280,300p' internal/provider/gemini.goRepository: jrswab/axe
Length of output: 516
🏁 Script executed:
# Check the SendStream method around lines 429-437
sed -n '425,445p' internal/provider/gemini.goRepository: jrswab/axe
Length of output: 511
🌐 Web query:
Gemini API structured output response_json_schema field documentation
💡 Result:
The response_json_schema field in the Gemini API is used for structured outputs to constrain model responses to a specific JSON schema, ensuring predictable, type-safe JSON output. It is specified in the generationConfig (or config in SDKs) alongside "response_mime_type": "application/json". Key details: - Location: Within generationConfig object of the generateContent request. - Purpose: Defines a valid JSON Schema (full support added in Gemini 2.5+ models) that the model must adhere to. Supports libraries like Pydantic (Python) and Zod (JS). - Usage example (REST/Python): { "contents": [...], "generationConfig": { "responseMimeType": "application/json", "responseJsonSchema": { "type": "object", "properties": { "recipe_name": {"type": "string"}, "ingredients": {"type": "array", "items": {"type": "object"}} }, "required": ["recipe_name", "ingredients"] } } } - Supported types: string, number, integer, boolean, object, array, null (as union). - Additional features: Implicit property ordering (matches schema key order), keywords like anyOf, $ref, min/max constraints. - Model support: Gemini 2.5+ and 3 series (e.g., gemini-2.5-flash, gemini-3-flash-preview). Preview for some 3.x models. - Differences: Newer than response_schema (OpenAPI subset); response_json_schema enables full JSON Schema. Vertex AI has both fields. - Limitations: Subset of JSON Schema; complex schemas may error. Note: Older docs reference response_schema (OpenAPI-based). Use response_json_schema for latest full JSON Schema support in Gemini API.
Citations:
- 1: https://ai.google.dev/gemini-api/docs/structured-output
- 2: Support
response_json_schemaas amethodforwith_structured_outputlangchain-ai/langchain-google#1078 - 3: https://blog.google/innovation-and-ai/technology/developers-tools/gemini-api-structured-outputs/
- 4: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/reference/rest/v1beta1/GenerationConfig
Rename ResponseSchema field to ResponseJSONSchema and use the correct Gemini API field name.
Gemini's structured-output API uses response_json_schema (not response_schema) for JSON Schema mode. The current code advertises schema support via SupportsFormat but serializes under the wrong field, so requests won't work as intended. Update the struct field, JSON tag, and both assignment locations to use the correct field name. (ai.google.dev)
🛠️ Minimal fix
type geminiGenerationConfig struct {
Temperature *float64 `json:"temperature,omitempty"`
MaxOutputTokens *int `json:"maxOutputTokens,omitempty"`
ResponseMimeType string `json:"response_mime_type,omitempty"`
- ResponseSchema map[string]interface{} `json:"response_schema,omitempty"`
+ ResponseJSONSchema map[string]interface{} `json:"response_json_schema,omitempty"`
}
@@
- genCfg.ResponseSchema = req.Format.Schema
+ genCfg.ResponseJSONSchema = req.Format.Schema
@@
- genCfg.ResponseSchema = req.Format.Schema
+ genCfg.ResponseJSONSchema = req.Format.SchemaAlso applies to: 285-293, 429-437
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@internal/provider/gemini.go` around lines 105 - 109, Rename the
geminiGenerationConfig struct field ResponseSchema to ResponseJSONSchema and
change its JSON tag from "response_schema,omitempty" to
"response_json_schema,omitempty"; then update all places that construct/assign
this field (the two assignment sites that set the schema on
geminiGenerationConfig) to use ResponseJSONSchema so the Gemini
structured-output API receives the correct "response_json_schema" field; also
update any references/serialization that rely on the old name (e.g., any use in
SupportsFormat logic or marshaling) to the new ResponseJSONSchema identifier.
| func convertToOCAnthropicJSONSchemaProps(schema map[string]interface{}) map[string]ocJSONSchemaProp { | ||
| propsRaw, ok := schema["properties"].(map[string]interface{}) | ||
| if !ok { | ||
| return nil | ||
| } | ||
| props := make(map[string]ocJSONSchemaProp, len(propsRaw)) | ||
| for name, val := range propsRaw { | ||
| valMap, ok := val.(map[string]interface{}) | ||
| if !ok { | ||
| continue | ||
| } | ||
| p := ocJSONSchemaProp{} | ||
| if t, ok := valMap["type"].(string); ok { | ||
| p.Type = t | ||
| } | ||
| if d, ok := valMap["description"].(string); ok { | ||
| p.Description = d | ||
| } | ||
| props[name] = p | ||
| } | ||
| return props | ||
| } | ||
|
|
||
| func convertToOCAnthropicRequiredProps(schema map[string]interface{}) []string { | ||
| reqRaw, ok := schema["required"].([]interface{}) | ||
| if !ok { | ||
| return nil | ||
| } | ||
| req := make([]string, len(reqRaw)) | ||
| for i, v := range reqRaw { | ||
| if s, ok := v.(string); ok { | ||
| req[i] = s | ||
| } | ||
| } | ||
| return req | ||
| } |
There was a problem hiding this comment.
This helper strips most of the user’s schema.
Only top-level type, description, and a best-effort required list survive this conversion. Nested objects, enum, items, additionalProperties, numeric/string constraints, and other JSON Schema rules all disappear before the schema reaches Claude, so the output is much less constrained than the config asked for. Passing the schema through unchanged would be a lot less “telephone game.”
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@internal/provider/opencode.go` around lines 230 - 265, The current helpers
convertToOCAnthropicJSONSchemaProps and convertToOCAnthropicRequiredProps strip
almost all JSON Schema constraints; instead preserve and forward the original
property schemas and required list. Modify ocJSONSchemaProp (or replace it) to
hold the raw property representation (e.g., map[string]interface{} or
json.RawMessage) so convertToOCAnthropicJSONSchemaProps returns the original
property objects unchanged (not just "type" and "description"); ensure
convertToOCAnthropicRequiredProps builds a []string containing only valid string
entries from schema["required"] (no nils) and return nil only when there is no
required field. Update any callers that expect the old lightweight
ocJSONSchemaProp shape to consume the preserved/raw schema representation.
| if req.Format != nil { | ||
| if req.Format.Type == FormatSchema { | ||
| // Add forced tool call for schema enforcement | ||
| body.Tools = append(body.Tools, ocAnthropicToolDef{ | ||
| Name: "print_output", | ||
| Description: "Outputs the response in the requested structured format.", | ||
| InputSchema: ocAnthropicInputSchema{ | ||
| Type: "object", | ||
| Properties: convertToOCAnthropicJSONSchemaProps(req.Format.Schema), | ||
| Required: convertToOCAnthropicRequiredProps(req.Format.Schema), | ||
| }, | ||
| }) | ||
| body.ToolChoice = &ocAnthropicToolChoice{Type: "tool", Name: "print_output"} | ||
| } else if req.Format.Type == FormatJSON { | ||
| // Prompt injection for JSON mode | ||
| if body.System != "" { | ||
| body.System += "\n\n" | ||
| } | ||
| body.System += "You must output your response in valid JSON. Do not include any markdown formatting, preamble, or conversational text. Output only the JSON." | ||
| } | ||
| } |
There was a problem hiding this comment.
The Claude structured-output shim also suppresses real tools.
This has the same trap as the Anthropic provider: tool_choice = print_output prevents any real tool from being selected, and on the streaming path that synthetic tool will surface as StreamEventTool* instead of JSON text. Right now “structured output” comes with a surprise side quest.
Also applies to: 461-481
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@internal/provider/opencode.go` around lines 313 - 333, The current code
appends a synthetic ocAnthropicToolDef named "print_output" and then forces
selection by setting body.ToolChoice = &ocAnthropicToolChoice{Type: "tool",
Name: "print_output"}, which prevents real tools from being chosen and causes
structured outputs to surface as tool-stream events; instead stop forcing the
tool choice: keep adding the ocAnthropicToolDef ("print_output") and add a clear
system/instruction hint to prefer using that tool for structured output but
remove or avoid setting body.ToolChoice to "print_output" so real tools remain
selectable; make the same change in the other similar block (the later block
around the 461-481 region) so both schema and JSON structured-output flows don't
suppress real tools.
| type ocGPTStreamRequest struct { | ||
| Model string `json:"model"` | ||
| Input []interface{} `json:"input"` | ||
| Temperature *float64 `json:"temperature,omitempty"` | ||
| MaxOutputTokens *int `json:"max_output_tokens,omitempty"` | ||
| Tools []ocOpenAIToolDef `json:"tools,omitempty"` | ||
| Stream bool `json:"stream"` | ||
| ResponseFormat *ocResponseFormat `json:"response_format,omitempty"` | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's look at the file structure and the specific lines mentioned
cd internal/provider && head -n 1 opencode.go && wc -l opencode.goRepository: jrswab/axe
Length of output: 87
🏁 Script executed:
# Examine the code at the specified line ranges
sed -n '606,614p' internal/provider/opencode.goRepository: jrswab/axe
Length of output: 507
🏁 Script executed:
# Check the other affected locations
echo "=== Lines 658-671 ===" && sed -n '658,671p' internal/provider/opencode.go
echo "=== Lines 987-994 ===" && sed -n '987,994p' internal/provider/opencode.go
echo "=== Lines 1128-1141 ===" && sed -n '1128,1141p' internal/provider/opencode.goRepository: jrswab/axe
Length of output: 1274
🏁 Script executed:
# Search for all uses of ResponseFormat in the file to understand the full scope
rg 'ResponseFormat|response_format' internal/provider/opencode.go -nRepository: jrswab/axe
Length of output: 919
🏁 Script executed:
# Let's look for where these structs are being used and understand the endpoint context
rg '/v1/responses' internal/provider/opencode.go -n -B 2 -A 2Repository: jrswab/axe
Length of output: 609
🏁 Script executed:
# Get context on the ocResponseFormat struct definition
sed -n '118,130p' internal/provider/opencode.goRepository: jrswab/axe
Length of output: 444
🌐 Web query:
OpenAI Responses API text.format structured outputs
💡 Result:
The OpenAI Responses API supports structured outputs using the text.format parameter to ensure model responses adhere to a specified JSON schema or produce valid JSON. ## Key Features - Structured Outputs: Guarantees adherence to a JSON schema. Enable with text: { format: { type: "json_schema", "strict": true, "schema": {...} } }. Supported on models like gpt-4o-2024-08-06, gpt-4o-mini, and later. - JSON Mode: Ensures valid JSON output (no schema adherence). Enable with text: { format: { type: "json_object" } }. Available on older models like gpt-3.5-turbo, gpt-4-*. ## Usage Structured Outputs via text.format is ideal for formatting the model's direct response to users (vs. function calling for tools). Example (Python SDK): response = client.responses.parse( model="gpt-4o-2024-08-06", input=[{"role": "user", "content": "Extract event info."}], text_format=YourSchema # Pydantic/Zod schema ) Direct API: text: { format: { type: "json_schema", strict: true, schema: { ... } } } Supports streaming, refusals (via refusal field), and a subset of JSON Schema. First request with a schema incurs extra latency for processing. Official docs: https://platform.openai.com/docs/guides/structured-outputs (Responses API mode).
Citations:
- 1: https://developers.openai.com/api/docs/guides/structured-outputs
- 2: https://developers.openai.com/docs/guides/structured-outputs
- 3: https://platform.openai.com/docs/guides/text-generation/json-mode
- 4: https://openai.com/index/introducing-structured-outputs-in-the-api/
- 5: https://platform.openai.com/docs/guides/structured-outputs?api-mode=responses
- 6: https://platform.openai.com/docs/guides/json-mode
🌐 Web query:
OpenAI Responses API request body schema json
💡 Result:
The OpenAI Responses API uses the POST /v1/responses endpoint. The request body is a JSON object with the following key parameters (all optional unless noted, based on official API reference; exact schema is complex with nested unions for inputs/tools): - model: string (e.g., "gpt-4o", "o3") - input: string | array (text prompt or list of input messages/objects like {content, role, type}) - instructions: string | array (system/developer instructions) - text: object {format: {type: "text" | "json_schema" | "json_object", ...}, verbosity?: ...} for output config, including Structured Outputs with schema - tools: array of tool objects (functions, web_search, file_search, etc., with parameters JSON schema) - reasoning: object {effort: "low" | "medium" | "high"} - conversation: string | {id: string} - background: boolean - stream: boolean - temperature: number (0-2) - include: array of strings (e.g., "web_search_call.action.sources") - context_management: array of {type: "compaction", compact_threshold: number} - metadata: object (key-value pairs) Full detailed schema and types are documented at https://platform.openai.com/docs/api-reference/responses/create (select JSON view for complete structure). Examples: Simple curl: curl https://api.openai.com/v1/responses -H "Authorization: Bearer $OPENAI_API_KEY" -d '{"model": "gpt-4o", "input": "Hello"}' With structured output: "text": {"format": {"type": "json_schema", "name": "response", "schema": {...}}}
Citations:
- 1: https://developers.openai.com/api/reference/resources/responses
- 2: https://platform.openai.com/docs/api-reference/responses/create
- 3: https://platform.openai.com/docs/api-reference/responses/create?lang=curl
- 4: https://developers.openai.com/docs/guides/text?api-mode=responses
Use text.format for the Responses API—response_format won't work here.
The code sends requests to /v1/responses, where OpenAI's API expects structured outputs via text.format, not a top-level response_format field. As written, the API will silently ignore all structured output configuration, breaking JSON mode and schema enforcement.
🛠️ Minimal fix sketch
+type ocTextConfig struct {
+ Format *ocResponseFormat `json:"format,omitempty"`
+}
+
type ocGPTStreamRequest struct {
Model string `json:"model"`
Input []interface{} `json:"input"`
Temperature *float64 `json:"temperature,omitempty"`
MaxOutputTokens *int `json:"max_output_tokens,omitempty"`
Tools []ocOpenAIToolDef `json:"tools,omitempty"`
Stream bool `json:"stream"`
- ResponseFormat *ocResponseFormat `json:"response_format,omitempty"`
+ Text *ocTextConfig `json:"text,omitempty"`
}
@@
- body.ResponseFormat = &ocResponseFormat{Type: "json_object"}
+ body.Text = &ocTextConfig{Format: &ocResponseFormat{Type: "json_object"}}
@@
- body.ResponseFormat = &ocResponseFormat{
- Type: "json_schema",
- JSONSchema: &ocJSONSchemaDef{
- Name: "structured_output",
- Strict: true,
- Schema: req.Format.Schema,
- },
- }
+ body.Text = &ocTextConfig{
+ Format: &ocResponseFormat{
+ Type: "json_schema",
+ JSONSchema: &ocJSONSchemaDef{
+ Name: "structured_output",
+ Strict: true,
+ Schema: req.Format.Schema,
+ },
+ },
+ }Apply the same change to all affected request structs: ocGPTStreamRequest, ocGPTRequest, and any others using ResponseFormat with the /v1/responses endpoint.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| type ocGPTStreamRequest struct { | |
| Model string `json:"model"` | |
| Input []interface{} `json:"input"` | |
| Temperature *float64 `json:"temperature,omitempty"` | |
| MaxOutputTokens *int `json:"max_output_tokens,omitempty"` | |
| Tools []ocOpenAIToolDef `json:"tools,omitempty"` | |
| Stream bool `json:"stream"` | |
| ResponseFormat *ocResponseFormat `json:"response_format,omitempty"` | |
| } | |
| type ocTextConfig struct { | |
| Format *ocResponseFormat `json:"format,omitempty"` | |
| } | |
| type ocGPTStreamRequest struct { | |
| Model string `json:"model"` | |
| Input []interface{} `json:"input"` | |
| Temperature *float64 `json:"temperature,omitempty"` | |
| MaxOutputTokens *int `json:"max_output_tokens,omitempty"` | |
| Tools []ocOpenAIToolDef `json:"tools,omitempty"` | |
| Stream bool `json:"stream"` | |
| Text *ocTextConfig `json:"text,omitempty"` | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@internal/provider/opencode.go` around lines 606 - 614, The request structs
(e.g., ocGPTStreamRequest and ocGPTRequest) are setting a top-level
response_format field which the /v1/responses endpoint ignores; change the
payload to use the text.format mechanism instead of
ResponseFormat/response_format: remove or stop populating the response_format
field and instead embed the structured output spec under the model input using
the text.format envelope (follow OpenAI Responses API text.format schema) so
JSON mode and schema enforcement are honored; update any place that marshals
ResponseFormat into requests to use the text.format payload format.
|
Hey @gavmor, thanks for this. Structured output from the LLM is a useful feature and I can see how it strengthens the pipeline use case. The scope of this PR is wider than I'd like to merge as-is. Would you be open to splitting this into a PR that adds only the format field to agent config and passes it through in a provider-agnostic way? The goal would be keeping the implementation simple enough that each provider doesn't need its own adapter logic for this. Let me know if you'd like a hand. Thanks again for contributing to Axe! 🙏 |
|
Great idea, happy to pare it down. |
2 participants
Summary
Adds a
formatfield toAgentConfig(TOML:format = "json"or a JSON Schema table),provider.Request, andollamaRequest. When set,formatis passed as-is to Ollama'sformatparameter, enabling both JSON mode and schema-constrained structured output.Example
Test plan
format = "json"in an agent TOML and verify Ollama returns valid JSONformatto a JSON Schema table and verify output is schema-constrainedformatbehave identically to before🤖 Generated with Claude Code
Summary
This PR introduces provider-agnostic structured output support across the platform. It adds a
formatfield to agent configuration that accepts either"json"for JSON mode or a JSON Schema table for schema-constrained output. A unifiedResponseFormattype was implemented in the provider interface, with each provider implementing aSupportsFormatcapability check. Format parameters are now validated at the CLI level, with early exit if the selected provider doesn't support the requested format. Provider-specific implementations handle format translation: OpenAI usesresponse_formatwith JSON object and JSON Schema modes; Anthropic uses prompt injection for JSON mode and forced tool calls for schemas; Gemini usesResponseMimeTypeandResponseSchemain generation config; Ollama passes the format field directly. Bedrock provider does not support structured output.Changelog
Added
Formatfield toAgentConfigsupporting"json"string or JSON Schemamap[string]interface{}FormatTypeenum (FormatNone,FormatJSON,FormatSchema) andResponseFormatstruct to provider interfaceSupportsFormat(format *ResponseFormat) boolmethod toProviderinterfaceTestRun_ProviderConfigValidationconsolidated test covering multiple configuration validation scenariosTestScaffold_IncludesFormatandTestValidate_Formatto verify format field handling in agent configurationagents showcommand outputChanged
Requesttype now includes optionalFormat *ResponseFormatfieldformatfieldresponse_formatfield with wire-format structs for JSON and JSON Schema modesResponseMimeTypeandResponseSchemafieldsSupportsFormatmethod instead of hardcoded provider checksRemoved
TestRun_InvalidModelFormatstandalone test (consolidated intoTestRun_ProviderConfigValidation)TestRun_UnsupportedProviderstandalone test (consolidated intoTestRun_ProviderConfigValidation)