Skip to content

feat(ai): support image-guided generation#37998

Merged
rekram1-node merged 2 commits into
v2from
image-inputs
Jul 20, 2026
Merged

feat(ai): support image-guided generation#37998
rekram1-node merged 2 commits into
v2from
image-inputs

Conversation

@opencode-agent

@opencode-agent opencode-agent Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add ordered images to the existing Image.generate({ model, prompt, images?, options?, http? }) API
  • dispatch image-conditioned requests to OpenAI, xAI, and Gemini edit behavior while rejecting unsupported Z.ai image inputs before network I/O
  • keep masks OpenAI-specific as typed options.mask?: ImageInput; only images is common across providers
  • preserve provider-native open options, raw native field precedence, and documented http overlays
  • add deterministic routing/type coverage plus live recorded edit cassettes with validated MIME and dimensions

Input model

ImageInput is a tagged union with constructors for MIME-bearing bytes, HTTP/data URLs, provider file IDs, and MIME-bearing provider file URIs. Raw strings are intentionally excluded to avoid ambiguous base64, URL, and provider-ID interpretation.

images is the only common image-editing input. OpenAI additionally accepts options: { mask: ImageInput.bytes(...) }; the adapter extracts that helper into the native multipart or JSON edit mask field rather than serializing the tagged helper object as a normal option.

Provider behavior

  • OpenAI: /images/generations for text-only requests; /images/edits using multipart for bytes/data URLs and the documented JSON reference shape for URLs/file IDs. Multipart owns its content-type boundary and protects structural model, prompt, image[], and mask fields from raw body overlays.
  • xAI: /images/generations without inputs; JSON /images/edits with singular image or ordered plural images; bytes become data URLs.
  • Google: same generateContent route with ordered inlineData/fileData parts; public HTTP URLs fail before network I/O.
  • Z.ai: image inputs fail before network I/O because the hosted international image API is text-only.
  • OpenAI Responses hosted image-tool behavior is unchanged.

Verification

  • package-scoped Prettier
  • bun typecheck
  • bun run build
  • bun test --timeout 30000 (395 passed, 29 skipped)
  • OpenAI, xAI, and Google edit cassette replays passed with cassette files unchanged
  • changed-file secret scan passed

Requested by: @rekram1-node (Aiden Cline via Slack)

@rekram1-node
rekram1-node merged commit e93fa06 into v2 Jul 20, 2026
9 checks passed
@rekram1-node
rekram1-node deleted the image-inputs branch July 20, 2026 21:21
github-actions Bot pushed a commit to ReStranger/opencode that referenced this pull request Jul 20, 2026
* upstream/v2: (63 commits)
  feat(codemode): add Set method parity (anomalyco#38000)
  fix(core): match dev patch behavior (anomalyco#37709)
  refactor(tui): migrate diff viewer to V2 theme (anomalyco#37999)
  feat(ai): support image-guided generation (anomalyco#37998)
  feat(ai): add Z.ai image generation (anomalyco#37780)
  chore: merge dev into v2 (anomalyco#37996)
  refactor(tui): expose theme colors as properties (anomalyco#37995)
  feat(ai): add Google image generation (anomalyco#37781)
  fix(cli): isolate bun updater cache
  feat(ai): add xAI image generation (anomalyco#37778)
  feat(codemode): add groupBy parity (anomalyco#37992)
  refactor(ai): infer image provider options (anomalyco#37948)
  fix(core): ignore empty provider api keys
  feat(tui): add categorical theme hues (anomalyco#37985)
  refactor(core): inject models options
  refactor(server): inject database options
  fix(tui): make migrated selected background transparent (anomalyco#37967)
  feat(codemode): complete date parity (anomalyco#37960)
  feat(codemode): complete regexp and math parity (anomalyco#37955)
  fix(tui): adapt file badge accent to theme mode (anomalyco#37953)
  ...
github-actions Bot pushed a commit to ReStranger/opencode that referenced this pull request Jul 21, 2026
* upstream/v2: (65 commits)
  refactor(server): inject runtime options
  refactor(server): centralize server options
  feat(codemode): add Set method parity (anomalyco#38000)
  fix(core): match dev patch behavior (anomalyco#37709)
  refactor(tui): migrate diff viewer to V2 theme (anomalyco#37999)
  feat(ai): support image-guided generation (anomalyco#37998)
  feat(ai): add Z.ai image generation (anomalyco#37780)
  chore: merge dev into v2 (anomalyco#37996)
  refactor(tui): expose theme colors as properties (anomalyco#37995)
  feat(ai): add Google image generation (anomalyco#37781)
  fix(cli): isolate bun updater cache
  feat(ai): add xAI image generation (anomalyco#37778)
  feat(codemode): add groupBy parity (anomalyco#37992)
  refactor(ai): infer image provider options (anomalyco#37948)
  fix(core): ignore empty provider api keys
  feat(tui): add categorical theme hues (anomalyco#37985)
  refactor(core): inject models options
  refactor(server): inject database options
  fix(tui): make migrated selected background transparent (anomalyco#37967)
  feat(codemode): complete date parity (anomalyco#37960)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant