Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 43 additions & 2 deletions packages/ai/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# @opencode-ai/ai

Schema-first LLM core for opencode. One typed request, response, event, and tool language; provider quirks live in adapters, not in calling code.
Schema-first AI primitives for opencode. Provider quirks live in adapters, not in calling code.

```ts
import { Effect } from "effect"
Expand All @@ -24,6 +24,45 @@ const program = Effect.gen(function* () {

Run `LLMClient.stream(request)` instead of `generate` when you want incremental `LLMEvent`s. The event stream is provider-neutral — same shape across OpenAI Chat, OpenAI Responses, Anthropic Messages, Gemini, Bedrock Converse, and any OpenAI-compatible deployment.

## Image generation

Use `Image.generate` with an image model for direct asset generation:

```ts
import { Image } from "@opencode-ai/ai"
import { OpenAI } from "@opencode-ai/ai/providers"

const program = Effect.gen(function* () {
const response = yield* Image.generate({
model: OpenAI.configure({ apiKey: process.env.OPENAI_API_KEY }).image("gpt-image-2"),
prompt: "A robot tending a rooftop garden",
count: 2,
size: { width: 1024, height: 1024 },
providerOptions: { openai: { quality: "high", outputFormat: "webp" } },
})

return response.images // GeneratedImage[] with owned bytes or a provider URL
})
```

Conversational image generation remains part of the LLM interaction. OpenAI Responses exposes it through its hosted image tool:

```ts
const program = Effect.gen(function* () {
const response = yield* LLM.generate(
LLM.request({
model: OpenAI.configure({ apiKey }).responses("gpt-5"),
prompt: "Design a solarpunk rooftop garden, then show me.",
tools: [OpenAI.imageGeneration({ quality: "high" })],
}),
)

return response.message
})
```

The hosted result is represented as a provider-executed tool call and tool result. Its image is a `file` content item with a data URI, so retaining `response.message` preserves the generated image for continuation.

## Public API

- **`LLM.request({...})`** — build a provider-neutral `LLMRequest`. Accepts ergonomic inputs (`system: string`, `prompt: string`) that normalize into the canonical Schema classes.
Expand All @@ -32,6 +71,8 @@ Run `LLMClient.stream(request)` instead of `generate` when you want incremental
- **`Model.make(...)` / `ToolCallPart.make(...)` / `ToolResultPart.make(...)` / `ToolDefinition.make(...)`** — model and tool-related constructors from the canonical schema model.
- **`LLMClient.prepare(request)`** — compile a request through protocol body construction, validation, and HTTP preparation without sending. Useful for inspection and testing.
- **`LLMEvent.is.*`** — typed guards (`is.textDelta`, `is.toolCall`, `is.finish`, …) for filtering streams.
- **`Image.generate({...})`** — generate images through a provider-neutral image request and response model.
- **`ImageClient`** — Effect service and layer for image execution, parallel to `LLMClient`.

## Caching

Expand Down Expand Up @@ -182,7 +223,7 @@ Adding a new model or deployment is usually 5-15 lines using `Route.make({ proto

## Effect

This package is built on Effect. Public methods return `Effect` or `Stream`; provide `LLMClient.layer` for runtime dispatch and import the provider/protocol modules for the routes you use. The example at `example/tutorial.ts` is a runnable walkthrough.
This package is built on Effect. Public methods return `Effect` or `Stream`; provide `LLMClient.layer` for LLM dispatch and `ImageClient.layer` for image dispatch, then import the provider/protocol modules for the routes you use. The example at `example/tutorial.ts` is a runnable walkthrough.

## See also

Expand Down
34 changes: 34 additions & 0 deletions packages/ai/src/image-client.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import { Context, Effect, Layer } from "effect"
import { RequestExecutor } from "./route/executor"
import type { ImageRequest, ImageResponse } from "./image"
import type { LLMError } from "./schema"

export type Execute = RequestExecutor.Interface["execute"]

export interface Interface {
readonly generate: (request: ImageRequest) => Effect.Effect<ImageResponse, LLMError>
}

export class Service extends Context.Service<Service, Interface>()("@opencode/ImageClient") {}

export const generate = (request: ImageRequest): Effect.Effect<ImageResponse, LLMError> =>
Effect.gen(function* () {
const client = yield* Service
return yield* client.generate(request)
}) as Effect.Effect<ImageResponse, LLMError>

export const layer: Layer.Layer<Service, never, RequestExecutor.Service> = Layer.effect(
Service,
Effect.gen(function* () {
const executor = yield* RequestExecutor.Service
return Service.of({
generate: (request) => request.model.route.generate(request, executor.execute),
})
}),
)

export const ImageClient = {
Service,
layer,
generate,
} as const
116 changes: 116 additions & 0 deletions packages/ai/src/image.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
import { Effect, Schema } from "effect"
import { HttpOptions, InvalidRequestReason, LLMError, ModelID, ProviderID, ProviderMetadata, Usage } from "./schema"
import { ImageClient, type Execute as ImageExecute } from "./image-client"

export interface ImageRoute {
readonly id: string
readonly generate: (request: ImageRequest, execute: ImageExecute) => Effect.Effect<ImageResponse, LLMError>
}

export class ImageModel {
readonly id: ModelID
readonly provider: ProviderID
readonly route: ImageRoute
readonly defaults?: ImageModelDefaults

constructor(input: ImageModel.Input) {
this.id = input.id
this.provider = input.provider
this.route = input.route
this.defaults = input.defaults
}

static make(input: ImageModel.MakeInput) {
return new ImageModel({
id: ModelID.make(input.id),
provider: ProviderID.make(input.provider),
route: input.route,
defaults: input.defaults,
})
}
}

export namespace ImageModel {
export interface Input {
readonly id: ModelID
readonly provider: ProviderID
readonly route: ImageRoute
readonly defaults?: ImageModelDefaults
}

export interface MakeInput extends Omit<Input, "id" | "provider"> {
readonly id: string | ModelID
readonly provider: string | ProviderID
}
}

export interface ImageModelDefaults {
readonly providerOptions?: Record<string, Record<string, unknown>>
readonly http?: HttpOptions
}

export const ImageModelSchema = Schema.declare((value): value is ImageModel => value instanceof ImageModel, {
expected: "Image.Model",
})

export const ImageSize = Schema.Struct({
width: Schema.Int.check(Schema.isGreaterThanOrEqualTo(1)),
height: Schema.Int.check(Schema.isGreaterThanOrEqualTo(1)),
}).annotate({ identifier: "Image.Size" })
export type ImageSize = Schema.Schema.Type<typeof ImageSize>

export class ImageRequest extends Schema.Class<ImageRequest>("Image.Request")({
model: ImageModelSchema,
prompt: Schema.String,
count: Schema.optional(Schema.Int.check(Schema.isGreaterThanOrEqualTo(1))),
size: Schema.optional(ImageSize),
aspectRatio: Schema.optional(Schema.String),
seed: Schema.optional(Schema.Number),
providerOptions: Schema.optional(Schema.Record(Schema.String, Schema.Record(Schema.String, Schema.Unknown))),
http: Schema.optional(HttpOptions),
metadata: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)),
}) {}

export type ImageRequestInput = Omit<ConstructorParameters<typeof ImageRequest>[0], "http"> & {
readonly http?: HttpOptions.Input
}

export class GeneratedImage extends Schema.Class<GeneratedImage>("Image.Generated")({
mediaType: Schema.String,
data: Schema.Union([Schema.String, Schema.Uint8Array]),
providerMetadata: Schema.optional(ProviderMetadata),
}) {}

export class ImageResponse extends Schema.Class<ImageResponse>("Image.Response")({
images: Schema.Array(GeneratedImage),
usage: Schema.optional(Usage),
providerMetadata: Schema.optional(ProviderMetadata),
}) {
get image() {
return this.images[0]
}
}

export const request = (input: ImageRequest | ImageRequestInput) => {
if (input instanceof ImageRequest) return input
return new ImageRequest({
...input,
http: input.http === undefined ? undefined : HttpOptions.make(input.http),
})
}

export const generate = (input: ImageRequest | ImageRequestInput) =>
Effect.try({
try: () => request(input),
catch: (error) =>
new LLMError({
module: "Image",
method: "generate",
reason: new InvalidRequestReason({ message: error instanceof Error ? error.message : String(error) }),
}),
}).pipe(Effect.flatMap(ImageClient.generate))

export const Image = {
request,
generate,
} as const
4 changes: 4 additions & 0 deletions packages/ai/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
export { LLMClient } from "./route/client"
export { ImageClient } from "./image-client"
export { Auth } from "./route/auth"
export { Provider } from "./provider"
export { ProviderPackage } from "./provider-package"
Expand All @@ -10,6 +11,9 @@ export type {
Service as LLMClientService,
} from "./route/client"
export * from "./schema"
export { GeneratedImage, ImageModel, ImageRequest, ImageResponse, ImageSize } from "./image"
export type { ImageModelDefaults, ImageRequestInput, ImageRoute } from "./image"
export { Image } from "./image"
export { Tool, ToolFailure, toDefinitions } from "./tool"
export { ToolRuntime } from "./tool-runtime"
export type { DispatchResult as ToolDispatchResult, ToolSettlement } from "./tool-runtime"
Expand Down
1 change: 1 addition & 0 deletions packages/ai/src/protocols/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ export * as AnthropicMessages from "./anthropic-messages"
export * as BedrockConverse from "./bedrock-converse"
export * as Gemini from "./gemini"
export * as OpenAIChat from "./openai-chat"
export * as OpenAIImages from "./openai-images"
export * as OpenAICompatibleChat from "./openai-compatible-chat"
export * as OpenAICompatibleResponses from "./openai-compatible-responses"
export * as OpenAIResponses from "./openai-responses"
Loading
Loading