A Julian, type-safe interface to LLM providers with first-class native backends — OpenAI (Chat Completions + Responses), Anthropic (Messages), and Google Gemini (generateContent + agentic Interactions) — plus any OpenAI-compatible provider (Azure, DeepSeek, Mistral, Ollama, vLLM, LM Studio). Covers the Chat Completions & Responses APIs, a cross-provider agentic respond verb, Image Generation/Edits, Embeddings, Files/Vector Stores, Conversations, Audio, Batch, Moderations, Fine-tuning, Webhooks, Realtime, and MCP (client & server) — with built-in token/cost accounting and illegal states made unrepresentable.
UniLM speaks each provider's own wire API, not just the OpenAI-compatible protocol. Reach for it when you need:
- Native Anthropic and Gemini backends — each speaks the provider's own format (Anthropic Messages, Gemini
generateContent) rather than an OpenAI-compat shim, and round-trips provider-verbatim content, so reasoning state such as Anthropic thinking signatures and Gemini thought signatures survives across turns. - An MCP client and an MCP server in one package — connect to external MCP servers (
MCPSession) with tool-loop integration, and expose your own Julia functions as tools over MCP (MCPServer). - Typed results with fail-loud invariants — every call resolves to a concrete
LLMSuccess/LLMFailure/LLMCallErrorresult (with matchingResponse…types for the Responses API), and genuine faults such as timeouts raise typed exceptions (UniLMTimeout) instead of returning silent defaults, and invalid conversation mutations raise a typedInvalidConversationErrorrather than corrupting the conversation. - Built-in per-conversation cost accounting — provider token counts are normalized to one shape, and each
Chataccumulates a running USD estimate you read withcumulative_cost. - Broad OpenAI platform-API coverage — well beyond chat: Responses, Images, Embeddings, Files, Vector Stores, Conversations, Audio, Batch, Moderations, Fine-tuning, Webhooks, and Realtime.
- Chat Completions — stateful conversations with automatic history management
- Responses API & Agentic Verb — OpenAI's Responses API with built-in tools, multi-turn chaining, and reasoning; the unified
respondverb also drives Google's Gemini Interactions - Image Generation & Edits — create and edit images with
gpt-image-2 - Tool/Function Calling — first-class support for function tools in both APIs, with automated
tool_loop - MCP (Model Context Protocol) — connect to MCP servers or build your own, with seamless tool loop integration
- Embeddings — text embedding generation with
text-embedding-3-small - Files, Vector Stores & Conversations — upload files, build vector stores for
file_search, and manage server-side conversation state - Audio, Batch & Moderations — TTS/transcription, async 50%-off bulk jobs, and free safety classification
- Realtime, Fine-tuning, Webhooks, Containers, Uploads & Videos — WebSocket realtime, custom models, signed-webhook verification, and more. See provider availability limits for fine-tuning and videos.
- Streaming — real-time token streaming with
do-block syntax - Structured Output — JSON Schema–constrained generation
- Multi-Backend — OpenAI, Azure, Gemini, Anthropic, DeepSeek, Ollama, Mistral, vLLM, LM Studio, and any OpenAI-compatible provider
- Type Safety — invalid states are unrepresentable; tested with JET.jl and Aqua.jl
UniLM requires Julia 1.12+ and is registered in Julia's General registry:
using Pkg
Pkg.add("UniLM")Or in the Pkg REPL:
pkg> add UniLM
For the latest unreleased changes, install directly from GitHub:
Pkg.add(url="https://github.com/algunion/UniLM.jl")💡 Costs & free local option: hosted API calls bill your provider key. To try UniLM for free with no key, use a local model via Ollama (
service=OllamaEndpoint()) — see Multi-Backend Support.
Set your API key:
export OPENAI_API_KEY="sk-..."julia> using UniLM, JSON
julia> result = respond("Explain Julia's multiple dispatch in 2-3 sentences.")
julia> output_text(result)
"Julia's multiple dispatch means a function can have many method definitions, and Julia chooses which one to run based on the types of *all* arguments in a call (not just the first). This makes it easy to write generic code while still getting specialized, high-performance behavior for specific type combinations."
julia> result.response.model
"gpt-5.6-sol"julia> chat = Chat(model="gpt-5.4-mini")
julia> push!(chat, Message(Val(:system), "You are a concise Julia programming tutor."))
julia> push!(chat, Message(Val(:user), "What is multiple dispatch? Answer in 2-3 sentences."))
julia> result = chatrequest!(chat)
julia> println(text(result))
Multiple dispatch is a feature in programming languages, including Julia, that allows the selection of a method to execute based on the types of all its arguments, rather than just the first one. This enables more flexible and expressive code, as it can define different behaviors for a function depending on the combination of argument types. It supports polymorphism, making it easier to write generic code that works with multiple types.
julia> length(chat) # system + user + assistant
3Use issuccess(result) / isfailure(result) to branch on the outcome; text(result) returns the reply text and throws LLMResultError on a failed call.
julia> result = chatrequest!(
systemprompt="You are a calculator. Respond only with the number.",
userprompt="What is 42 * 17?",
model="gpt-5.4-mini",
temperature=0.0
)
julia> println(text(result))
714julia> result = generate_image(
"A watercolor painting of a friendly robot reading a Julia programming book",
size="1024x1024", quality="medium"
)
julia> result isa ImageSuccess
true
julia> save_image(image_data(result)[1], "robot_julia.png")
"robot_julia.png"julia> emb = Embeddings("Julia is a high-performance programming language for technical computing.")
julia> embeddingrequest!(emb)
julia> emb.embeddings[1:5]
5-element Vector{Float64}:
-0.039474
-0.009283
0.001706
-0.028087
0.063363julia> task = respond("Write a haiku about Julia programming.") do chunk, close
if chunk isa String
print(chunk)
elseif chunk isa ResponseObject
println("\nDone! Status: ", chunk.status)
end
end
Multiple dispatch sings,
Types align in swift fusion—
Loops bloom into speed.
Done! Status: completedjulia> fmt = json_schema_format(
"languages", "A list of programming languages",
Dict(
"type" => "object",
"properties" => Dict(
"languages" => Dict(
"type" => "array",
"items" => Dict(
"type" => "object",
"properties" => Dict(
"name" => Dict("type" => "string"),
"year" => Dict("type" => "integer"),
"paradigm" => Dict("type" => "string")
),
"required" => ["name", "year", "paradigm"],
"additionalProperties" => false
)
)
),
"required" => ["languages"],
"additionalProperties" => false
),
strict=true
)
julia> result = respond("List Julia, Python, and Rust with their release year and primary paradigm.", text=fmt)
julia> JSON.parse(output_text(result))
{
"languages": [
{"name": "Julia", "year": 2012, "paradigm": "Multi-paradigm (scientific/numerical, functional, concurrent)"},
{"name": "Python", "year": 1991, "paradigm": "Multi-paradigm (object-oriented, imperative, functional)"},
{"name": "Rust", "year": 2010, "paradigm": "Multi-paradigm (systems programming, functional, imperative)"}
]
}Responses API:
julia> weather_tool = function_tool(
"get_weather", "Get the current weather for a given location",
parameters=Dict(
"type" => "object",
"properties" => Dict(
"location" => Dict("type" => "string", "description" => "City name"),
"unit" => Dict("type" => "string", "enum" => ["celsius", "fahrenheit"])
),
"required" => ["location", "unit"],
"additionalProperties" => false
),
strict=true
)
julia> result = respond("What's the weather in Tokyo? Use celsius.", tools=[weather_tool])
julia> calls = function_calls(result)
julia> calls[1]["name"]
"get_weather"
julia> JSON.parse(calls[1]["arguments"])
{"location": "Tokyo", "unit": "celsius"}Web Search:
julia> result = respond(
"What is the latest stable release of the Julia programming language?",
tools=[web_search()]
)
julia> output_text(result)
"The latest **stable** release of the Julia programming language is **Julia v1.12.5**."Responses API (via previous_response_id):
julia> r1 = respond("Tell me a one-liner programming joke.", instructions="Be concise.")
julia> output_text(r1)
"There are only 10 kinds of people in the world: those who understand binary and those who don't."
julia> r2 = respond("Explain why that's funny, in one sentence.", previous_response_id=r1.response.id)
julia> output_text(r2)
"It's funny because \"10\" looks like ten in decimal but equals two in binary, so it sets up a nerdy misdirection that only people who know binary immediately get."UniLM.jl is built around neutral verbs: the same Chat + chatrequest! (tools, streaming, and cost accounting included) run unchanged across OpenAI, Anthropic, Gemini, DeepSeek, and any OpenAI-compatible provider — you only change service. The agentic respond verb is neutral the same way across OpenAI (Responses) and Gemini (Interactions). Native OpenAI/Anthropic/Gemini are first-class backends with their own wire formats (each exercised by live integration tests), not OpenAI-compatible shims. Switch via the service parameter:
| Backend | Type | Env Variables |
|---|---|---|
| OpenAI (default) | OPENAIServiceEndpoint |
OPENAI_API_KEY |
| Azure OpenAI | AZUREServiceEndpoint |
AZURE_OPENAI_BASE_URL, AZURE_OPENAI_API_KEY, AZURE_OPENAI_API_VERSION |
| Google Gemini | GEMINIServiceEndpoint |
GEMINI_API_KEY |
| Anthropic | ANTHROPICServiceEndpoint |
ANTHROPIC_API_KEY |
| DeepSeek | DeepSeekEndpoint() |
DEEPSEEK_API_KEY |
| Ollama (local) | OllamaEndpoint() |
none |
| Mistral | MistralEndpoint() |
MISTRAL_API_KEY |
| Any OpenAI-compat | GenericOpenAIEndpoint(url, key) |
custom |
# Azure
chat = Chat(service=AZUREServiceEndpoint, model="gpt-5.2")
# Gemini (native generateContent)
chat = Chat(service=GEMINIServiceEndpoint) # default: gemini-3.8-flash
# Anthropic (native Messages API)
chat = Chat(service=ANTHROPICServiceEndpoint) # default: claude-opus-4-8
# DeepSeek
chat = Chat(service=DeepSeekEndpoint(), model="deepseek-chat")
# Ollama (local)
chat = Chat(service=OllamaEndpoint(), model="llama3.1")UniLM speaks each provider's own API (see Multi-Backend Support). For OpenAI, you can use either of two conversational APIs. Chat Completions (Chat + chatrequest!) is the portable path — it's also how the native Anthropic and Gemini backends and every OpenAI-compatible provider work. Responses (respond) is OpenAI's newer API, and the basis for the cross-provider agentic verb (which also targets Gemini Interactions). They map like this:
| Feature | Chat Completions | Responses API |
|---|---|---|
| Stateful conversations | Chat + push! |
previous_response_id |
| System prompt | Message(Val(:system), ...) |
instructions kwarg |
| Tool calling | Tool / ToolCall |
FunctionTool / function_tool |
| Web search | — | WebSearchTool |
| File search | — | FileSearchTool |
| Streaming | stream=true + callback |
do-block syntax |
| Structured output | ResponseFormat |
TextConfig / json_schema_format |
| Reasoning (O-series) | — | Reasoning |
| Automated tool loop | tool_loop! |
tool_loop |
| MCP integration | mcp_tools bridge |
MCPTool / mcp_tool |
Full documentation with guides and API reference: https://algunion.github.io/UniLM.jl/dev/
- Getting Started — setup and first requests
- Chat Completions Guide —
Chatandchatrequest! - Responses API Guide — the newer Responses API
- Image Generation Guide — create images from text
- Tool Calling Guide — function calling
- Agentic Workflows Guide — cross-provider
respond(OpenAI Responses + Gemini Interactions) - Streaming Guide — real-time streaming
- Structured Output Guide — JSON Schema output
- Multi-Backend Guide — Azure, Gemini, DeepSeek, Ollama, and more
- MCP Guide — MCP client/server
- Timeouts & Retries Guide — bounds, typed failures, retry and concurrency contracts
Every network operation waits on a peer only under a bounded, configurable limit,
and reports a breach as a typed error. All bounds live on one RequestConfig,
resolved per call:
# Per call
chatrequest!(chat; config=RequestConfig(request_timeout=60.0, max_attempts=1))
# For a block of calls (propagates into spawned tasks)
with_request_config(request_timeout=30.0) do
chatrequest!(chat)
embeddingrequest!(emb)
end
set_default_config!(stream_idle_timeout=300.0) # process-wide, for notebooksA timeout surfaces as the call's usual error result with status = nothing and a
UniLMTimeout (phase, elapsed, limit) on .cause — never a hang and never a
fabricated HTTP status. max_attempts (default 3) applies to the inference verbs;
platform and lifecycle verbs make a single bounded attempt.
Two concurrency rules are worth knowing before you fan out:
- One
Chatper in-flight call. AChatis unsynchronized mutable state, so usefork(chat)/fork(chat, n)to fan out rather than sharing one. The stateless verbs (respond,embeddingrequest!,generate_image) need no such care, and anMCPSessionis concurrency-1 — one session per worker. - Prefer HTTP 2.x for high fan-out. HTTP 1.x shares one process-global
connection pool across all hosts, capped at
max(16, 4 × nthreads()), so a wide fan-out silently queues there.
The Timeouts & Retries guide has the full contract, including the stream idle bound and the sharp edges.
UniLM is pre-1.0. While on 0.x, MINOR releases (e.g. 0.13 → 0.14) may carry breaking changes — each is listed under a Breaking heading in the CHANGELOG with migration notes — while PATCH releases never break. Breaking changes are batched into infrequent minors rather than dribbled across releases, and renamed identifiers keep working as aliases until at least 1.0. See the Versioning & Stability policy for the full contract.