OpenJev

Fast, calibrated, typed decisions from an open model. OpenJev is an open-source "System One" decision server. Send it a state and typed questions (yes/no, choice, score). It returns a probability and a confidence for each answer in tens of milliseconds. It reads the answers directly from the model's probabilities and parses no text, so an answer cannot go off-schema. Questions can also ask about images.

OpenJev uses the same wire API as TypeSafe's Jev, so their SDKs work with it unchanged. It runs DiffusionGemma 26B-A4B (Apache-2.0) through vLLM on an NVIDIA GPU, or through MLX on Apple silicon.

Hosted for free on Codiv, an inference platform for open System One models. Sign up and get 100M input tokens, no card required. https://api.codiv.ai/v1/systemone

OpenJev is an independent project. It is not affiliated with or endorsed by TypeSafe AI.

Try it

pip install typesafe-sdk
export TYPESAFE_BASE_URL=https://api.codiv.ai   # or http://127.0.0.1:8080 for your own server
export TYPESAFE_API_KEY=sk-codiv-...

from typesafe_sdk import TypeSafeClient

client = TypeSafeClient()
r = client.system_one(
    "Everything is down and we have a demo with our biggest client at noon.",
    {
        "urgent": {"type": "noul", "instructions": "Does the customer need a reply within the hour?"},
        "team":   {"type": "choice", "instructions": "Which team should handle it?",
                   "criteria": {"outage": "service down", "billing": "charges, refunds", "feature": "requests, how-to"}},
        "tone":   {"type": "score", "instructions": "How upset is the customer?",
                   "criteria": ["calm", "annoyed", "furious"]},
    },
)
r.nouls["urgent"].noul        # 1.00
r.choices["team"].choice      # "outage", confidence 1.00
r.scores["tone"].score        # 2.00 (expected level, 0-indexed)

Or with curl:

curl https://api.codiv.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openjev-latest", "state": "I was charged twice this month.",
       "questions": {"is_billing": {"type": "noul", "instructions": "Is this a billing issue?"}}}'

API

POST /v1/systemone	{state, model, questions} → {model, answers, usage}
POST /v1/chat/completions	OpenAI-style text generation with model diffusiongemma-26b (below)
GET /v1/models	openjev-0.1, its alias openjev-latest, diffusiongemma-26b, and the small encoder models when they run. The server also accepts jev-latest and jev-preview, so TypeSafe SDK defaults work.

Question types:

• noul (yes/no): takes optional criteria: {true, false}. Returns {noul: P(yes)}.
• choice: takes criteria: {name: description}. Returns {choice, probabilities, confidence}.
• score: takes criteria: [level0, level1, …] (2–10 levels). Returns {score: Σ i·pᵢ, legend, probabilities, confidence}.

confidence is 1 − H(p)/ln K: 1 when the model is certain, 0 when the distribution is uniform. usage.input_tokens counts prompt tokens, image tokens included. usage.output_tokens is 0 unless you set think.

Each response has a Server-Timing header:

server-timing: model;dur=41.2, server;dur=2.8, total;dur=44.0

model is the time spent on the model, summed over the request's reads. It can be more than total when reads run in parallel. server is the remaining time: schema compile, tokenization, validation and serialization. Neither includes your network.

Errors use the same shapes as Jev, checked against the live API:

• 422 with a FastAPI validation list for a field of the wrong shape.
• 400 with a plain-text reason for a question the server cannot ask (no options, too many options, or too many score levels).
• 400 api_usage_error for an unknown model or question type.
• {"detail": {"error_type", "message"}} for auth errors (401/403).
• 429 for rate limits. 529 when the server is overloaded.

Differences from Jev:

• Model names are OpenJev's own. jev-latest and jev-preview are aliases. A pinned Jev version such as jev-1.13.0 gets 400 Unknown model.
• The server reads many questions in chunks of about 12 per read. The chunks run in parallel.

Extensions

These optional request fields are OpenJev additions. A request without them behaves exactly like Jev. TypeSafe's SDKs never send them. They come from the example server in vllm-project/vllm#57250.

Field	Values	What it does	Cost
images	up to 8	Images for the questions, placed before the state. Each is a data:image/...;base64, URL or {"content_type", "base64"}. JPEG, PNG, WebP or GIF, 5 MB each.	about 280 input tokens per image
steps	1–8, default 1	Denoise steps per read. More steps let the answers settle against each other.	same tokens, more GPU time
samples	1–32	Read N times with different noise and average. This replaces the automatic re-reads. samples: 1 gives one read, the fastest answer.	N × input tokens
think	0–4096 tokens	The model writes a thought, then reads the answers after it. The number is a hard cap, and a longer thought is cut. Give multi-step problems 512 or more.	input tokens twice, plus the thought as output tokens
sequential	true	For long question lists: read the chunks in order. Each chunk sees the answers before it.	one read per chunk, in series

curl https://api.codiv.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openjev-latest", "state": "Look at the photo.",
       "images": ["data:image/jpeg;base64,/9j/4AAQ..."],
       "questions": {"hotdog": {"type": "noul", "instructions": "The photo shows a hot dog"}}}'

think and sequential need a text state. A request that combines them with images gets a 400.

Text generation

POST /v1/chat/completions generates text with the same model, OpenAI style. Use model diffusiongemma-26b. It supports streaming and tools. The server ignores sampling fields (temperature, seed, penalties and similar) because vLLM refuses them for diffusion models. max_tokens defaults to 1024, with a cap of 8192. At most 8 generations run at once, so reads always have room. The MLX backend ignores tools and logprobs and does not return the thought.

How it works

DiffusionGemma is a discrete diffusion model. It denoises a full canvas of tokens in each forward pass, instead of writing left to right. OpenJev uses this to read answers, not write them.

OpenJev builds a canvas in which only the answer slots are masked, one token per question:

canvas in                 one read-only pass         answer out
  q1: [?]        ──►      P(yes) 0.001        ──►    noul  0.001
  q2: [?]                 P(A) 0.000                 choice "billing"
                          P(B) 0.999                 confidence 0.997
                          P(C) 0.000
  q3: [?]                 P(0) 0.000                 score 1.00
                          P(1) 0.996
                          P(2) 0.004

Each label is one token: yes/no for a noul, A/B/C for a choice, 0/1/2 for a score. The model never writes into these slots. One read-only pass gives the probability distribution for each slot, and that distribution is the answer. The numbers above are a real read of "The invoice looks wrong again. Second time this quarter.": not urgent, billing, mildly annoyed.

The read scores only the label tokens, so an answer cannot go off-schema. The confidence comes from the model's own distribution, not from a number that the model reports about itself.

If a slot is uncertain (entropy > 0.1), OpenJev reads three more times with fresh noise and averages the four results. One uncertain question causes a re-read of all the questions in the request. These extra reads add no tokens to usage. Set samples: 1 to get one read only. Question ids never go to the model. It sees q1, q2, q3.

The vLLM part is vllm-project/vllm#57250, merged on 2026-09-22. It adds seeded canvases, read-only steps, step caps and pinned canvas positions for DiffusionGemma. openjev/engine.py adapts that PR's structured_server.py example, with async I/O, bounded concurrency and backpressure.

Run your own

Two backends serve the same /v1/systemone. Select one by hardware:

	vLLM (default)	MLX
Hardware	NVIDIA GPU, 24 GB or more	Apple silicon, about 16 GB free
Setup	Docker image	pip install -e '.[mlx]'
Reads	up to 64 in flight	one at a time
images, steps > 1, think	yes	yes
Text generation	yes	yes, streaming included

NVIDIA GPU

You need an NVIDIA GPU with at least 24 GB of memory for the NVFP4 checkpoint. We tested on an RTX PRO 6000 Blackwell (sm_120).

The prebuilt image razorback16/openjev runs vLLM and the API server in one container, on CUDA 13. It pins the upstream vLLM commit below, with the two changes in Caveats.

git clone https://github.com/razorback16/openjev && cd openjev
docker compose up -d          # OpenJev on 127.0.0.1:8080 when the model is loaded
curl localhost:8080/v1/models

Or without compose:

docker run -d --gpus all --ipc=host -p 127.0.0.1:8080:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface razorback16/openjev:0.5.0

The first start downloads the weights (about 18 GB) into ~/.cache/huggingface. To build the images yourself, build the shared base first, then run docker compose build:

docker build -f docker/Dockerfile.base -t razorback16/openjev-base:cu130-torch2.13 .
docker compose build

vLLM listens only inside the container. Set OPENJEV_UPSTREAM to use a vLLM server that you already run.

Measured on an RTX PRO 6000 at 38% of the GPU, with 3 questions per request and cache-busted states:

Concurrency	req/s	p50	p95
1	10.7	94 ms	94 ms
16	43.3	367 ms	369 ms
32	51.7	545 ms	618 ms
64	57.4	760 ms	1109 ms

One request at a time, on the same GPU, with samples: 1:

Request	p50	p95
1 question	27 ms	28 ms
3 questions	31 ms	32 ms

Without Docker:

git clone https://github.com/vllm-project/vllm && cd vllm
VLLM_COMMIT=1b3b88ec2b7457aa030db4d0e7d8aaf04f6d0fb8   # the commit the image pins
git checkout $VLLM_COMMIT
# a choice of more than 128 options needs a larger cap, as in the image
sed -i 's/^MAX_LOGPROB_TOKEN_IDS = 128$/MAX_LOGPROB_TOKEN_IDS = 512/' vllm/sampling_params.py
VLLM_USE_PRECOMPILED=1 VLLM_PRECOMPILED_WHEEL_COMMIT=$VLLM_COMMIT pip install -e .
vllm serve nvidia/diffusiongemma-26B-A4B-it-NVFP4 --served-model-name dgemma \
  --diffusion-config '{"canvas_length": 64}' --max-logprobs 32 --enable-prefix-caching \
  --async-scheduling --attention-backend TRITON_ATTN \
  --limit-mm-per-prompt '{"image": 8, "video": 0}' \
  --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 \
  --override-generation-config '{"max_new_tokens": null}'
pip install -e path/to/openjev && python -m openjev

To match the image's image reads, also apply docker/patches/vision_prefix_lm.py to this checkout. Without it, vLLM prefills images causally.

Apple silicon

A Mac needs no vLLM and no Docker. OPENJEV_BACKEND=mlx runs DiffusionGemma inside the OpenJev process through MLX and mlx-vlm. The 4-bit weights need about 16 GB of memory.

pip install -e '.[mlx]'
OPENJEV_BACKEND=mlx python -m openjev     # 127.0.0.1:8080

This backend uses the same prompts, canvases and seeds as vLLM. It supports images, samples, sequential, steps, think and the automatic re-reads, and it bills the same. Each step after the first reuses one prefill of the prompt. More steps cost GPU time, not prompt tokens. The re-reads and samples of one request share one vision pass.

Reads run one at a time, so this backend is for local use, not for serving. A 3-question request takes about 0.2–0.4 s on an M3 Ultra and about 0.39 s on an M4 Max, both with the 4-bit weights. 16 concurrent requests finish at about 4 req/s.

Small encoder models

OpenJev also serves two small System One models that other people built and trained. The credit is theirs. OpenJev only puts them behind the same API. Each is a bidirectional encoder with a classification head, not a diffusion model. It reads each question in one forward pass, so an answer is still a distribution over your options.

Model id	Model	Size	State limit	Choices
laya-1.0	Laya by Nandakishor M / Convai Innovations. The laya-typed-decisions checkpoint: ModernBERT-large, fine-tuned on the typed-decisions workflows.	421M	1,024 tokens, options included	up to 255. The options share 256 tokens, so with many options each option is cut to a few tokens. Use about 20 at most, or split the question.
verdict-1.4	Verdict by Heman10x. The rlcd-modernbert-151m checkpoint with Verdict's v1.4 inference engine: ModernBERT-base with a GLiClass head, calibrated per option count.	151M	512 tokens, options included	up to 24

Each model runs in its own container (OPENJEV_BACKEND=laya or verdict). docker compose up -d starts both next to the vLLM container on the same GPU. The openjev container sends their requests to them (OPENJEV_MODEL_ROUTES), so :8080 serves all three models. /v1/models lists a routed model even when its container is stopped. A request for it then gets a 503. On a GPU both models use bf16 weights and need about 3.7 GB together. Set OPENJEV_GPU_UTIL to keep that memory free.

Measured on an RTX PRO 6000 with 16 questions per request. GPU memory is the nvidia-smi value, CUDA context included. A short state is about 50 tokens. A full state fills the model's limit.

Model	GPU memory	16 questions, short state	16 questions, full state
laya-1.0	2.5 GB	10 ms	109 ms (16 × 1,024 tokens)
verdict-1.4	1.2 GB	7 ms	21 ms (16 × 512 tokens)

FlashAttention 2 gave no improvement. Memory was the same, speed was within 6%, and short states were slower.

To run one model alone, on a GPU or on the CPU:

docker build -f docker/Dockerfile.laya -t openjev-laya .   # after the base, as above
docker run -d --gpus all -p 127.0.0.1:8081:8080 \
  -v ~/.cache/huggingface:/root/.cache/huggingface openjev-laya
# or without Docker:
pip install -e '.[laya]' && OPENJEV_BACKEND=laya python -m openjev

A server that runs one of these models alone also accepts jev-latest and jev-preview for it.

Each image has its own Dockerfile in docker/. All three start from docker/Dockerfile.base and share its CUDA, Python and PyTorch layers (about 8.7 GB on disk). A server that runs all three stores these layers once, about 21 GB in total.

Differences from the DiffusionGemma model:

• Text only. images, steps above 1, samples above 1, think and sequential get a 400.
• The server cuts a state that is longer than the limit, with no error. usage.input_tokens counts what the model read. Each question is a separate sequence, so each question bills the state again.
• Verdict adds an "insufficient evidence" option to each question. OpenJev removes it and scales the other probabilities to a sum of 1, as Jev's answer shapes require. Verdict also ignores a noul's criteria.
• Laya rounds each probability to 4 decimal places.
• On a GPU the weights are bf16, not fp32. On 36 test answers per model, no top option changed. Probabilities moved by at most 0.021 (Laya) and 0.007 (Verdict).
• The Verdict prompt format and temperatures come from Verdict's inference engine (v1.4). For the same input, the probabilities are the same as that engine's output.

For benchmarks, training, fine-tuning and known limits, read the authors' repositories: Laya and Verdict. The laya container runs Laya's laya package. Verdict uses ModernBERT (Answer.AI, LightOn) and GLiClass (Knowledgator).

Settings

The server reads its settings from the environment.

Variable	Default	Meaning
OPENJEV_BACKEND	vllm	mlx to run the model in-process on Apple silicon. laya or verdict for a small encoder model
OPENJEV_MODEL_ROUTES	unset	name=url,...: other OpenJev servers. A request for one of these model names goes to that server unchanged.
OPENJEV_LAYA_MODEL	convaiinnovations/laya-typed-decisions	Laya weights: a local directory or a Hugging Face id
OPENJEV_VERDICT_MODEL	heman10x/rlcd-modernbert-151m	Verdict weights: a local directory or a Hugging Face id
OPENJEV_DEVICE	unset	laya/verdict: cuda or cpu. Unset uses CUDA when a GPU is present
OPENJEV_ENCODER_BATCH	16	laya/verdict: the most questions in one forward pass. A larger request uses more passes.
OPENJEV_UPSTREAM	unset	external vLLM server URL. When set, the container does not start its own
OPENJEV_MODEL	nvidia/diffusiongemma-26B-A4B-it-NVFP4	weights for the built-in vLLM
OPENJEV_MLX_MODEL	mlx-community/diffusiongemma-26B-A4B-it-4bit	MLX weights: a local directory or a Hugging Face id. Also gives the tokenizer. 8bit and bf16 builds are also available.
OPENJEV_MLX_MAX_PROMPT	32768	longest request, in tokens, before a 400
OPENJEV_GPU_UTIL	0.9	vLLM --gpu-memory-utilization
OPENJEV_MAX_NUM_SEQS	64	vLLM --max-num-seqs
OPENJEV_MAX_MODEL_LEN	65536	vLLM --max-model-len
OPENJEV_VLLM_ARGS	unset	extra vllm serve flags
OPENJEV_CANVAS	64	canvas length. Also sets the built-in vLLM's --diffusion-config
OPENJEV_MAX_INFLIGHT	64	reads in flight to vLLM
OPENJEV_MAX_QUEUE	512	waiting decisions before the server returns 529
OPENJEV_API_KEY	unset	require Authorization: Bearer <key>
OPENJEV_ORIGIN_SECRET	unset	require an X-Origin-Secret header (for use behind a proxy)
OPENJEV_MAX_IMAGES	8	images per request. Also sets the built-in vLLM's --limit-mm-per-prompt
OPENJEV_MAX_IMAGE_BYTES	5242880	size limit per image, after base64 decoding
OPENJEV_GEN_MAX_INFLIGHT	8	text generations that run at once
OPENJEV_GEN_MAX_QUEUE	32	waiting generations before the server returns 529
OPENJEV_GEN_MAX_TOKENS	8192	cap on max_tokens for text generation
OPENJEV_WARMUP	1	0 skips the warmup requests before the API opens. Warmup saves the first users several seconds of compilation.

Caveats

• The image pins upstream vLLM and makes two changes. A build fails if either change no longer applies.
  • It raises the limit of exact label ids per request from 128 to 512, for choices of up to 255 options.
  • docker/patches/vision_prefix_lm.py gives image tokens bidirectional attention, as the checkpoint config asks. Upstream vLLM does this for Gemma4 but not yet for DiffusionGemma.
• Answer quality is the quality of DiffusionGemma 26B-A4B in this mode. Evaluate it on your own tasks before you rely on it.

Development

pip install -e '.[test]' && pytest
OPENJEV_LIVE_URL=http://127.0.0.1:8080 pytest tests/test_live.py   # end to end against a running server
OPENJEV_MLX_TEST_MODEL=path/to/weights pytest tests/test_mlx_model.py   # MLX against the real model

Run the live checks after you build an image and before a cutover. They cover each read option, images, chat, and the encoder models that the server lists.

License

Apache-2.0. The DiffusionGemma weights are Apache-2.0 (NVIDIA / Google). Laya (Nandakishor M / Convai Innovations) and Verdict (Heman10x) are Apache-2.0, weights and code. openjev/encoders.py adapts Verdict's prompt format and calibration from its repository.