KORA

Deterministic-first execution control before model inference.

KORA is an AI Workload Control Layer that inspects and routes deterministic, reusable, retrieval-needed, tool-needed, and provider-needed work before model inference.

Most AI systems treat every task as a model task. KORA starts one step earlier: it inspects the workload, chooses a route, and makes provider-needed work explicit.

View the architecture diagram

What KORA Does

• Inspect workloads before deployment.
• Route deterministic work without provider calls.
• Reuse repeated work through cache paths.
• Separate retrieval-needed and tool-needed work.
• Mark provider-needed tasks explicitly.
• Inspect privacy-safe local device and runtime-candidate metadata.
• Run optional local execution paths through explicitly configured MLX or llama.cpp runtimes.
• Validate independently packaged AI Solutions before installation or execution.
• Build and query a deterministic local evidence index from text-layer PDFs.

Quick Start

Current latest-feature use is from source:

git clone https://github.com/Krako-Labs/KORA.git
cd KORA
python3 -m pip install -e .

Run the first-value paths:

python3 -m kora doctor examples/kora_doctor/customer_support_workload.json
python3 -m kora proxy-demo examples/openai_compatible_proxy/requests.json
python3 examples/cache_reuse/run.py

Inspect the local system without starting a runtime or calling a provider:

python3 -m kora system inventory

Solution Protocol v0alpha1

Create a deterministic offline Solution scaffold, then run its package-declared conformance suite:

python3 -m kora solution scaffold example.my-solution --output ./my-solution --json
python3 -m kora solution conform ./my-solution --json

The scaffold command refuses to overwrite an existing path and generates a manifest, Task Graph, input/output schemas, example input, and an integrity-bound conformance case. The conformance runner uses a fresh isolated Host store and emits a schema-validated report covering install, run, status, result, lifecycle, and offline activity facts.

Validate the bundled reference Solutions without executing them:

python3 -m kora solution validate examples/solutions/hello-solution --json
python3 -m kora solution validate examples/solutions/document-transform-fixture --json

A bounded reference Host can then install and run either Solution through the same local lifecycle:

python3 -m kora solution runtimes --store /tmp/kora-host --json
python3 -m kora solution install examples/solutions/hello-solution --store /tmp/kora-host --json
python3 -m kora solution run example.hello --store /tmp/kora-host --input examples/solutions/inputs/hello.json --json

The input file is a JSON object such as {"message":"Hello"}. The Host verifies both the installed snapshot and its local runtime registry, resolves one compatible trusted runtime before every run, records the runtime descriptor digest, and persists schema-validated status and result records. Its reference capabilities are deterministic and offline; stop/resume, provider/model/GPU execution, and production validation remain deferred. See Solution Protocol v0alpha1, the SDK and Conformance Kit, and the reference Solution guide.

Research Foundry Alpha

Install the optional PDF dependency, ingest your own or public text-layer PDFs, and retrieve an evidence card:

python3 -m pip install -e '.[research]'
python3 -m kora research ingest ./papers --state-dir ./.kora-research --json
python3 -m kora research query ./.kora-research "reflection tokens" --top-k 3 --markdown
python3 -m kora solution conform examples/solutions/research-foundry-reference --json

Research Foundry is an implemented reference vertical, not the definition of the KORA platform or a commercial-Solution selection. Its output is deterministic retrieved evidence—not model-generated synthesis. Each result includes a source title, page, stable chunk/evidence ID, and verbatim retrieved excerpt. The standalone CLI uses the state directory you explicitly select; the bounded Solution reference instead rebuilds state inside each isolated run workspace from one integrity-bound synthetic PDF. See Research Foundry Alpha and the Task 022 migration report for their Local Only boundaries and limitations.

Optional local runtime adapters

KORA includes fail-closed adapters named mlx_local and llama_cpp_local. They use only explicitly configured local runtimes and model files, never download a model or fall back to a remote provider. These are secondary execution adapters rather than a production-serving claim; see their environment-variable validation in the adapter modules and focused tests.

Package Availability

pip install kora is not this project.

The planned future PyPI package name is getkora, with CLI command kora and Python import package kora.

getkora is not published yet. Use the source install path above for the latest KORA features.

Flagship Examples

Example	Shows	Run	Details
KORA Doctor	Workload inspection	python3 -m kora doctor examples/kora_doctor/customer_support_workload.json	README
Deterministic Classification	Rule-routed classification	python3 examples/deterministic_classification/run.py	README
OpenAI-Compatible Proxy	OpenAI-style request routing	python3 -m kora proxy-demo examples/openai_compatible_proxy/requests.json	README
RAG Routing	Retrieval-aware control	python3 examples/rag_routing/run.py	README
Agent Workflow Optimization	Multi-step workflow routing	python3 examples/agent_workflow_optimization/run.py	README
Cache Reuse	Repeated-work reuse	python3 examples/cache_reuse/run.py	README

See the full example catalog.

How It Works

A workload enters KORA before it reaches a model.

KORA evaluates each unit of work and routes it to one of several paths:

• deterministic handling
• cache reuse
• retrieval-needed handling
• tool-needed handling
• provider-needed fallback

The included examples are offline and make zero provider calls.

Evidence Boundaries

In a reproducible 100-task deterministic-heavy benchmark workload, KORA-controlled execution avoided 80 of 100 simulated model invocations versus a naive direct baseline. This is a bounded simulated benchmark result, not production, cost-saving, model-quality, or broader workload evidence. See the benchmark report.

The repository does not claim:

• production cost reduction proof
• real API-cost reduction proof beyond the single-domain benchmark above
• production readiness
• benchmark superiority
• full OpenAI API compatibility
• production RAG, agent, or cache correctness
• Research Foundry extraction accuracy, retrieval relevance, factuality, or synthesis
• model replacement

See the claim registry and public language guide.

Documentation

• Documentation index
• Solution Protocol v0alpha1
• Solution SDK and Conformance Kit
• Vision: AI Workload Control Layer
• Example catalog
• Packaging: getkora strategy
• Reports and evidence
• Contributing

License

Apache License 2.0. See LICENSE.