This document lists where the project is going and why. Versions are scoped by outcome, not by date. Items move between versions when reality disagrees with the plan; the architecture document records the decisions once they are made. Suggestions are welcome as issues.
The operability release: everything needed to run SibillaOS beyond the first demo. Shipped: llmfit as a Debian package built from the pinned upstream release, Open WebUI as an opt-in container managed by sibilla-webui (container plumbing exercised in CI, the web interface itself still pending a manual pass), HTTPS on the gateway via sibilla-tls (local CA for LAN hostnames, ACME for public ones), multi-model serving through the gateway, and the sibilla-connect kit with ready-to-paste configuration for VS Code (Continue and Cline), aider and any OpenAI-compatible client. Release notes in docs/releases/v0.2.0.md.
The release for people who run SibillaOS on real infrastructure. Release notes in docs/releases/v0.3.0.md.
- A unified
sibillaCLI as the single entry point (sibilla status,sibilla model,sibilla tls,sibilla webui,sibilla connect), with the current commands kept as aliases. Done. sibilla statusgrown into a real health view: engine state, served models, disk usage of the model store, GPU utilization when present, gateway reachability; exits nonzero on failure so scripts can use it as a check. Done.- Observability opt-in:
sibilla metrics enableserves gateway-level Prometheus metrics (request rate, latency histograms, status codes, upstream health) at an authenticated /metrics/gateway endpoint, with a ready-made Grafana dashboard and scrape config in docs/observability/. vLLM's native /metrics passes through the gateway; Ollama at the pinned 0.31.1 exposes no Prometheus endpoint (verified in its source, revisit on engine bumps). Done. - Model store management:
sibilla model rmandsibilla model prune, with disk usage reporting. Done. - A cloud image alongside the ISO: the same stack published as a qcow2 with cloud-init, for Proxmox, libvirt and cloud providers. Built by baking the official Ubuntu cloud image in one QEMU boot and resealing cloud-init; engine and model detection moved into llmd-firstboot so it happens on the deployed hardware. CI deploys the image with a real user seed and gets a chat completion. Done.
- arm64 build: delivered as the cloud image (qcow2 for Ampere, Graviton and other arm64 VMs), built and deployed in CI on native arm64 runners with the llmfit deb repackaged from the aarch64 musl release. GitHub's arm64 runners have no KVM, so the CI deploy runs under TCG emulation: the API surface is asserted, token generation is exercised but not asserted (documented in the test). The arm64 ISO is deferred: the unattended-install test that guards every ISO change is impractical under pure emulation; revisit if arm64 KVM runners appear, and note Raspberry Pi needs its own image flavor anyway (not the generic cloudimg). Done for the cloud image.
The release that makes a security review pleasant. Shipped: multiple gateway API keys with per-key revocation and structured access logs (sibilla key; rate limiting deferred), the signed APT repository on GitHub Pages with installed systems preconfigured for plain apt upgrades, the GPG-signed model catalog with per-quant artifact digests verified after every pull, SBOM generation and CVE scanning with a written triage policy (docs/supply-chain.md), and Secure Boot verified in CI on every ISO. Release notes in docs/releases/v0.4.0.md.
The adoption release: trying SibillaOS no longer costs a reinstall, and walking away is provably clean. Shipped: the zero-reinstall path (apt install llmd plus sibilla setup on an existing Ubuntu 24.04 machine, staying a good guest: no firewall takeover, no source flips, appliance behavior kept behind the image marker), sibilla remove for a verified reversible teardown that takes only what SibillaOS installed and restores what it displaced, sibilla doctor for a paste-ready secret-free diagnostic report (every configured API key scrubbed by construction, asserted in CI), sibilla connect --write to place the Continue configuration with backup, and the README overhaul with the two-command quick start. The whole lifecycle, install through verified-clean removal, runs in CI on every push. Release notes in docs/releases/v0.5.0.md.
The release for three audiences: agent users, the network-less on-premise, and whoever runs more than one machine. Shipped: the MCP server (sibilla mcp enable, the local model as chat and list_models tools behind the gateway keys, stateless Python stdlib), air-gapped installs end to end (digest-gated sibilla model import, the companion payload builder, first-boot detection, proven in CI on a VM with outbound network dropped), configuration as code (declared profile plus idempotent sibilla apply and apply export, picked up by cloud-init and first boot), embeddings for local RAG (/v1/embeddings, an embedding role in the signed catalog that can never become the chat default, sibilla model pull), the developer kit (connect --env/--mcp/--snippet, aider written next to Continue), sibilla bench with a shareable result table, the kernel-level per-IP rate limit baseline (per-key fairness deferred with the Caddy-vs-nginx decision recorded in the architecture log), and the upgrade path turned into a CI assert: the published release is installed from the live repository, upgraded and required to survive on every push. Release notes in docs/releases/v0.6.0.md.
The adoption cycle: found, believed, working on real hardware. Shipped: the front-and-center "why not just Ollama" answer and the CI-recorded demo (generated, not acted); consumer GPU support with AMD/ROCm completion at first boot, loud runtime visibility and a tester guide with a structured issue form; supply freshness (ollama 0.32.1, vLLM v0.25.1 pinned, OLLAMA_NO_CLOUD=1 everywhere, 161 digests covering every catalog entry, with the signing pipeline hardened by the four-failure saga it took); the config audit (sibilla apply check); the RAG quickstart run by CI; the keyless remote workstation kit; and the characterization pass (oracle CLI style, the SSH login banner, Server: SibillaOS, the quiet-corner wallpaper). Release notes in docs/releases/v0.7.0.md.
The cycle in detail:
- Supply freshness, opening the cycle: the ollama pin moves to 0.32.1 and the vLLM container to v0.25.1 (the ":latest" that had survived from the PoC is gone), with the whole pipeline as the safety net; sha256 digests recorded for every entry of the catalog (tools/update-digests.sh); and one finding turned into policy - recent ollama can transparently offload ":cloud" models to ollama.com, so every SibillaOS install now pins OLLAMA_NO_CLOUD=1 in the hardened unit and CI asserts it on both the image and the apt path: nothing in a prompt or an answer leaves the machine, including via the engine's own cloud. Done - the digest recording turned into a four-failure saga (a truncating updater, CRLF in URLs, CRLF under signatures, dot-separated quant names) that left the whole signing pipeline hardened for good: dependency guards, write-then-move, LF pinned for signed files, substance checks at lint and build. 161 digests now cover every entry.
- Consumer GPUs, following the only real inbound signal the project has received (a question about Strix Halo/ROCm). AMD detection already existed in llmd-hw-detect; what was missing and now ships: first boot completes the ollama ROCm runtime on AMD hardware whose GPU was not visible at image bake time (cloud images are baked in a GPU-less VM, so AMD deployments silently ran on CPU), AMD visibility in
sibilla status,doctorandbench(including a loud "ROCm runtime missing" instead of silence), a deterministic hardware-detection matrix in CI (PATH-stubbed nvidia-smi/lspci/rocm-smi: datacenter NVIDIA, consumer NVIDIA, AMD, APU, none), and docs/validation/gpu.md - the tester guide with the explicit call for NVIDIA/AMD/APU hands, HSA_OVERRIDE_GFX_VERSION collection for APUs included. Done; the physical validation now waits on tester hands, which is what the distribution push below recruits. - Positioning and proof: a front-and-center "Why not just Ollama?" section in the README, because it is the question every single visitor silently asks and the answer (mandatory auth from first boot, verified artifacts, reversible trial, off-network, config as code - around the very same engine) deserves better than being deducible; and the terminal demo made authentic - the CI pipeline already runs the real session on every push, so it records it (asciinema) and the artifact replaces the hand-drawn representative SVG. A demo generated by the pipeline, not acted.
- Deeper developer integration, building on the v0.6 kit.
sibilla connect --remotegenerates a self-contained, deliberately keyless script for the developer workstation: it prompts for the key on stdin there (never argv, never shell history), refuses to write a single file until the gateway accepts the key, then places Continue and aider configs with the backup-once rule plus an optional project-directory .env (the scaffolding). The Zed survey landed honestly: Zed takes an api_url override in settings.json but keeps keys in its own credential store, so it gets a print-only kit section rather than a pretend --write. Framework quickstarts shipped with the RAG stage. Done. - Configuration audit, the third piece of the config track:
sibilla apply checkverifies without changing - drift against the declared profile, a Caddyfile that differs from its render (hand edits, which the next toggle would silently overwrite), the served model against its catalog digest, key file permissions - and exits nonzero on findings, so it doubles as a cron or fleet health probe. CI proves it passes on a clean machine and catches a tampered Caddyfile and a drifted profile. Done. - Local RAG quickstart on the embeddings endpoint: a zero-dependency worked example (examples/rag-quickstart.py, ~100 lines of stdlib, environment straight from sibilla connect --env) plus LangChain and LlamaIndex configurations with the local-endpoint gotchas stated. CI runs the documented flow; the hard inference guarantee lives in the pinned VM, where the airgap payload now carries the embedding model too. Done.
The big-model bet. The v0.7 distribution push met silence, recorded here without varnish; the response was not louder posting but a bet on the loudest niche there is: running very large models on hardware you own. Shipped: the large catalog tier (two Apache-2.0 MoE entries, RAM floor refusal, real GPU/CPU placement in status, docs/large-models.md), the registry as the tier's verified second source, the serving guards closed on the import path, the coexistence design for the future third engine recorded, and man pages. Release notes in docs/releases/v0.8.0.md.
One principle bounds this whole cycle: the big-model work is ADDITIVE SUPPORT, never the default. SibillaOS's identity is that first boot serves what the machine honestly runs - so no large model is ever auto-selected, the RAM floor refuses (fail closed, no --force) anything past what physical memory carries, and opting in prints what it will cost before a byte is downloaded. A cycle that made the giant the default would not extend the philosophy, it would break it.
- Big-model mode on the engine we already ship, re-scoped by the spike that opened the cycle: the pinned ollama does layer offload (larger than VRAM) but no expert streaming from disk (an open upstream request), so that is the honest claim. Large models on HF are almost always sharded GGUF, which the hf.co pull path does not support, so the HF-only source policy is extended FOR THE LARGE TIER ONLY to the ollama.com registry, whose manifests carry sha256 digests - the whole verification machinery survives the second source. Shipped as a registry-backed entry type in the signed catalog and the digest updater, large MoE entries, layer-offload guidance surfaced by the tooling, and
sibilla benchas the honest measure of what offload costs. Done. - ds4 (DwarfStar) as the third engine and the owner of the "larger than RAM" story, promoted from exploratory but GATED: the spike found zero releases, tags or binary assets upstream (the original veto morphed into "nothing to pin"). Step one, the upstream issue requesting tagged releases with the Debian packaging offered in exchange, was posted on 2026-08-20 - SibillaOS's first visible ecosystem participation; the integration now waits on the answer. When something pinnable exists, the integration follows the llmfit playbook: repackaged deb, hardened unit, its OpenAI-compatible API behind the same gateway and keys, llmd-hw-detect learning the unified-memory tier (Strix Halo, DGX Spark, 96GB+ Macs). Design decided (architecture changelog v0.13): ds4 COEXISTS with ollama on that tier rather than replacing it - a deliberately single-model engine must not cost the machine its embeddings, small models and RAG - with llama-swap (pinnable zero-dependency Go binary, real release cadence) between the gateway and the engines, routing by the model field and stopping/starting upstreams so the resident model yields RAM to the giant when requested. Every other machine keeps the straight gateway-to-engine path unchanged.
- The tier's completion pass: a second large entry (qwen3:235b, Apache-2.0, ~170 GB floor) so the tier is a choice rather than a single model; docs/large-models.md stating what the tier promises, how to read the placement line, how to measure the cost with bench and when the giant is the wrong call; and the serving guards closed on the import path too -
import --usenow enforces the same embedding refusal and RAM floor asuse, and registry ids no longer double their tag on import. Done. - Man pages, deferred from v0.7: sibilla(1) documents every subcommand, ships gzipped in llmd-hw, and CI both renders it fatally-on-warnings and asserts it inside the deb. Done.
Scope to be set with the lesson of two quiet cycles in hand. What carries over regardless:
- Tell the big-model story: it shipped and it is a genuinely new reason to post, not a repost. The distribution pack this time leads with the large tier and the honest refusal.
- The ds4 answer: if upstream tags a release, the coexistence integration (llama-swap between the gateway and the engines, design recorded in architecture changelog v0.13) becomes this cycle's core; if not, the gate holds and the tier stays "larger than VRAM".
- vLLM on physical datacenter GPUs: the standing v1.0 debt, rented instance if no tester appears.
- Manual validation of Open WebUI (docs/validation/webui.md), carried since v0.2: one real machine, thirty minutes, a filled result record.
Tracked here so they are not lost between feature cycles, because each is a v1.0 criterion that no amount of new code satisfies:
- vLLM on physical datacenter GPUs: implemented and gated in CI, but never run on real hardware. Needs a machine or an external tester.
- Open WebUI web interface: packaged and CI-exercised around the container, the login-and-chat flow never validated by hand (folded into v0.7 above; checklist in docs/validation/webui.md).
- A release cycle with external users and no critical install bugs: only starts counting once people install the published images.
A dedicated track rather than a single milestone, because "your data stays on your machine" is the core promise and it has to hold at every release. What already holds today: engines bound to loopback with the gateway as the only entry point, a mandatory bearer token, systemd sandboxing on the Ollama unit, pinned engine versions, the upstream Host header rewritten by the proxy, and the Ubuntu base image verified against official checksums at build time.
Planned, in order of appearance:
- SECURITY.md with a private disclosure channel and a supported-versions table. Done in v0.2.
- Default firewall profile: ufw enabled at first boot with only SSH and the gateway port open; Open WebUI listens on all interfaces (host networking), so its port stays closed until the user opens it deliberately, and
sibilla tls --acmeopens 80/443 itself. Done in v0.3. - Sandboxing extended to every llmd unit and the containers (NoNewPrivileges, kernel and realtime restrictions), with a CI check that asserts the directives on the installed system; capability drops on the containers deferred until their runtime can be exercised in CI. Done in v0.3.
- Automatic security updates: unattended-upgrades enabled by default for the security pocket, since an appliance that nobody patches must patch itself. Done in v0.3.
- API key lifecycle:
sibilla keywith add, revoke and rotate, multiple keys folded into the gateway matcher, structured JSON access logs. Done in v0.4 (rate limiting re-scoped in v0.6: see there for the packaging constraint). - Supply chain: signed catalog, SBOM in SPDX and CycloneDX attached to releases, CVE scanning of the actually installed system with a written triage policy and expiring exceptions. Done in v0.4.
- Secure Boot: the repack keeps Ubuntu's signed shim and GRUB, and CI now boots every ISO under OVMF with Secure Boot enforced and Microsoft keys enrolled, asserting that the kernel itself reports Secure Boot active; validation on physical firmware from external users remains welcome. Done in v0.4.
- Model integrity: catalog entries carry per-quant sha256 digests (the ollama blob name after a pull);
sibilla model usefails closed on a mismatch, first boot warns loudly. Done in v0.4.
The release where the "proof of concept" label comes off. Criteria, not features: the vLLM path validated on physical datacenter GPUs, at least one release cycle with external users and no critical install bugs, documented upgrade path between versions, and the desktop variant with a model selection screen in the installer (Calamares module or first-boot console menu, decision pending).
Ideas that look promising but need a use case or a champion:
- Speech endpoints: whisper.cpp for transcription behind the same gateway.
- ds4 (DwarfStar): promoted in v0.8, then gated on upstream having anything to pin; the upstream issue is posted and the coexistence design recorded. Tracked in v0.9 above, kept here only as the category's origin.
- kimi-k3-in-c (2026 survey): Kimi K3, 2.78T parameters from a 1.56 TB checkpoint, in 8.24 GB of RAM - portable C99, Apache-2.0, CI and byte-identical determinism, 5.7k stars. Second viral single-model disk-streaming project after ds4, which proves the category's pull; but CLI-only (no server, no API), 26.5 s/token on 8 GB, base model without a chat template, no confirmable tagged release. Not an engine candidate; watch as the category's proof of demand.
- KTransformers as an alternative big-model backend: CPU/GPU hybrid MoE inference, 100B+ models on a single consumer GPU, SOSP-published. Powerful but dependency-heavy relative to this project's pinning discipline; watch, do not integrate yet.
- exo and distributed-llama for multi-machine inference: model sharding across the devices of a household or office LAN. A different product story (many machines, one model) that would pair naturally with the fleet/profile work; needs a champion and a use case.
- ik_llama.cpp, the llama.cpp fork with state-of-the-art quants and performance work: better quants mean more model in the same RAM, which is the big-model bet attacked from the compression side. Watch its stability and whether its gains land upstream.
- BitNet (Microsoft's 1-bit LLM framework): the "large capability on small hardware" problem attacked from the opposite end. Model availability is the constraint; watch.
- LDAP or OIDC authentication on the gateway for team deployments.
- A Debian stable base variant for shops that prefer it over Ubuntu.
Things SibillaOS deliberately does not try to be: a Kubernetes distribution (use the engines' own charts), a model marketplace (the catalog stays small and curated), or a managed service.