AI Infrastructure Control Plane

Kubernetes-native reference control plane for governed private AI inference.
Part of the AI Infrastructure OS architecture with the AI Runtime Platform execution plane.

Detect → Decide → Approve → GitOps Proposal → Verify

Tier	What you get
Supported	/governance/evaluate, durable decisions/approvals (Postgres HA / SQLite single-node), tenant isolation + JWT-only tenant, RBAC roles, quota onUnavailable, signed OCI Helm/images, capability contracts registry, durable PolicyBundle generations + OCI load/signature verify, optional GitHub draft-PR provider, runtime verification contract, capability-bound approvals
Reference	Argo sync status not_checked in verify, CRDs without in-tree controller, live Redis/Prometheus demo integrations
Experimental	Forecasting sims, FinOps CSV helpers, intent heuristics

Current release line: v2.4.0.
Recommended pair: Control Plane v2.4.x + Runtime v2.3.x
(Runtime compatibility matrix).
Upgrade: v1 → v2.0 · maturity: maturity-boundary.md.

Run in 2 minutes

make platform-demo && make platform-demo-verify
# production overlay: Redis + Prometheus
make platform-demo-production && make platform-demo-production-verify
# enterprise overlay: + Keycloak OIDC
make platform-demo-enterprise && make platform-demo-enterprise-verify

Golden path:

request → versioned PolicyBundle evaluate → decision_id
       → allow | block | approval_required (+ approval_id)
       → human approve/reject → retry with x-ai-approval-id → allow

Docs: durable governance · remediation · RBAC · capability contracts · roadmap · architecture · demo

Operator Dashboard

The control API serves a live operator dashboard at / with platform status, topology health, model inventory, a governance playground, and inventory drift detection, refreshed every few seconds.

Governance Playground

Interactive demo | Click the animated preview to watch the governance playground and drift detection walkthrough.

Submit an AI platform request through the live governance pipeline without leaving the browser:

• POST /governance/evaluate — returns decision_id, policy_bundle_id, and approval_id when needed
• GET/POST /approvals/* — durable human approval lifecycle
• Presets on / for low-risk dev, production external, and budget-exceeded scenarios

Inventory Drift Detection

Compare configured model inventory (MODEL_INVENTORY_PATH / Helm ConfigMap) against live Ollama and vLLM probes:

• GET /drift — desired vs actual models per backend, missing/unexpected lists
• Prometheus gauges: ai_control_inventory_in_sync, ai_control_inventory_drift

How the Projects Fit Together

This repository is the Control Plane of the AI Infrastructure OS. The Execution Plane lives in ai-runtime-platform.

Layer	Role in AI Infrastructure OS	Repository
Execution Plane	Runs inference, routes traffic, enforces governance verdicts	justrunme/ai-runtime-platform
Control Plane	Policy, cost, topology, drift, SLO, fleet operations	justrunme/ai-infra-control-plane

The Execution Plane runs workloads. The Control Plane evaluates policy and the runtime enforces verdicts at the inference boundary via CONTROL_PLANE_URL.

Product Walkthroughs

Governance Decision Flow

AI Infrastructure Digital Twin

Forecast-driven Scaling

Architecture (short)

Control Plane decides; Execution Plane enforces. Detailed diagrams live in docs/platform-architecture.md.

Client → Runtime gateway → POST /governance/evaluate → verdict
                              ↑ PolicyBundle + durable decision store

Scope

• Production path: governed evaluate with decision_id / policy_bundle_id, durable approvals, probes, drift, Helm, CI supply chain.
• Integrated demo: Redis quota, Prometheus inputs, Keycloak OIDC, gateway enforcement.
• Prototype / reference: TimesFM forecasting, FinOps CSV heuristics, static multi-cluster fleet, OTel sample emitter.

Repository Layout

apps/
  control-api/        FastAPI service with health and model status endpoints
infra/
  helm/               Kubernetes packaging
  terraform/          Cloud bootstrap modules
    k3s-bootstrap/    Example Hetzner VM bootstrap with cloud-init and k3s
observability/
  grafana/            Dashboards and metrics notes
  loki/               Loki and Promtail logging examples
  otel-genai/         OpenTelemetry GenAI telemetry reference implementation
forecasting/
  timesfm/            Experimental capacity forecasting module
experiments/
  inference-autoscaling/ Forecast-driven inference scaling recommendations
governance/
  agents/             Agent-to-model/tool/policy bindings
  cost/               AI cost governance policy engine
  intent/             Natural-language intent to governed execution plan
  policy-packs/       Environment and team-specific policy overlays
  prompt-security/    PII, secret, and prompt-injection checks
  quota/              Tenant quota policy checks
  registry/           Signed model registry and attestation metadata
  risk/               AI request risk scoring engine
  approval/           Human approval workflow reference implementation
  pipeline/           End-to-end AI governance decision pipeline
  tools/              MCP tool registry and action allowlists
security/
  trivy/              Container and IaC scan configuration
  opa/                Kubernetes policy gates for rendered manifests
docs/
  architecture.md     System design notes
  case-study.md       Portfolio case study and demo flow
  digital-twin.md     AI infrastructure topology model
  platform-architecture.md Technical platform architecture

Local Development

cd apps/control-api
python3.12 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
uvicorn app.main:app --reload

Run tests:

make venv
make test

The project targets Python 3.12 for local development and CI.

Run the portfolio demo:

make demo

The demo prints the key control API endpoints and runs the end-to-end governance pipeline from governance/pipeline/sample_requests.csv.

Control API

The control API exposes operator-facing signals for private AI infrastructure:

• GET / - live operator dashboard with governance playground and drift panel (HTML).
• GET /livez - process liveness (no store dependency).
• GET /readyz - readiness (decision store ping + policy bundle).
• GET /health - operator-facing status (ok / degraded with store flags).
• GET /healthz - liveness alias for older probes and demos.
• GET /models - configured model backends and status.
• GET /metrics - Prometheus-compatible text metrics.
• GET /capacity - aggregate model serving capacity.
• GET /cost - estimated hourly, daily, and monthly cost.
• GET /summary - compact status for dashboards and demos.
• GET /governance/inputs/status - Redis quota and Prometheus input status.
• POST /intent/resolve - intent-to-agent/model/tool/region execution planning.
• POST /governance/evaluate-tool - governed MCP tool call decision.
• POST /governance/evaluate-response - post-response quality, latency, and cost evaluation.

Model Inventory

The model inventory is configuration-driven. By default the API loads app/model_inventory.json shipped with the image, but you can point it at any JSON file:

export MODEL_INVENTORY_PATH=/etc/ai-control-plane/model_inventory.json

The file is a JSON array of model entries; see apps/control-api/examples/model_inventory.sample.json for a multi-backend example. If the file is missing or malformed, the API falls back to a built-in inventory so the control plane stays observable.

Ollama Backend Probe

Set OLLAMA_BASE_URL to point the control API at an Ollama backend:

export OLLAMA_BASE_URL=http://localhost:11434

The API exposes:

• GET /backends/ollama/health - backend reachability and status.
• GET /backends/ollama/models - model names returned by Ollama /api/tags.
• GET /backends/ollama/latency - lightweight latency measurement for /api/tags.

vLLM Backend Probe

Set VLLM_BASE_URL to point the control API at a vLLM OpenAI-compatible server:

export VLLM_BASE_URL=http://localhost:8000

The API exposes:

• GET /backends/vllm/health - backend reachability and status.
• GET /backends/vllm/models - model ids returned by vLLM /v1/models.
• GET /backends/vllm/latency - lightweight latency measurement for /v1/models.

Prometheus Metrics

GET /metrics exposes Prometheus-compatible metrics for request traffic, backend health, model inventory, capacity, and estimated cost.

Core metrics:

• ai_control_http_requests_total
• ai_control_http_request_latency_ms
• ai_control_backend_up
• ai_control_backend_latency_ms
• ai_control_model_available
• ai_control_capacity_available
• ai_control_estimated_hourly_cost_usd

AI Infrastructure Digital Twin

GET /topology exposes a live platform graph for private AI infrastructure components, dependencies, health, telemetry, and operational signals. The Ollama and vLLM nodes reflect live backend probe results (healthy/degraded plus measured latency). See docs/digital-twin.md.

GET /drift compares configured model inventory against live Ollama and vLLM probes and reports missing or unexpected models per backend.

POST /governance/evaluate runs a single AI request through workload identity, policy packs, prompt security, tenant quota, model registry, cost, risk, approval, sovereign AI, and live telemetry checks, then returns the final verdict for the operator dashboard playground and runtime enforcement path.

AI Cost Governance

governance/cost evaluates model usage, team budgets, token spend, and forecasted monthly cost into allow, warn, or block decisions.

AI Approval Workflow

governance/approval evaluates high-risk AI platform requests into allow, approval_required, or block decisions for human approval gates.

AI Governance Layer

governance/cost, governance/risk, and governance/approval model cost control, risk scoring, and human approval gates for private AI infrastructure.

governance/pipeline connects those signals into an end-to-end decision flow: request telemetry, cost decision, risk score, approval decision, and final verdict.

Run the demo pipeline:

python3.12 governance/pipeline/run_pipeline.py \
  --requests governance/pipeline/sample_requests.csv

Container Images

Every merge to main builds and pushes the control API image to GitHub Container Registry:

ghcr.io/justrunme/ai-infra-control-plane:latest
ghcr.io/justrunme/ai-infra-control-plane:<git-sha>

Images are signed with cosign and accompanied by an SPDX SBOM artifact from the release workflow. Tag releases with v* (for example v1.0.0) to publish semver tags. Current release line: v2.4.0.

Kubernetes Deployment

From the published OCI chart (preferred):

helm upgrade --install ai-control-plane \
  oci://ghcr.io/justrunme/charts/ai-control-plane \
  --version 2.4.0 \
  --set image.repository=ghcr.io/justrunme/ai-infra-control-plane \
  --set image.tag=2.4.0

Or from a git checkout:

helm upgrade --install ai-control-plane infra/helm/ai-control-plane \
  --set image.repository=ghcr.io/justrunme/ai-infra-control-plane \
  --set image.tag=2.4.0

The chart ships security-hardened single-node defaults. Use values-production.yaml for PostgreSQL HA, HPA, PDB, JWT fail-closed, NetworkPolicy and monitoring resources. See infra/helm/ai-control-plane/README.md.

Portfolio Docs

• docs/case-study.md explains the problem, architecture, capabilities, governance pipeline, observability, forecasting, GitOps, security, and demo flow.
• docs/platform-architecture.md describes the system boundary, logical layers, control API, governance architecture, delivery path, and extension points.

Project status

The v2.4 control-plane contract is feature-complete for the declared Supported scope. Pair with AI Runtime Platform v2.3.x for the closed loop. Further work is maintenance-oriented: compatibility testing, security updates, dependency upgrades and external deployment evidence.

Honest limitations and deferred items: docs/maturity-boundary.md, docs/backlog.md.

bash demo/e2e/kind-e2e.sh   # Helm on kind: allow/block/approval + PVC restart