Skip to content

Repository files navigation

Helix Ops — Workload Orchestrator

A Kubernetes-lite control plane simulator. Declarative DSL → scheduler → rolling-update controller → health checks → multiplexed log streaming → TSDB-lite → AES-encrypted secrets vault → autoscaling → multi-tenant. Built so an SRE can experience the full control-loop without spinning up an actual cluster — perfect for demos, integration training, chaos experiments and "what would the reconciler do" thought experiments.


What's inside

Declarative DSL

A YAML document modelled after the shape every Kubernetes operator recognises:

apiVersion: helix/v1
kind: Workload
metadata:
  name: api-gateway
  namespace: prod
  labels:
    app: edge
spec:
  image: ghcr.io/aurora/api:1.4.2
  kind: Service                              # Service | Job | StatefulSet
  replicas: 3
  strategy: { type: rolling, maxSurge: 1, maxUnavailable: 0 }
  resources: { cpu: 500m, memory: 256Mi }
  nodeSelector: { zone: a }                  # every label must match
  env:
    - { name: NODE_ENV, value: production }
    - { name: DB_URL, valueFrom: { secret: pg-creds, key: url } }
  ports:
    - { containerPort: 8080, name: http }
  health:
    liveness:  { httpPath: /healthz, intervalSec: 10, failureThreshold: 3 }
    readiness: { httpPath: /ready,   intervalSec: 5,  failureThreshold: 2 }

POST /api/workloads parses it, validates every field, persists the spec, creates revision #1, and the reconciler immediately starts scheduling pods. A PATCH with a new spec bumps the generation and the rolling controller takes over.

Control plane

Subsystem File Responsibility
DSL parser lib/dsl.ts YAML → typed spec + structured validation errors
Scheduler lib/scheduler.ts Best-fit-decreasing bin-packing, node selectors, tenant quota
Reconciler lib/reconciler.ts The control loop — ticks every HELIX_TICK_MS
Rolling lib/rolling.ts maxSurge / maxUnavailable rolling-update strategy
Runtime lib/runtime.ts Pod state machine: Pending → Scheduled → Running → …
Autoscaler lib/autoscaler.ts HPA-style with cooldown + ±10% deadband
Vault lib/vault.ts AES-256-GCM secrets with IV + auth tag per row
TSDB-lite lib/tsdb.ts Append-only metric points + SQL-side bucketing
Log stream lib/log-stream.ts Multiplexed SSE fanout of per-pod log lines
Event bus lib/events.ts Process-wide pub/sub, survives Next.js hot reload

UI — the Control Room

A dark-first ops console designed for operators who stare at it for hours:

  • Mission Control — KPI rail, cluster pod-ready trend, per-node CPU/MEM stripe gauges, top loaded workloads, live cluster event feed.
  • Workloads — filterable table → detail page with 7 tabs (Overview, Pods, Revisions, Events, Metrics, Logs, YAML). The Logs tab is a live multiplexed text/event-stream of every pod's stdout/stderr.
  • Nodes — capacity tiles, cordon / uncordon / mark-down actions.
  • Tenants — admin-only quota view with bar gauges.
  • Secrets — vault listing. Key names visible, values sealed with AES-256-GCM; reveal is audit-logged.
  • Autoscaling — live editor for min/max/target/cooldown across all rules.
  • Event log — full cluster-wide SSE stream with pause/resume.
  • Settings — identity + tenant info.

Theme: "Control Room" — carbon background (#0a0d12), dotted-grid backdrop, signal-green for healthy, amber for pending, magenta for failed, ice-blue for informational accents, JetBrains Mono for every numeric.


Quick start

git clone https://github.com/vugarfamiloglu/Helix-Ops.git
cd Helix-Ops

npm install
cp .env.example .env.local
# Generate the two required secrets:
node -e "console.log('HELIX_JWT_SECRET=' + require('crypto').randomBytes(32).toString('hex'))"
node -e "console.log('HELIX_VAULT_KEY='  + require('crypto').randomBytes(32).toString('hex'))"
# (paste those into .env.local)

npm run seed          # 2 tenants, 6 nodes, 10 workloads + history
npm run dev           # http://localhost:5080

Sign in with:

Email Password Role
admin@helix.local ChangeMe1! admin
ops@aurora.co ChangeMe1! operator
viewer@lumen.labs ChangeMe1! viewer

The reconciler tick auto-starts on the first authenticated request and immediately begins scheduling pods, recording metrics, and emitting events. Refresh the Mission Control page after ~10 seconds and you'll see the pod-ready KPI climb.


Tech stack

  • Next.js 15 App Router + React 19 RC + TypeScript strict
  • Tailwind CSS — custom "Control Room" theme (no framework presets)
  • better-sqlite3 — single file, WAL, FK, numbered migrations
  • bcryptjs + jose — JWT-cookie sessions (7-day rotation)
  • yaml — DSL parser
  • node:crypto — AES-256-GCM vault
  • SSE — multiplexed log fanout + cluster event stream (defensive teardown — no "Controller is already closed" cascades)

Architecture

                        ┌──────────────────────────────────┐
                        │   Browser (Next.js + React 19)   │
                        │     /dashboard /workloads /…     │
                        └─────────────┬────────────────────┘
                                      │ HTTPS + SSE
                                      ▼
┌──────────────────────────────────────────────────────────────┐
│   Next.js API routes   (app/api/*)                           │
│   /workloads · /pods · /nodes · /secrets · /metrics …        │
└─┬───────────────┬──────────────────┬───────────────┬─────────┘
  │               │                  │               │
  ▼               ▼                  ▼               ▼
┌──────────┐  ┌──────────┐      ┌───────────┐  ┌──────────┐
│  Auth    │  │  Vault   │      │  DSL      │  │  TSDB    │
│  bcrypt  │  │ AES-GCM  │      │  YAML     │  │  metrics │
│  + jose  │  │          │      │  parser   │  │  table   │
└──────────┘  └──────────┘      └───────────┘  └──────────┘
                                          │
                                          ▼
                ┌────────────────────────────────────────┐
                │           Reconciler (lib/)            │
                │  • setInterval(HELIX_TICK_MS)          │
                │  • diff desired vs actual              │
                │  • scheduler → mark pods Scheduled     │
                │  • rolling controller advances rev     │
                │  • autoscaler ticks                    │
                │  • per-tick metric points              │
                └───────────────┬────────────────────────┘
                                │ emits to bus
                                ▼
                ┌────────────────────────────────────────┐
                │   In-process pub/sub bus (EventEmitter)│
                └───┬──────────────┬─────────────────────┘
                    │              │
                    ▼              ▼
              SSE: events    SSE: pod_log multiplex
              /api/events    /api/workloads/[id]/logs

Everything runs in a single Node.js process — same one that hosts the Next.js dev server. No queue, no extra worker, no Redis. When you'd swap this for production you'd move the reconciler to its own process and push the bus over Redis Streams; the route handlers wouldn't change.


Pod state machine

   Pending                                   ┌─────► Succeeded
      │ scheduler.schedulePod()              │  (Job/manual stop)
      ▼                                      │
   Scheduled  ── image-pull simulation ──►  Running, ready=0
                                              │
              readiness probe passes ─────►  Running, ready=1
                                              │
              liveness probe fails ───────►  restart_count++ (stays Running)
                                              │
              terminate() called   ───────►  Terminating ── grace ─► Succeeded
                                              │
              fail()         called   ─────► Failed

The reconciler counts Pending + Scheduled + Running toward replicas and ignores Succeeded / Failed. Rolling updates increment the generation and the rolling controller surges new-rev pods and drains old-rev pods up to maxSurge / maxUnavailable budgets per tick.


Scheduler algorithm (the gist)

For each Pending pod:

  1. Tenant quota — would creating this pod push the tenant past tenants.cpu_quota_milli or tenants.mem_quota_bytes? If yes, stay Pending (event: FailedScheduling).
  2. Node filter — drop Cordoned / Down nodes; drop nodes that don't match every nodeSelector key/value.
  3. Capacity filter — node free CPU ≥ pod request AND node free memory ≥ pod request.
  4. Best-fit — pick the candidate whose remaining headroom (CPU first, then memory) is the smallest non-negative number. Packs tightly.

If no node qualifies, the pod stays Pending — same shape kube-scheduler uses. Next tick re-tries: maybe an old pod terminated, or an autoscaler shrank something, freeing room.


TSDB-lite

metrics is an append-only column with an index on (scope, scope_id, metric, ts). The query API does the bucketing server-side with strftime('%s', ts) / bucket_seconds * bucket_seconds, which means the JS layer never touches the raw rows. Retention is 30 days (the reconciler prunes anything older on every tick).

The autoscaler is the heaviest reader — for each enabled rule it asks for the 5-minute average of the configured metric and adjusts replicas by ±1.


API reference

Endpoint Auth Notes
POST /api/auth/login Sets helix_session cookie
POST /api/auth/logout
GET /api/auth/me Used by the AppShell auth gate
GET /api/dashboard session KPIs + node load + top workloads
GET /api/workloads session tenant-scoped (admin = all)
POST /api/workloads op / admin body: { dsl, tenant_id? }
GET /api/workloads/:id session + pods + revisions + events
PATCH /api/workloads/:id op / admin { dsl } (rollout) or { replicas }
DELETE /api/workloads/:id op / admin
POST /api/workloads/:id/restart op / admin cycles every live pod
GET /api/workloads/:id/logs (SSE) session multiplexed pod logs
GET /api/pods/:id session
DELETE /api/pods/:id op / admin terminate one pod
GET /api/nodes session
PATCH /api/nodes/:id admin cordon / uncordon / mark down
GET /api/tenants session
POST /api/tenants admin
GET /api/secrets session metadata only
POST /api/secrets op / admin seals via vault
GET /api/secrets/:id?reveal=1 op / admin audit-logged
DELETE /api/secrets/:id op / admin
GET /api/autoscalers session
POST /api/autoscalers op / admin upsert
DELETE /api/autoscalers/:id op / admin
GET /api/metrics?scope&id&metric&… session range query, downsampled
GET /api/events (SSE) session cluster-wide event stream
POST /api/dsl/parse session validate without persisting

File layout

helix-ops/
├── app/
│   ├── (root pages)         dashboard, workloads, nodes, tenants, secrets,
│   │                        autoscaling, events, settings, login
│   ├── workloads/[id]/      detail + 7-tab client
│   ├── nodes/[id]/          detail
│   └── api/                 REST + SSE handlers
├── components/
│   ├── AppShell.tsx         sidebar + topbar + auth gate
│   ├── PasswordInput.tsx    reusable, eye toggle (ALL password inputs use this)
│   ├── Modal, ConfirmModal, PromptModal
│   ├── Brand, Logo, PhaseDot, BarGauge, Sparkline
├── lib/
│   ├── db.ts                schema + migrations + audit()
│   ├── types.ts             full TS vocabulary
│   ├── auth.ts              bcrypt + JWT cookie
│   ├── dsl.ts               YAML parser + canonical render
│   ├── vault.ts             AES-256-GCM
│   ├── events.ts            in-process pub/sub
│   ├── tsdb.ts              metrics range + downsample
│   ├── log-stream.ts        multiplexed log fanout
│   ├── scheduler.ts         bin-packing scheduler
│   ├── runtime.ts           pod state machine simulator
│   ├── rolling.ts           rolling-update controller
│   ├── autoscaler.ts        HPA-style autoscaler
│   ├── reconciler.ts        the control loop
│   ├── api-helpers.ts
│   └── format.ts
├── scripts/seed.ts          admin + tenants + nodes + workloads + history
└── data/helix.db            created on first boot

What's intentionally NOT here

  • A real container runtime — pods don't actually run anything; the runtime simulates start-up, readiness, log emission, and the occasional liveness failure with setTimeout and random noise. Swap lib/runtime.ts for dockerode / containerd / k8s.io/client-go bindings to make it real.
  • etcd / CRDs / RBAC graph — multi-tenant is enforced by tenant_id columns and the session role. Good enough for a simulator, not for production.
  • A queue — every state change is a synchronous SQLite write inside the same Node process. Add a queue if you want multi-host scheduling.

License

MIT.

About

Kubernetes-lite workload orchestrator simulator — declarative DSL, scheduler, rolling-update controller, multiplexed logs, TSDB-lite, AES-encrypted secrets vault, autoscaling, multi-tenant.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages