feat(k8s): production parity — workers, MinIO, pull secrets, CI render gate - #169
Merged
Conversation
…r gate The manifests deployed the four HTTP services but none of the machinery behind them. In a cluster, async analysis runs, scheduled tasks, and process-control job execution would be accepted by the APIs and never executed; attachments/exports would have nowhere to go; and no pod could pull the (private) images at all. - workers.yaml: celery-worker (analysis queues, 2 replicas), celery-beat (1 replica, Recreate — two beats double-fire), pc-worker (ion/rtp queues). Compose-parity env incl. broker on redis DB 2; non-root, read-only rootfs, celery inspect ping probes; beat schedules to /tmp (rootfs is read-only). - minio.yaml: standalone StatefulSet (20Gi PVC) + Service + idempotent minio-init Job (bucket + `mc ilm import` of the canonical lifecycle policy from infra/minio/lifecycle.json). - OBJECT_STORE_*/AWS_* env added to analysis + lims deployments (compose parity); minio-credentials via secretGenerator + template. - serviceaccounts.yaml: all four SAs reference ghcr-pull-secret; template documents the one-time per-cluster provisioning command. - cd.yml: deploy step pins the worker deployments to the released tag (they share the analysis/process-control images) and waits on celery-worker rollout. - ci.yml: new k8s-manifests job renders both overlays with `kubectl kustomize`, wired into the CI Status aggregate — the deploy job's "manifests validated in CI" notice is now actually true. Proof: both overlays render clean locally; all five new workloads (celery-worker, celery-beat, pc-worker, minio, minio-init) present in rendered output; secrets guardrail passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The k8s manifests deployed the four HTTP services but none of the machinery behind them. In a real cluster:
What
workers.yamlcelery-worker(analysis queues, 2 replicas),celery-beat(1 replica,Recreate— two beats double-fire every periodic task),pc-worker(ion/rtp queues). Compose-parity env including broker on redis DB 2; non-root, read-only rootfs,celery inspect pingliveness; beat writes its schedule to /tmp.minio.yamlminio-initJob — bucket create +mc ilm importof the canonical lifecycle policy (telemetry 90d, exports 7d, multipart-abort 1d). Comment documents the managed-S3 alternative.OBJECT_STORE_*/AWS_*added to analysis + lims (compose parity);minio-credentialssecretGenerator + template.ghcr-pull-secret; the template documents the one-timekubectl create secret docker-registrycommand per cluster.cd.ymlci.ymlk8s-manifestsjob renders both overlays withkubectl kustomize, wired into the CI Status aggregate — the deploy job's "manifests validated in CI" notice is now actually true.Proof
After this merges, no code stands between the repo and a live deploy — the remaining steps are provisioning: cluster, domain/TLS, real secret values, and the
KUBE_CONFIGGitHub secret.🤖 Generated with Claude Code