Skip to content

Add Kubernetes dev tooling (kubectl, kind, helm) + a compose provider #62

Description

@repentsinner

Problem

Projects on this image that target Kubernetes (e.g. a k3s-class cluster)
need local K8s tooling on the dev host, but tilefin-nvidia-open doesn't
ship any. Right now it gets installed ad-hoc into ~/.local
(imperative, per-machine, not reproducible) — which defeats the
immutable-image model the whole platform is built on.

Relatedly, podman compose has no provider on the image. podman compose is a thin shim that execs an external provider (docker-compose
or podman-compose); with neither installed, every compose workflow fails
out of the box with:

Error: looking up compose provider failed
  * exec: "docker-compose": executable file not found in $PATH
  * exec: "podman-compose": executable file not found in $PATH

Proposed

Bake the client tooling + enabling config into the image
(declarative, reproducible). Running clusters stay ephemeral or on
dedicated nodes — only the tools belong here.

  • kubectl
  • kind — ephemeral local K8s clusters on podman
  • helm
  • a compose provider — prefer docker-compose v2 (the Go binary);
    podman auto-prefers it over podman-compose, and it implements Compose
    semantics faithfully (e.g. stop <service> stops all replicas;
    podman-compose only stops one)
  • rootless-kind enabling config: cgroup v2 delegation,
    KIND_EXPERIMENTAL_PROVIDER=podman, and any sysctls kind needs to run
    rootless on podman

Why the image, not per-project

Host dev tooling is environment, not project. On an immutable
rpm-ostree host, it should be baked into the image, not
uv tool install'd / curl | sh'd per machine. Running clusters are the
opposite — ephemeral (kind, on demand) or on dedicated cluster nodes
(k3s); never baked-in image state.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions