diff --git a/README.md b/README.md index c60b10d..c2dfec9 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ but for the deploy step: S2I builds your image, and Deployah runs your release. - [Platform file](#platform-file) - [Profiles](#profiles) - [Health checks](#health-checks) +- [Custom manifests and CRDs](#custom-manifests-and-crds) - [Commands](#commands) - [Environments and variables](#environments-and-variables) - [Precedence rules](#precedence-rules) @@ -828,6 +829,133 @@ components: alive: false ``` +## Custom manifests and CRDs + +Deployah can ship raw Kubernetes YAML next to the generated Helm chart. Use +this for resources Deployah does not generate (for example a `PrometheusRule`, +a `NetworkPolicy`, or a CRD your app needs). + +Extra **manifests** join the same Helm release as your generated resources +(via a Helm post-renderer). Extra **CRDs** are applied to the cluster first, +outside the release, then Deployah waits for each CRD to become +`Established` before installing or upgrading the chart. + +### Layout + +Place files under `.deployah/` next to your `deployah.yaml`. `deployah init` +creates `.deployah/manifests/` and `.deployah/crds/` with short README files: + +```text +.deployah/ + manifests/ + common-networkpolicy.yaml # every environment + prod/ + extra-ingress.yaml # only when deploying to prod (or prod/*) + crds/ + my-crd.yaml # shared; no per-environment subdirs +``` + +Rules: + +- Only `*.yaml` / `*.yml` are loaded. `README*` and markdown files are skipped. + Dotfiles (including `.old.yaml`) are skipped. Any other visible non-YAML file + is an error. +- Subdirectories under `manifests/` must be declared environment keys. A + subdirectory named `review` also applies when you deploy `review/pr-123`. +- Unknown directories under `manifests/` fail the deploy. +- Nested directories under a manifests env dir (or under `crds/`) are not + allowed. +- `CustomResourceDefinition` belongs in `.deployah/crds/`, not under + `manifests/`. CRDs must use `apiVersion: apiextensions.k8s.io/v1`. + +### Literal YAML + +Extra manifests are applied **literally**. There is no Helm templating, no +Sprig, and no environment-variable substitution. Content such as +`{{ $labels.instance }}` in a PrometheusRule is left untouched. + +### Example + +```yaml +# .deployah/manifests/web-alerts.yaml +apiVersion: monitoring.coreos.com/v1 +kind: PrometheusRule +metadata: + name: web-alerts +spec: + groups: + - name: web + rules: + - alert: WebDown + expr: up == 0 + annotations: + summary: instance {{ $labels.instance }} is down +``` + +```sh +deployah plan prod # extras appear in the plan diff +deployah deploy prod -y # CRDs (if any) first, then the release +``` + +### Labels and annotations + +Deployah merges identity metadata into `metadata.labels` / +`metadata.annotations` only (never into selectors or pod templates), and never +rewrites object names: + +| Object | Labels | Annotations | +|---|---|---| +| Generated (from the spec) | `deployah.dev/project`, `deployah.dev/environment`, `deployah.dev/component`, ... | `deployah.dev/source=spec`, `deployah.dev/project` | +| Extra manifests | `deployah.dev/project`, `deployah.dev/environment` | `deployah.dev/source=manifests`, `deployah.dev/project` | +| Extra CRDs | project only (no environment) | `deployah.dev/source=crds`, `deployah.dev/project` | + +Reserved `deployah.dev/*` keys that Deployah does not own are stripped from +extras so they cannot impersonate managed metadata. Your other labels and +annotations are kept. + +Empty `metadata.namespace` on namespaced extras is filled with the release +namespace. A different namespace is an error. Cluster-scoped objects must omit +namespace. + +### Validation and collisions + +- Each document needs `apiVersion`, `kind`, and `metadata.name`. +- Duplicate identities (same apiVersion/kind/namespace/name) across extras + files fail the load. +- An extra that collides with a generated chart object fails the render + (Deployah will not overwrite chart resources). +- Custom resource kinds must be known: put their CRD under `.deployah/crds/`, + or have the type installed on the cluster. A small offline allowlist covers + common operator APIs (cert-manager and prometheus-operator). With + `deployah plan --offline`, unknown kinds are allowed so you can still + preview; scope defaults to namespaced unless an in-repo CRD says otherwise. + +### Plan vs deploy + +- `deployah plan` includes extra manifests in the rendered diff. It does not + apply CRDs; when `.deployah/crds/` is non-empty it prints how many CRDs are + pending. +- `deployah deploy` applies CRDs first (see below), then the Helm release + with extras attached. + +### CRD policy + +```sh +deployah deploy prod # --crds create (default): install if missing +deployah deploy prod --crds create-replace +``` + +| Policy | Behavior | +|---|---| +| `create` (default) | Create the CRD when it is missing; leave an existing CRD unchanged. | +| `create-replace` | Create when missing, or server-side-apply over an existing CRD (force ownership). | + +Deployah waits for each CRD to report `Established` (bounded by `--timeout`), +then applies the Helm release. If the Helm plan has no changes but +`.deployah/crds/` is non-empty, Deployah still applies those CRDs (the chart is +left alone). CRDs are never pruned and are never deleted on uninstall. Extra +manifests leave with the release. + ## Commands Run `deployah --help` for the full details of any command. A complete, @@ -855,13 +983,13 @@ These work with every command: | Command | What it does | |---|---| -| `deployah init` | Create a new spec and platform file by answering a few questions. Use `-o` to set the output file, `--force` to overwrite an existing one, or `--dry-run` to preview. Non-interactive: `--project`, `--environments`, `--set key=value`, or `--defaults` to skip every prompt. | +| `deployah init` | Create a new spec and platform file by answering a few questions. Also scaffolds `.deployah/manifests/` and `.deployah/crds/` for [custom manifests and CRDs](#custom-manifests-and-crds). Use `-o` to set the output file, `--force` to overwrite an existing one, or `--dry-run` to preview. Non-interactive: `--project`, `--environments`, `--set key=value`, or `--defaults` to skip every prompt. | | `deployah validate` | Check the manifest schema (offline). When a platform file exists, also cross-check `expose.domain` keys and environment names against it. | | `deployah validate ` | Also load the platform file and check the resolved configuration for that environment. | | `deployah resolve ` | Preview the fully resolved hostname, TLS mode, and context, offline. Use `--output json` for machine-readable output. | | `deployah resolve --environments` | List every environment from both files: where it is registered, its context (or the kubeconfig fallback), domains, and overrides. | -| `deployah plan ` | Preview what a deploy would change, without applying anything. Use `--offline` to render and validate with no cluster access, `--drift` to also compare against live cluster state, or `--output json` for CI. | -| `deployah deploy ` | Deploy your project. Shows the plan and asks for confirmation before applying; use `-y`/`--yes` to skip the prompt, `--reapply` to upgrade even with no changes, `--explain` to print the resolution report first, or `--force-hostname-change` to bypass the hostname guard. | +| `deployah plan ` | Preview what a deploy would change, without applying anything. Extra manifests from `.deployah/manifests/` appear in the diff; pending CRDs are reported but not applied. Use `--offline` to render with no cluster access, `--drift` to also compare against live cluster state, or `--output json` for CI. | +| `deployah deploy ` | Deploy your project. Shows the plan and asks for confirmation before applying; use `-y`/`--yes` to skip the prompt, `--reapply` to upgrade even with no changes, `--crds` for [CRD install policy](#crd-policy) (`create` or `create-replace`), `--explain` to print the resolution report first, or `--force-hostname-change` to bypass the hostname guard. | | `deployah status ` | Show the status of a deployed project. Use `--detailed` for pod details, `-e` for an environment. | | `deployah logs ` | Stream logs. Filter with `--component`, `-e`, `--container`, `--since`, `--tail`. Use `--no-follow` for a one-off read. | | `deployah shell ` | Open a shell in a running container. Choose with `--component` and `--container`. | @@ -1176,6 +1304,26 @@ Check that your cluster is reachable with `kubectl cluster-info`. For a local cluster, run `deployah cluster up` and deploy with the `local` environment (or pass `--context kind-deployah`). +### Custom manifests and CRDs + +**Unknown type / add its CRD under `.deployah/crds/`.** + +The kind is not a built-in type, not on the small operator allowlist, and not +declared by an in-repo CRD. Put the `CustomResourceDefinition` under +`.deployah/crds/`, or install that API on the cluster before deploying. See +[Custom manifests and CRDs](#custom-manifests-and-crds). + +**Extra collides with a generated object.** + +An object in `.deployah/manifests/` has the same apiVersion, kind, namespace, +and name as something the chart already generates. Rename the extra, or stop +generating that resource from the spec. + +**CRD must use `apiVersion: apiextensions.k8s.io/v1`.** + +Older `apiextensions.k8s.io/v1beta1` CRDs are rejected. Convert the document to +v1. + ### Deploy succeeds but the app returns 503 / times out over HTTPS **Symptom.** `deployah deploy` completes, the pod is Running and 1/1, but diff --git a/codecov.yml b/codecov.yml index fdf2c10..b1621ec 100644 --- a/codecov.yml +++ b/codecov.yml @@ -30,8 +30,9 @@ flag_management: - type: project target: auto threshold: 1% - - type: patch - target: 80% + # Patch coverage is gated once under coverage.status.patch. Flags only + # report project trends so unit and integration uploads are not each + # held to a separate patch target. individual_flags: - name: unittests carryforward: true diff --git a/docs/cli/deployah_deploy.md b/docs/cli/deployah_deploy.md index acfd2bf..e6fbf95 100644 --- a/docs/cli/deployah_deploy.md +++ b/docs/cli/deployah_deploy.md @@ -13,6 +13,7 @@ deployah deploy [flags] ### Options ```text + --crds string CRD install policy: create (install if missing) or create-replace (default "create") --explain Print the resolution report before cluster checks (visible even when cluster is unreachable) --force-hostname-change Allow changing the resolved hostname even though it may break existing traffic (skips the hostname guard) --reapply Upgrade the release even when the plan shows no changes diff --git a/flake.nix b/flake.nix index db29054..bff4cb0 100644 --- a/flake.nix +++ b/flake.nix @@ -25,7 +25,7 @@ buildGoModule' = pkgs.buildGoModule.override { inherit go; }; - deployahVendorHash = "sha256-g4JcFOCXjQGykGgc8fL7OhEU+mDMCbilastfomSO5SY="; + deployahVendorHash = "sha256-q/Kp/5UGNfuwPf27jvg5YNoloDpU9ticuToNXXH+Ynw="; deployah = import ./nix/deployah.nix { buildGoModule = buildGoModule'; diff --git a/go.mod b/go.mod index c7cd8e3..a540759 100644 --- a/go.mod +++ b/go.mod @@ -28,6 +28,7 @@ require ( gopkg.in/yaml.v3 v3.0.1 helm.sh/helm/v4 v4.2.1 k8s.io/api v0.36.2 + k8s.io/apiextensions-apiserver v0.36.2 k8s.io/apimachinery v0.36.2 k8s.io/client-go v0.36.2 nabat.dev v0.6.3 @@ -216,7 +217,6 @@ require ( gopherly.dev/termio v0.3.0 // indirect gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect - k8s.io/apiextensions-apiserver v0.36.2 // indirect k8s.io/apiserver v0.36.2 // indirect k8s.io/cli-runtime v0.36.2 // indirect k8s.io/component-base v0.36.2 // indirect diff --git a/internal/action/deploy.go b/internal/action/deploy.go index 12ce075..5f9165e 100644 --- a/internal/action/deploy.go +++ b/internal/action/deploy.go @@ -4,12 +4,14 @@ import ( "context" "fmt" + "helm.sh/helm/v4/pkg/postrenderer" + "deployah.dev/deployah/internal/spec" ) // Deployer abstracts Helm install/upgrade operations. type Deployer interface { - InstallApp(ctx context.Context, m *spec.Spec, environment string, dryRun bool, resolved *spec.ResolvedSpec) error + InstallApp(ctx context.Context, m *spec.Spec, environment string, dryRun bool, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) error } // SpecLoader loads and validates a spec for an environment. @@ -34,7 +36,7 @@ func (d *Deploy) Run(ctx context.Context, environment string, dryRun bool) (*spe if err != nil { return nil, fmt.Errorf("load spec: %w", err) } - if err = d.deployer.InstallApp(ctx, m, environment, dryRun, nil); err != nil { + if err = d.deployer.InstallApp(ctx, m, environment, dryRun, nil, nil); err != nil { return nil, fmt.Errorf("install: %w", err) } return m, nil diff --git a/internal/action/deploy_test.go b/internal/action/deploy_test.go index faac6b6..b4d1833 100644 --- a/internal/action/deploy_test.go +++ b/internal/action/deploy_test.go @@ -7,6 +7,7 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + "helm.sh/helm/v4/pkg/postrenderer" "deployah.dev/deployah/internal/action" "deployah.dev/deployah/internal/spec" @@ -16,7 +17,7 @@ type mockDeployer struct { err error } -func (m *mockDeployer) InstallApp(_ context.Context, _ *spec.Spec, _ string, _ bool, _ *spec.ResolvedSpec) error { +func (m *mockDeployer) InstallApp(_ context.Context, _ *spec.Spec, _ string, _ bool, _ *spec.ResolvedSpec, _ postrenderer.PostRenderer) error { return m.err } diff --git a/internal/cmd/deploy/deploy.go b/internal/cmd/deploy/deploy.go index ebb30da..6631438 100644 --- a/internal/cmd/deploy/deploy.go +++ b/internal/cmd/deploy/deploy.go @@ -9,11 +9,13 @@ import ( "strings" "sync" + "helm.sh/helm/v4/pkg/postrenderer" "k8s.io/client-go/kubernetes" "nabat.dev/nabat" "nabat.dev/theme" "deployah.dev/deployah/internal/cmd/common" + "deployah.dev/deployah/internal/extras" "deployah.dev/deployah/internal/k8s" "deployah.dev/deployah/internal/readiness" "deployah.dev/deployah/internal/render" @@ -30,8 +32,12 @@ type Options struct { ForceHostnameChange bool `nabat:"force-hostname-change"` Yes bool `nabat:"yes"` Reapply bool `nabat:"reapply"` + CRDs string `nabat:"crds"` } +// crdPolicies are the allowed values for --crds (same order as help text). +var crdPolicies = []string{string(extras.PolicyCreate), string(extras.PolicyCreateReplace)} + // Register adds the deploy command to app. func Register(app *nabat.App) { app.MustCommand("deploy", @@ -42,6 +48,7 @@ func Register(app *nabat.App) { nabat.WithFlag("force-hostname-change", false, nabat.WithUsage("Allow changing the resolved hostname even though it may break existing traffic (skips the hostname guard)")), nabat.WithFlag("yes", false, nabat.WithShort('y'), nabat.WithUsage("Apply without an interactive confirmation prompt")), nabat.WithFlag("reapply", false, nabat.WithUsage("Upgrade the release even when the plan shows no changes")), + nabat.WithSelectFlag("crds", string(extras.PolicyCreate), crdPolicies, nabat.WithUsage("CRD install policy: create (install if missing) or create-replace")), nabat.WithExample(` # Deploy to production using the default spec path (./deployah.yaml) deployah deploy prod @@ -52,6 +59,9 @@ deployah deploy staging -s ./path/to/deployah.yaml # Deploy without an interactive confirmation prompt (e.g. in CI) deployah deploy prod --yes +# Replace existing CRDs from .deployah/crds/ when deploying +deployah deploy prod --crds create-replace + # Show resolution report before deploying deployah deploy prod --explain @@ -175,12 +185,26 @@ func runDeploy(c *nabat.Context, resolvedTheme theme.ResolvedTheme) error { } } - plan, err := computePlan(c, helmClient, cluster, manifest, opts.Environment, resolvedSpec) + restCfg, restErr := cluster.RESTConfig() + if restErr != nil { + c.Logger().Debug("rest config unavailable for extras scope discovery", "err", restErr) + } + bundle, err := extras.LoadFromSpec(sess.SpecPath(), manifest, platform, opts.Environment, cluster.Namespace(), restCfg) + if err != nil { + return fmt.Errorf("load extras: %w", err) + } + postRenderer := bundle.PostRendererFor() + + plan, err := computePlan(c, helmClient, cluster, manifest, opts.Environment, resolvedSpec, postRenderer) if err != nil { return err } defer plan.cleanup() + if n := len(bundle.CRDs); n > 0 { + c.Printf("CRDs: %d from .deployah/crds/ (policy %s)\n", n, opts.CRDs) + } + textOpts := planengine.TextOptions{Mode: planengine.ModeCompact, Theme: resolvedTheme} if renderErr := planengine.RenderText(c.IO().Out, plan.diff, textOpts); renderErr != nil { return fmt.Errorf("render plan: %w", renderErr) @@ -194,21 +218,30 @@ func runDeploy(c *nabat.Context, resolvedTheme theme.ResolvedTheme) error { } } - if !plan.diff.HasChanges() && !opts.Reapply { + helmIdle := !plan.diff.HasChanges() && !opts.Reapply + if skipWhenIdle(helmIdle, len(bundle.CRDs)) { return skipDeploy(c, k8sClient, k8sErr, plan) } // Required-API check runs before confirmation: a missing CRD/API is a // precondition failure the user shouldn't have to confirm past first. - if k8sErr == nil { - if reqs := requiredAPIs(manifest, opts.Environment, resolvedSpec); len(reqs) > 0 { + // Skip when Helm is idle and only CRDs will apply (CRDs are the APIs). + // Also skip group/versions that .deployah/crds/ will install in this deploy. + if !helmIdle && k8sErr == nil { + reqs := filterCoveredAPIs(requiredAPIs(manifest, opts.Environment, resolvedSpec), extras.GroupVersionsFromCRDs(bundle.CRDs)) + if len(reqs) > 0 { if capErr := k8s.CheckAPIRequirements(k8sClient, reqs); capErr != nil { return capErr } } } - proceed, confirmErr := confirmApply(c, opts) + prompt := "Apply these changes?" + if helmIdle { + n := len(bundle.CRDs) + prompt = fmt.Sprintf("Helm release is unchanged. Apply %d %s?", n, crdNoun(n)) + } + proceed, confirmErr := confirmApply(c, opts, prompt) if confirmErr != nil { return confirmErr } @@ -217,20 +250,29 @@ func runDeploy(c *nabat.Context, resolvedTheme theme.ResolvedTheme) error { return nil } - return applyDeploy(c, sess, cluster, helmClient, platform, manifest, opts, resolvedSpec, plan, k8sClient, k8sErr) + if helmIdle { + return applyCRDsOnly(c, sess, cluster, k8sClient, k8sErr, plan, bundle, opts) + } + return applyDeploy(c, sess, cluster, helmClient, platform, manifest, opts, resolvedSpec, plan, k8sClient, k8sErr, bundle, postRenderer) +} + +// skipWhenIdle reports whether deploy should exit without cluster writes: +// Helm plan idle and no CRDs to apply. +func skipWhenIdle(helmIdle bool, crdCount int) bool { + return helmIdle && crdCount == 0 } // confirmApply gates the real apply behind --yes or an interactive prompt. // proceed is false with a nil error on a clean "no"; err is non-nil only when -// non-interactive without --yes, or the prompt fails. -func confirmApply(c *nabat.Context, opts *Options) (proceed bool, err error) { +// non-interactive without --yes, or the prompt fails. prompt must be non-empty. +func confirmApply(c *nabat.Context, opts *Options, prompt string) (proceed bool, err error) { if opts.Yes { return true, nil } if !c.IsInteractive() { return false, errors.New("refusing to deploy without confirmation; re-run with --yes") } - confirmed, confirmErr := c.Confirm("Apply these changes?") + confirmed, confirmErr := c.Confirm(prompt) if confirmErr != nil { return false, fmt.Errorf("confirmation: %w", confirmErr) } @@ -240,8 +282,8 @@ func confirmApply(c *nabat.Context, opts *Options) (proceed bool, err error) { // computePlan renders the chart client-side and diffs it against the last // successful release. It never mutates the cluster or Helm's release history. // The caller must invoke deployPlan.cleanup when finished with the result. -func computePlan(c *nabat.Context, helmClient session.HelmClient, cluster *session.Cluster, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) (*deployPlan, error) { - diff, result, cleanup, err := planengine.BuildPlan(c, helmClient, manifest, environment, cluster.Context(), resolved) +func computePlan(c *nabat.Context, helmClient session.HelmClient, cluster *session.Cluster, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) (*deployPlan, error) { + diff, result, cleanup, err := planengine.BuildPlan(c, helmClient, manifest, environment, cluster.Context(), resolved, postRenderer) if err != nil { cleanup() return nil, fmt.Errorf("%w%s", err, common.ClusterHint(err)) @@ -249,11 +291,48 @@ func computePlan(c *nabat.Context, helmClient session.HelmClient, cluster *sessi return &deployPlan{diff: diff, result: result, cleanup: cleanup}, nil } -// skipDeploy handles a plan with no changes: Helm is never invoked, but pod -// readiness for the release is still shown for parity with a real deploy. +// skipDeploy handles a plan with no changes and no CRDs: Helm is never +// invoked, but pod readiness for the release is still shown for parity with +// a real deploy. func skipDeploy(c *nabat.Context, k8sClient kubernetes.Interface, k8sErr error, plan *deployPlan) error { c.Success(fmt.Sprintf("No changes. Release %s unchanged (revision %d).", plan.diff.Header.Release, plan.diff.Header.Revision)) + return printReadiness(c, k8sClient, k8sErr, plan) +} + +// applyCRDsOnly applies .deployah/crds when the Helm plan is idle. CRDs are +// outside the Helm release, so a no-op chart must not skip them. +func applyCRDsOnly(c *nabat.Context, sess *session.Session, cluster *session.Cluster, k8sClient kubernetes.Interface, k8sErr error, plan *deployPlan, bundle *extras.Bundle, opts *Options) error { + stats, err := applyBundleCRDs(c, sess, cluster, bundle, opts) + if err != nil { + return err + } + c.Success(crdIdleSuccessMessage(stats, plan.diff.Header.Release, plan.diff.Header.Revision)) + return printReadiness(c, k8sClient, k8sErr, plan) +} + +func crdIdleSuccessMessage(stats extras.CRDStats, release string, revision int) string { + suffix := fmt.Sprintf("Release %s unchanged (revision %d).", release, revision) + if stats.Created == 0 && stats.Replaced == 0 { + return fmt.Sprintf("CRDs ready (%d already present). %s", stats.Ready, suffix) + } + parts := make([]string, 0, 2) + if stats.Created > 0 { + parts = append(parts, fmt.Sprintf("%d created", stats.Created)) + } + if stats.Replaced > 0 { + parts = append(parts, fmt.Sprintf("%d replaced", stats.Replaced)) + } + return fmt.Sprintf("Applied %d %s (%s). %s", stats.Ready, crdNoun(stats.Ready), strings.Join(parts, ", "), suffix) +} +func crdNoun(n int) string { + if n == 1 { + return "CRD" + } + return "CRDs" +} + +func printReadiness(c *nabat.Context, k8sClient kubernetes.Interface, k8sErr error, plan *deployPlan) error { if k8sErr != nil { c.Logger().Debug("skipping readiness summary: k8s client unavailable", "err", k8sErr) return nil @@ -269,10 +348,26 @@ func skipDeploy(c *nabat.Context, k8sClient kubernetes.Interface, k8sErr error, return nil } +func applyBundleCRDs(c *nabat.Context, sess *session.Session, cluster *session.Cluster, bundle *extras.Bundle, opts *Options) (extras.CRDStats, error) { + var stats extras.CRDStats + if len(bundle.CRDs) == 0 { + return stats, nil + } + restCfg, restErr := cluster.RESTConfig() + if restErr != nil { + return stats, fmt.Errorf("rest config for CRDs: %w%s", restErr, common.ClusterHint(restErr)) + } + stats, crdErr := extras.ApplyCRDs(c, restCfg, bundle.CRDs, extras.Policy(opts.CRDs), sess.Timeout()) + if crdErr != nil { + return stats, fmt.Errorf("apply CRDs: %w%s", crdErr, common.ClusterHint(crdErr)) + } + return stats, nil +} + // applyDeploy re-renders and verifies determinism before the real Helm -// install/upgrade. -func applyDeploy(c *nabat.Context, sess *session.Session, cluster *session.Cluster, helmClient session.HelmClient, platform *spec.PlatformConfig, manifest *spec.Spec, opts *Options, resolved *spec.ResolvedSpec, plan *deployPlan, k8sClient kubernetes.Interface, k8sErr error) error { - verify, verifyCleanup, err := helmClient.RenderManifests(c, manifest, opts.Environment, resolved) +// install/upgrade. CRDs from the bundle are applied first. +func applyDeploy(c *nabat.Context, sess *session.Session, cluster *session.Cluster, helmClient session.HelmClient, platform *spec.PlatformConfig, manifest *spec.Spec, opts *Options, resolved *spec.ResolvedSpec, plan *deployPlan, k8sClient kubernetes.Interface, k8sErr error, bundle *extras.Bundle, postRenderer postrenderer.PostRenderer) error { + verify, verifyCleanup, err := helmClient.RenderManifests(c, manifest, opts.Environment, resolved, postRenderer) if verifyCleanup != nil { defer verifyCleanup() } @@ -285,6 +380,11 @@ func applyDeploy(c *nabat.Context, sess *session.Session, cluster *session.Clust return errors.New("rendered manifests changed between plan and apply; re-run 'deployah deploy' to see the current plan") } + crdStats, crdErr := applyBundleCRDs(c, sess, cluster, bundle, opts) + if crdErr != nil { + return crdErr + } + resolvedCtx := cluster.Context() ctxSuffix := "" if resolvedCtx != "" { @@ -320,7 +420,7 @@ func applyDeploy(c *nabat.Context, sess *session.Session, cluster *session.Clust watcher.Run(watchCtx, st) }) } - helmErr := helmClient.InstallApp(c, manifest, opts.Environment, false, resolved) + helmErr := helmClient.InstallApp(c, manifest, opts.Environment, false, resolved, postRenderer) if cancel != nil { cancel() } @@ -337,10 +437,27 @@ func applyDeploy(c *nabat.Context, sess *session.Session, cluster *session.Clust } summary := buildSummaryMsg(watcher) - c.Success("Deployed"+summary, "project", manifest.Project, "environment", opts.Environment) + c.Success("Deployed"+summary+crdApplySuffix(crdStats), "project", manifest.Project, "environment", opts.Environment) return nil } +// crdApplySuffix appends a short CRD count when this deploy created or +// replaced any CRDs (already-present-only is silent on the Helm path). +func crdApplySuffix(stats extras.CRDStats) string { + if stats.Created == 0 && stats.Replaced == 0 { + return "" + } + parts := make([]string, 0, 2) + if stats.Created > 0 { + parts = append(parts, fmt.Sprintf("%d created", stats.Created)) + } + if stats.Replaced > 0 { + parts = append(parts, fmt.Sprintf("%d replaced", stats.Replaced)) + } + n := stats.Created + stats.Replaced + return fmt.Sprintf("; applied %d %s (%s)", n, crdNoun(n), strings.Join(parts, ", ")) +} + // checkHostnameGuard blocks FQDN changes relative to the last successful // release, unless force is set. Skips when there is no prior successful // release (first install or every revision failed). History fetch errors @@ -445,6 +562,29 @@ func buildSummaryMsg(w *DeployWatcher) string { return " (" + summary + ")" } +// filterCoveredAPIs drops requirements that will be satisfied by CRDs this +// deploy installs from .deployah/crds/ (any one matching group/version is +// enough, matching [k8s.CheckAPIRequirements]). +func filterCoveredAPIs(reqs []k8s.APIRequirement, covered map[string]struct{}) []k8s.APIRequirement { + if len(covered) == 0 || len(reqs) == 0 { + return reqs + } + out := make([]k8s.APIRequirement, 0, len(reqs)) + for _, req := range reqs { + pending := false + for _, gv := range req.GroupVersions { + if _, ok := covered[gv]; ok { + pending = true + break + } + } + if !pending { + out = append(out, req) + } + } + return out +} + // requiredAPIs derives the Kubernetes API group/version requirements for // components deployed in the target environment, including cert-manager // (TLS mode) via the resolved spec. diff --git a/internal/cmd/deploy/deploy_flow_test.go b/internal/cmd/deploy/deploy_flow_test.go index 5ef33cd..b1c20d4 100644 --- a/internal/cmd/deploy/deploy_flow_test.go +++ b/internal/cmd/deploy/deploy_flow_test.go @@ -15,13 +15,17 @@ package deploy import ( + "path/filepath" "testing" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/client-go/kubernetes" "k8s.io/client-go/kubernetes/fake" + "deployah.dev/deployah/internal/extras" + "deployah.dev/deployah/internal/k8s" "deployah.dev/deployah/internal/render" "deployah.dev/deployah/internal/session" "deployah.dev/deployah/internal/spec" @@ -115,7 +119,7 @@ func TestConfirmApply(t *testing.T) { t.Run(tt.name, func(t *testing.T) { t.Parallel() c := nabatContext(t) // nabattest.NewIO reports non-TTY by default - proceed, err := confirmApply(c, tt.opts) + proceed, err := confirmApply(c, tt.opts, "Apply these changes?") if tt.wantErr { require.Error(t, err) assert.False(t, proceed) @@ -154,7 +158,7 @@ func TestApplyDeploy_RenderMismatch_AbortsBeforeApply(t *testing.T) { opts := &Options{Environment: "production"} manifest := &spec.Spec{Project: "web"} - err := applyDeploy(c, sess, cluster, stub, nil, manifest, opts, nil, planned, nil, nil) + err := applyDeploy(c, sess, cluster, stub, nil, manifest, opts, nil, planned, nil, nil, &extras.Bundle{}, nil) require.Error(t, err) assert.Contains(t, err.Error(), "changed between plan and apply") assert.Equal(t, 1, stub.renderCallCount, "must re-render exactly once before comparing") @@ -199,3 +203,228 @@ func TestSkipDeploy_NoChanges_ShowsReadinessSummary(t *testing.T) { assert.Contains(t, stdout.String(), "Readiness:") assert.Contains(t, stdout.String(), "web: 1/1") } + +// TestSkipWhenIdle locks the gate that used to skip CRD apply on a no-op +// Helm plan. +func TestSkipWhenIdle(t *testing.T) { + t.Parallel() + tests := []struct { + name string + helmIdle bool + crdCount int + want bool + }{ + {name: "idle no CRDs", helmIdle: true, crdCount: 0, want: true}, + {name: "idle with CRDs", helmIdle: true, crdCount: 1, want: false}, + {name: "helm changes no CRDs", helmIdle: false, crdCount: 0, want: false}, + {name: "helm changes with CRDs", helmIdle: false, crdCount: 2, want: false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + assert.Equal(t, tt.want, skipWhenIdle(tt.helmIdle, tt.crdCount)) + }) + } +} + +// TestCRDIdleSuccessMessage covers wait-only vs created/replaced wording. +func TestCRDIdleSuccessMessage(t *testing.T) { + t.Parallel() + tests := []struct { + name string + stats extras.CRDStats + want string + }{ + { + name: "already present", + stats: extras.CRDStats{Ready: 2}, + want: "CRDs ready (2 already present). Release web-production unchanged (revision 3).", + }, + { + name: "created and replaced", + stats: extras.CRDStats{Created: 1, Replaced: 1, Ready: 2}, + want: "Applied 2 CRDs (1 created, 1 replaced). Release web-production unchanged (revision 3).", + }, + { + name: "one created", + stats: extras.CRDStats{Created: 1, Ready: 1}, + want: "Applied 1 CRD (1 created). Release web-production unchanged (revision 3).", + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + assert.Equal(t, tt.want, crdIdleSuccessMessage(tt.stats, "web-production", 3)) + }) + } +} + +// TestCRDApplySuffix covers the Helm-path success footnote for CRD writes. +func TestCRDApplySuffix(t *testing.T) { + t.Parallel() + tests := []struct { + name string + stats extras.CRDStats + want string + }{ + {name: "noop", stats: extras.CRDStats{Ready: 2}, want: ""}, + {name: "created", stats: extras.CRDStats{Created: 1, Ready: 1}, want: "; applied 1 CRD (1 created)"}, + {name: "both", stats: extras.CRDStats{Created: 1, Replaced: 2, Ready: 3}, want: "; applied 3 CRDs (1 created, 2 replaced)"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + assert.Equal(t, tt.want, crdApplySuffix(tt.stats)) + }) + } +} + +// TestFilterCoveredAPIs drops requirements satisfied by pending CRDs. +func TestFilterCoveredAPIs(t *testing.T) { + t.Parallel() + reqs := []k8s.APIRequirement{ + {GroupVersions: []string{"cert-manager.io/v1"}, Reason: "tls"}, + {GroupVersions: []string{"autoscaling/v2", "autoscaling/v2beta2"}, Reason: "hpa"}, + } + covered := map[string]struct{}{"cert-manager.io/v1": {}} + got := filterCoveredAPIs(reqs, covered) + require.Len(t, got, 1) + assert.Equal(t, []string{"autoscaling/v2", "autoscaling/v2beta2"}, got[0].GroupVersions) + assert.Equal(t, reqs, filterCoveredAPIs(reqs, nil)) +} + +// TestApplyCRDsOnly_ReportsSuccessAndReadiness covers the Helm-idle success +// message and readiness poll. applyBundleCRDs is a no-op for an empty CRD +// list (real CRD apply needs a live apiextensions API). +func TestApplyCRDsOnly_ReportsSuccessAndReadiness(t *testing.T) { + t.Parallel() + k8sClient := fake.NewSimpleClientset( + &corev1.Pod{ + ObjectMeta: metav1.ObjectMeta{ + Name: "web-1", + Namespace: "default", + Labels: map[string]string{ + "app.kubernetes.io/instance": "web-production", + "app.kubernetes.io/component": "web", + }, + }, + Status: corev1.PodStatus{ + Phase: corev1.PodRunning, + ContainerStatuses: []corev1.ContainerStatus{ + {Ready: true}, + }, + }, + }, + ) + stub := &stubHelmClient{} + cluster := newClusterWithStub(t, stub, k8sClient) + sess := cluster.Session + c, _, stdout, stderr := nabatContextWithIO(t) + plan := &deployPlan{ + diff: &planengine.Plan{ + Header: planengine.Header{Release: "web-production", Revision: 3}, + }, + result: testRenderResult(deployFlowManifestV1), + cleanup: func() {}, + } + opts := &Options{Environment: "production", CRDs: string(extras.PolicyCreate)} + + err := applyCRDsOnly(c, sess, cluster, k8sClient, nil, plan, &extras.Bundle{}, opts) + require.NoError(t, err) + assert.Contains(t, stderr.String(), "CRDs ready (0 already present). Release web-production unchanged (revision 3).") + assert.Contains(t, stdout.String(), "Readiness:") +} + +func sampleBundleCRD(t *testing.T) *extras.Bundle { + t.Helper() + obj := &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "apiextensions.k8s.io/v1", + "kind": "CustomResourceDefinition", + "metadata": map[string]any{"name": "widgets.example.com"}, + "spec": map[string]any{ + "group": "example.com", + "scope": "Namespaced", + "names": map[string]any{"kind": "Widget", "plural": "widgets"}, + "versions": []any{ + map[string]any{"name": "v1", "served": true, "storage": true}, + }, + }, + }} + o := extras.Object{Path: "widget.yaml", Obj: obj} + raw, err := o.MarshalYAML() + require.NoError(t, err) + o.Raw = raw + return &extras.Bundle{CRDs: []extras.Object{o}} +} + +// TestApplyBundleCRDs_RESTConfigError surfaces kubeconfig failures before apply. +func TestApplyBundleCRDs_RESTConfigError(t *testing.T) { + t.Parallel() + stub := &stubHelmClient{} + sess := session.New( + session.WithKubeconfig(filepath.Join(t.TempDir(), "missing-kubeconfig")), + session.WithHelmFactory(func(*session.Session) (session.HelmClient, error) { + return stub, nil + }), + ) + cluster, err := sess.Target(t.Context(), "production") + require.NoError(t, err) + c := nabatContext(t) + opts := &Options{Environment: "production", CRDs: string(extras.PolicyCreate)} + + _, err = applyBundleCRDs(c, sess, cluster, sampleBundleCRD(t), opts) + require.Error(t, err) + assert.Contains(t, err.Error(), "rest config for CRDs") +} + +// TestApplyDeploy_CallsInstallAfterEmptyCRDs applies Helm when CRD list is empty. +func TestApplyDeploy_CallsInstallAfterEmptyCRDs(t *testing.T) { + t.Parallel() + manifest := deployFlowManifestV1 + stub := &stubHelmClient{ + renderResults: []*render.RenderResult{testRenderResult(manifest)}, + } + cluster := newClusterWithStub(t, stub, nil) + sess := cluster.Session + planned := &deployPlan{ + diff: &planengine.Plan{Header: planengine.Header{Release: "web-production", Revision: 1}}, + result: testRenderResult(manifest), + cleanup: func() {}, + } + c, _, _, stderr := nabatContextWithIO(t) + opts := &Options{Environment: "production", CRDs: string(extras.PolicyCreate)} + + err := applyDeploy(c, sess, cluster, stub, nil, &spec.Spec{Project: "web"}, opts, nil, planned, nil, assertNever{}, &extras.Bundle{}, nil) + require.NoError(t, err) + assert.Equal(t, 1, stub.installCallCount) + assert.Contains(t, stderr.String(), "Deployed") +} + +// TestApplyDeploy_PropagatesCRDApplyError skips InstallApp when CRDs fail. +func TestApplyDeploy_PropagatesCRDApplyError(t *testing.T) { + t.Parallel() + manifest := deployFlowManifestV1 + stub := &stubHelmClient{ + renderResults: []*render.RenderResult{testRenderResult(manifest)}, + } + sess := session.New( + session.WithKubeconfig(filepath.Join(t.TempDir(), "missing-kubeconfig")), + session.WithHelmFactory(func(*session.Session) (session.HelmClient, error) { + return stub, nil + }), + ) + cluster, err := sess.Target(t.Context(), "production") + require.NoError(t, err) + planned := &deployPlan{ + diff: &planengine.Plan{}, + result: testRenderResult(manifest), + cleanup: func() {}, + } + c := nabatContext(t) + opts := &Options{Environment: "production", CRDs: string(extras.PolicyCreate)} + + err = applyDeploy(c, sess, cluster, stub, nil, &spec.Spec{Project: "web"}, opts, nil, planned, nil, nil, sampleBundleCRD(t), nil) + require.Error(t, err) + assert.Contains(t, err.Error(), "rest config for CRDs") + assert.Equal(t, 0, stub.installCallCount) +} diff --git a/internal/cmd/deploy/deploy_test.go b/internal/cmd/deploy/deploy_test.go index 1d95bd6..3681688 100644 --- a/internal/cmd/deploy/deploy_test.go +++ b/internal/cmd/deploy/deploy_test.go @@ -11,6 +11,7 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + "helm.sh/helm/v4/pkg/postrenderer" "helm.sh/helm/v4/pkg/release/common" "k8s.io/apimachinery/pkg/labels" "nabat.dev/nabat" @@ -40,16 +41,18 @@ type stubHelmClient struct { renderErr error renderCallCount int - installErr error + installErr error + installCallCount int } func (s *stubHelmClient) IsReachable() error { return nil } -func (s *stubHelmClient) InstallApp(context.Context, *spec.Spec, string, bool, *spec.ResolvedSpec) error { +func (s *stubHelmClient) InstallApp(context.Context, *spec.Spec, string, bool, *spec.ResolvedSpec, postrenderer.PostRenderer) error { + s.installCallCount++ return s.installErr } -func (s *stubHelmClient) RenderManifests(context.Context, *spec.Spec, string, *spec.ResolvedSpec) (*render.RenderResult, func(), error) { +func (s *stubHelmClient) RenderManifests(context.Context, *spec.Spec, string, *spec.ResolvedSpec, postrenderer.PostRenderer) (*render.RenderResult, func(), error) { if s.renderErr != nil { return nil, func() {}, s.renderErr } @@ -61,7 +64,7 @@ func (s *stubHelmClient) RenderManifests(context.Context, *spec.Spec, string, *s return s.renderResults[i], func() {}, nil } -func (s *stubHelmClient) RenderOffline(context.Context, *spec.Spec, string, *spec.ResolvedSpec) (*render.RenderResult, func(), error) { +func (s *stubHelmClient) RenderOffline(context.Context, *spec.Spec, string, *spec.ResolvedSpec, postrenderer.PostRenderer) (*render.RenderResult, func(), error) { panic("unexpected RenderOffline call") } diff --git a/internal/cmd/initialize/summary.go b/internal/cmd/initialize/summary.go index 080d4dc..44e9493 100644 --- a/internal/cmd/initialize/summary.go +++ b/internal/cmd/initialize/summary.go @@ -73,6 +73,16 @@ func showSummaryAndSave(c *nabat.Context, config *ProjectConfig) error { } printPlatformEnvironmentGuidance(c, platformPath, envNames) + createdExtras, scaffoldErr := scaffoldExtrasDirs(filepath.Dir(config.OutputPath)) + switch { + case scaffoldErr != nil: + c.Warn(fmt.Sprintf("failed to create .deployah extras directories: %v", scaffoldErr)) + case createdExtras: + c.Success("Created .deployah/manifests/ and .deployah/crds/") + default: + c.Info(".deployah/manifests/ and .deployah/crds/ already exist; no changes made.") + } + c.Println("Next steps:") c.Println(" 1. Review: cat " + config.OutputPath) c.Println(" 2. Validate: deployah validate") @@ -81,6 +91,63 @@ func showSummaryAndSave(c *nabat.Context, config *ProjectConfig) error { return nil } +const ( + manifestsREADME = `# Extra manifests + +Put raw Kubernetes YAML here. Deployah loads it into the same Helm release +as your generated resources. + +- Common files (all environments): place *.yaml / *.yml in this directory +- Per environment: use a subdirectory named after a declared environment + key (for example manifests/prod/extra.yaml) +- Files are applied literally: no Helm templating and no env substitution +- CustomResourceDefinition belongs in ../crds/, not here +` + crdsREADME = `# Extra CRDs + +Put CustomResourceDefinition YAML here. Deployah applies these to the +cluster before the Helm release, then waits for each CRD to become +Established. + +- Shared across all environments (no per-env subdirectories) +- Install policy: deployah deploy --crds create|create-replace +- CRDs are never deleted on uninstall +` +) + +// scaffoldExtrasDirs creates .deployah/manifests and .deployah/crds with +// short README files. Returns whether anything new was written. +func scaffoldExtrasDirs(specDir string) (created bool, err error) { + root := filepath.Join(specDir, spec.DeployahConfigDir) + entries := []struct { + dir string + file string + body string + }{ + {filepath.Join(root, spec.ManifestsDir), "README.md", manifestsREADME}, + {filepath.Join(root, spec.CRDsDir), "README.md", crdsREADME}, + } + for _, e := range entries { + if _, statErr := os.Stat(e.dir); os.IsNotExist(statErr) { + created = true + } + if mkdirErr := os.MkdirAll(e.dir, 0o750); mkdirErr != nil { + return false, fmt.Errorf("create %s: %w", e.dir, mkdirErr) + } + path := filepath.Join(e.dir, e.file) + if _, statErr := os.Stat(path); statErr == nil { + continue + } else if !os.IsNotExist(statErr) { + return false, fmt.Errorf("stat %s: %w", path, statErr) + } + if writeErr := os.WriteFile(path, []byte(e.body), 0o600); writeErr != nil { + return false, fmt.Errorf("write %s: %w", path, writeErr) + } + created = true + } + return created, nil +} + // printPlatformEnvironmentGuidance reports env names with no entry in the // platform file and registered entries with no context, so the user knows // what to fill in before deploying. diff --git a/internal/cmd/initialize/summary_test.go b/internal/cmd/initialize/summary_test.go index 8fd1f61..f4443cc 100644 --- a/internal/cmd/initialize/summary_test.go +++ b/internal/cmd/initialize/summary_test.go @@ -1,6 +1,7 @@ package initialize import ( + "os" "path/filepath" "testing" @@ -82,6 +83,13 @@ func TestShowSummaryAndSave_RoleAwareComponentsProduceValidSpec(t *testing.T) { require.NoError(t, loadErr) _, hasLocal := platform.Environments["local"] assert.True(t, hasLocal, "the local environment should have a full platform entry") + + manifestsDir := filepath.Join(dir, spec.DeployahConfigDir, spec.ManifestsDir) + crdsDir := filepath.Join(dir, spec.DeployahConfigDir, spec.CRDsDir) + require.DirExists(t, manifestsDir) + require.DirExists(t, crdsDir) + require.FileExists(t, filepath.Join(manifestsDir, "README.md")) + require.FileExists(t, filepath.Join(crdsDir, "README.md")) } // TestShowSummaryAndSave_ServiceHealthCheckWithoutPortIsValid locks in the @@ -147,3 +155,53 @@ func TestShowSummaryAndSave_NonLocalOnlyRegistersEmptyEntry(t *testing.T) { require.True(t, hasProduction, "production must be registered in the platform file") assert.Empty(t, production.Context) } + +// TestScaffoldExtrasDirs_Idempotent reports already-exist when dirs are present. +func TestScaffoldExtrasDirs_Idempotent(t *testing.T) { + t.Parallel() + dir := t.TempDir() + created, err := scaffoldExtrasDirs(dir) + require.NoError(t, err) + assert.True(t, created) + + created, err = scaffoldExtrasDirs(dir) + require.NoError(t, err) + assert.False(t, created) +} + +// TestShowSummaryAndSave_ExtrasAlreadyExist prints the Info branch for extras. +func TestShowSummaryAndSave_ExtrasAlreadyExist(t *testing.T) { + dir := t.TempDir() + outputPath := filepath.Join(dir, "deployah.yaml") + manifestsDir := filepath.Join(dir, spec.DeployahConfigDir, spec.ManifestsDir) + crdsDir := filepath.Join(dir, spec.DeployahConfigDir, spec.CRDsDir) + require.NoError(t, os.MkdirAll(manifestsDir, 0o750)) + require.NoError(t, os.MkdirAll(crdsDir, 0o750)) + require.NoError(t, os.WriteFile(filepath.Join(manifestsDir, "README.md"), []byte("x"), 0o600)) + require.NoError(t, os.WriteFile(filepath.Join(crdsDir, "README.md"), []byte("x"), 0o600)) + + config := &ProjectConfig{ + Name: "shop", + EnvironmentNames: []string{"local"}, + Components: map[string]spec.Component{ + "web": { + Role: spec.ComponentRoleService, + Image: "nginx:1.28.0-alpine", + Port: 8080, + ResourcePreset: spec.ResourcePresetSmall, + }, + }, + OutputPath: outputPath, + } + + io, _, _, errOut := nabattest.NewIO() + var captured *nabat.Context + app := nabat.MustNew("test", nabat.WithIO(io)) + app.MustCommand("run", nabat.WithRun(func(c *nabat.Context) error { + captured = c + return nil + })) + require.NoError(t, nabattest.Run(t, app, []string{"run"})) + require.NoError(t, showSummaryAndSave(captured, config)) + assert.Contains(t, errOut.String(), ".deployah/manifests/ and .deployah/crds/ already exist") +} diff --git a/internal/cmd/plan/plan.go b/internal/cmd/plan/plan.go index 7e66695..0b385db 100644 --- a/internal/cmd/plan/plan.go +++ b/internal/cmd/plan/plan.go @@ -23,6 +23,7 @@ import ( "deployah.dev/deployah/internal/cmd/common" "deployah.dev/deployah/internal/drift" + "deployah.dev/deployah/internal/extras" "deployah.dev/deployah/internal/k8s" "deployah.dev/deployah/internal/session" "deployah.dev/deployah/internal/spec" @@ -160,17 +161,15 @@ func runPlan(c *nabat.Context, resolvedTheme theme.ResolvedTheme) error { } if opts.Offline { - return runOffline(c, manifest, opts, resolvedSpec) + return runOffline(c, sess, platform, manifest, opts, resolvedSpec) } - return runOnline(c, sess, manifest, opts, resolvedSpec, resolvedTheme) + return runOnline(c, sess, platform, manifest, opts, resolvedSpec, resolvedTheme) } // runOffline renders the chart without contacting the cluster and prints a // resource count instead of a diff: there is no reachable release history to // compare against. -func runOffline(c *nabat.Context, manifest *spec.Spec, opts *Options, resolvedSpec *spec.ResolvedSpec) error { - sess := session.FromContext(c) - +func runOffline(c *nabat.Context, sess *session.Session, platform *spec.PlatformConfig, manifest *spec.Spec, opts *Options, resolvedSpec *spec.ResolvedSpec) error { cluster, err := sess.Target(c, opts.Environment) if err != nil { return fmt.Errorf("target cluster: %w", err) @@ -190,7 +189,13 @@ func runOffline(c *nabat.Context, manifest *spec.Spec, opts *Options, resolvedSp } } - result, cleanup, err := helmClient.RenderOffline(c, manifest, opts.Environment, resolvedSpec) + bundle, err := extras.LoadFromSpec(sess.SpecPath(), manifest, platform, opts.Environment, cluster.Namespace(), nil) + if err != nil { + return fmt.Errorf("load extras: %w", err) + } + postRenderer := bundle.PostRendererFor() + + result, cleanup, err := helmClient.RenderOffline(c, manifest, opts.Environment, resolvedSpec, postRenderer) if cleanup != nil { defer cleanup() } @@ -204,13 +209,16 @@ func runOffline(c *nabat.Context, manifest *spec.Spec, opts *Options, resolvedSp } c.Println(fmt.Sprintf("Rendered %d resources for environment '%s' (no cluster comparison).", count, opts.Environment)) + if n := len(bundle.CRDs); n > 0 { + c.Printf("CRDs: %d pending from .deployah/crds/ (not applied in plan)\n", n) + } c.Println("validation: OK") return nil } // runOnline renders the chart, diffs it against the last successful // release, and displays the resulting plan. -func runOnline(c *nabat.Context, sess *session.Session, manifest *spec.Spec, opts *Options, resolvedSpec *spec.ResolvedSpec, resolvedTheme theme.ResolvedTheme) error { +func runOnline(c *nabat.Context, sess *session.Session, platform *spec.PlatformConfig, manifest *spec.Spec, opts *Options, resolvedSpec *spec.ResolvedSpec, resolvedTheme theme.ResolvedTheme) error { cluster, err := sess.Target(c, opts.Environment) if err != nil { return fmt.Errorf("target cluster: %w", err) @@ -240,12 +248,26 @@ func runOnline(c *nabat.Context, sess *session.Session, manifest *spec.Spec, opt } } - p, result, cleanup, err := planengine.BuildPlan(c, helmClient, manifest, opts.Environment, cluster.Context(), resolvedSpec) + restCfg, restErr := cluster.RESTConfig() + if restErr != nil { + c.Logger().Debug("rest config unavailable for extras scope discovery", "err", restErr) + } + bundle, err := extras.LoadFromSpec(sess.SpecPath(), manifest, platform, opts.Environment, cluster.Namespace(), restCfg) + if err != nil { + return fmt.Errorf("load extras: %w", err) + } + postRenderer := bundle.PostRendererFor() + + p, result, cleanup, err := planengine.BuildPlan(c, helmClient, manifest, opts.Environment, cluster.Context(), resolvedSpec, postRenderer) defer cleanup() if err != nil { return fmt.Errorf("%w%s", err, common.ClusterHint(err)) } + if n := len(bundle.CRDs); n > 0 { + c.Printf("CRDs: %d pending from .deployah/crds/ (not applied in plan)\n", n) + } + if opts.Drift { if driftErr := checkDrift(c, cluster, p, result.Manifest); driftErr != nil { return fmt.Errorf("check drift: %w%s", driftErr, common.ClusterHint(driftErr)) diff --git a/internal/cmd/plan/plan_test.go b/internal/cmd/plan/plan_test.go index eb10335..6a6d36d 100644 --- a/internal/cmd/plan/plan_test.go +++ b/internal/cmd/plan/plan_test.go @@ -17,11 +17,14 @@ package plan import ( "bytes" "context" + "os" + "path/filepath" "testing" "time" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" + "helm.sh/helm/v4/pkg/postrenderer" "helm.sh/helm/v4/pkg/release/common" "k8s.io/apimachinery/pkg/labels" "nabat.dev/nabat" @@ -56,14 +59,14 @@ type stubHelmClient struct { func (s *stubHelmClient) IsReachable() error { return s.reachableErr } -func (s *stubHelmClient) RenderManifests(context.Context, *spec.Spec, string, *spec.ResolvedSpec) (*render.RenderResult, func(), error) { +func (s *stubHelmClient) RenderManifests(context.Context, *spec.Spec, string, *spec.ResolvedSpec, postrenderer.PostRenderer) (*render.RenderResult, func(), error) { if s.renderErr != nil { return nil, nil, s.renderErr } return s.renderResult, func() {}, nil } -func (s *stubHelmClient) RenderOffline(context.Context, *spec.Spec, string, *spec.ResolvedSpec) (*render.RenderResult, func(), error) { +func (s *stubHelmClient) RenderOffline(context.Context, *spec.Spec, string, *spec.ResolvedSpec, postrenderer.PostRenderer) (*render.RenderResult, func(), error) { if s.offlineErr != nil { return nil, nil, s.offlineErr } @@ -77,7 +80,7 @@ func (s *stubHelmClient) GetReleaseHistory(context.Context, string, string) ([]* return s.history, nil } -func (s *stubHelmClient) InstallApp(context.Context, *spec.Spec, string, bool, *spec.ResolvedSpec) error { +func (s *stubHelmClient) InstallApp(context.Context, *spec.Spec, string, bool, *spec.ResolvedSpec, postrenderer.PostRenderer) error { panic("unexpected InstallApp call") } @@ -309,7 +312,7 @@ func TestRunOnline(t *testing.T) { opts := testOptions() opts.DetailedExitCode = tt.detailed - err := runOnline(c, sess, testManifest(), opts, nil, theme.ResolvedTheme{}) + err := runOnline(c, sess, nil, testManifest(), opts, nil, theme.ResolvedTheme{}) if tt.wantErrIs != nil { require.Error(t, err) assert.ErrorIs(t, err, tt.wantErrIs) @@ -349,7 +352,7 @@ func TestRunOnline_DriftOnFreshInstall_NoStdoutFootprint(t *testing.T) { opts := testOptions() opts.Drift = true - err := runOnline(captured, sess, testManifest(), opts, nil, theme.ResolvedTheme{}) + err := runOnline(captured, sess, nil, testManifest(), opts, nil, theme.ResolvedTheme{}) require.NoError(t, err, "checkDrift must short-circuit cleanly without a working cluster config") assert.NotContains(t, out.String(), "Drift (cluster changed outside deployah):", @@ -373,9 +376,123 @@ func TestRunOffline_RendersResourceCount(t *testing.T) { opts := testOptions() opts.Offline = true - err := runOffline(c, testManifest(), opts, nil) + err := runOffline(c, sess, nil, testManifest(), opts, nil) require.NoError(t, err) assert.Contains(t, out.String(), "Rendered 2 resources for environment 'production' (no cluster comparison).") assert.Contains(t, out.String(), "validation: OK") } + +func writePlanExtras(t *testing.T, dir, relative, content string) { + t.Helper() + path := filepath.Join(dir, relative) + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o750)) + require.NoError(t, os.WriteFile(path, []byte(content), 0o600)) +} + +// TestRunOffline_PrintsPendingCRDs notes CRDs that plan will not apply. +func TestRunOffline_PrintsPendingCRDs(t *testing.T) { + t.Parallel() + dir := t.TempDir() + specPath := filepath.Join(dir, "deployah.yaml") + writePlanExtras(t, dir, "deployah.yaml", "apiVersion: deployah.dev/v1-alpha.2\nproject: web\n") + writePlanExtras(t, dir, ".deployah/crds/widget.yaml", ` +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.example.com +spec: + group: example.com + scope: Namespaced + names: + kind: Widget + plural: widgets + versions: + - name: v1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object +`) + stub := &stubHelmClient{offlineResult: renderResult(deploymentV1)} + sess := session.New( + session.WithSpecPath(specPath), + session.WithHelmFactory(func(*session.Session) (session.HelmClient, error) { + return stub, nil + }), + ) + c, out := nabatContext(t) + opts := testOptions() + opts.Offline = true + + err := runOffline(c, sess, nil, testManifest(), opts, nil) + require.NoError(t, err) + assert.Contains(t, out.String(), "CRDs: 1 pending from .deployah/crds/") +} + +// TestRunOffline_LoadExtrasError fails when .deployah YAML is invalid. +func TestRunOffline_LoadExtrasError(t *testing.T) { + t.Parallel() + dir := t.TempDir() + specPath := filepath.Join(dir, "deployah.yaml") + writePlanExtras(t, dir, "deployah.yaml", "apiVersion: deployah.dev/v1-alpha.2\nproject: web\n") + writePlanExtras(t, dir, ".deployah/manifests/bad.yaml", "not: [valid") + stub := &stubHelmClient{offlineResult: renderResult(deploymentV1)} + sess := session.New( + session.WithSpecPath(specPath), + session.WithHelmFactory(func(*session.Session) (session.HelmClient, error) { + return stub, nil + }), + ) + c, _ := nabatContext(t) + opts := testOptions() + opts.Offline = true + + err := runOffline(c, sess, nil, testManifest(), opts, nil) + require.Error(t, err) + assert.Contains(t, err.Error(), "load extras") +} + +// TestRunOnline_PrintsPendingCRDs notes CRDs ahead of the plan diff. +func TestRunOnline_PrintsPendingCRDs(t *testing.T) { + t.Parallel() + dir := t.TempDir() + specPath := filepath.Join(dir, "deployah.yaml") + writePlanExtras(t, dir, "deployah.yaml", "apiVersion: deployah.dev/v1-alpha.2\nproject: web\n") + writePlanExtras(t, dir, ".deployah/crds/widget.yaml", ` +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.example.com +spec: + group: example.com + scope: Namespaced + names: + kind: Widget + plural: widgets + versions: + - name: v1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object +`) + stub := &stubHelmClient{ + historyErr: helm.ErrReleaseNotFound, + renderResult: renderResult(deploymentV1), + } + sess := session.New( + session.WithSpecPath(specPath), + session.WithHelmFactory(func(*session.Session) (session.HelmClient, error) { + return stub, nil + }), + ) + c, out := nabatContext(t) + c.SetContext(session.WithContext(c.Context(), sess)) + + err := runOnline(c, sess, nil, testManifest(), testOptions(), nil, theme.ResolvedTheme{}) + require.NoError(t, err) + assert.Contains(t, out.String(), "CRDs: 1 pending from .deployah/crds/") +} diff --git a/internal/extras/bundle.go b/internal/extras/bundle.go new file mode 100644 index 0000000..bc33f0c --- /dev/null +++ b/internal/extras/bundle.go @@ -0,0 +1,53 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras + +import ( + "path/filepath" + + "helm.sh/helm/v4/pkg/postrenderer" + "k8s.io/client-go/rest" + + "deployah.dev/deployah/internal/spec" +) + +// LoadFromSpec loads extras for a deploy/plan of the given spec. +// specPath is the path to deployah.yaml. When cfg is non-nil, live discovery +// is used for scope resolution; otherwise Offline is set so unknown types are +// not rejected (scope still comes from the built-in table and in-repo CRDs). +func LoadFromSpec(specPath string, spc *spec.Spec, platform *spec.PlatformConfig, environment, releaseNamespace string, cfg *rest.Config) (*Bundle, error) { + scope, err := NewDiscoveryResolver(cfg, nil) + if err != nil { + return nil, err + } + return Load(LoadConfig{ + SpecDir: filepath.Dir(specPath), + Project: spc.Project, + Environment: environment, + DeclaredEnvs: spec.DeclaredEnvironments(spc.Environments, platform), + ReleaseNamespace: releaseNamespace, + Scope: scope, + Offline: cfg == nil, + }) +} + +// PostRendererFor returns a Helm post-renderer for the bundle's manifests, or +// a true nil interface when there are none (avoids a typed-nil *PostRenderer). +func (b *Bundle) PostRendererFor() postrenderer.PostRenderer { + if b == nil || len(b.Manifests) == 0 { + return nil + } + return &PostRenderer{Manifests: b.Manifests} +} diff --git a/internal/extras/crd.go b/internal/extras/crd.go new file mode 100644 index 0000000..e5325c8 --- /dev/null +++ b/internal/extras/crd.go @@ -0,0 +1,228 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "time" + + "k8s.io/apimachinery/pkg/types" + "k8s.io/apimachinery/pkg/util/wait" + "k8s.io/client-go/rest" + + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apiextensionsclient "k8s.io/apiextensions-apiserver/pkg/client/clientset/clientset" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + sigsyaml "sigs.k8s.io/yaml" +) + +// Policy controls how Deployah installs CRDs from .deployah/crds/. +type Policy string + +const ( + // PolicyCreate installs a CRD only when it does not already exist. + PolicyCreate Policy = "create" + // PolicyCreateReplace creates a missing CRD or server-side-applies over + // an existing one (force ownership). + PolicyCreateReplace Policy = "create-replace" + + crdFieldManager = "deployah" +) + +// CRDStats summarizes what [ApplyCRDs] did for one deploy. +type CRDStats struct { + // Created is how many CRDs were newly installed. + Created int + // Replaced is how many existing CRDs were server-side-applied. + Replaced int + // Ready is how many CRDs were waited on (created, replaced, or already present). + Ready int +} + +// crdClient is the subset of the apiextensions API used by ApplyCRDs. +type crdClient interface { + Create(ctx context.Context, crd *apiextensionsv1.CustomResourceDefinition, opts metav1.CreateOptions) (*apiextensionsv1.CustomResourceDefinition, error) + // Apply server-side-applies patch (JSON, content-type apply-patch+yaml) for name. + Apply(ctx context.Context, name string, patch []byte) (*apiextensionsv1.CustomResourceDefinition, error) + Get(ctx context.Context, name string, opts metav1.GetOptions) (*apiextensionsv1.CustomResourceDefinition, error) +} + +type crdClientAdapter struct { + inner apiextensionsclient.Interface +} + +func (a crdClientAdapter) Create(ctx context.Context, crd *apiextensionsv1.CustomResourceDefinition, opts metav1.CreateOptions) (*apiextensionsv1.CustomResourceDefinition, error) { + return a.inner.ApiextensionsV1().CustomResourceDefinitions().Create(ctx, crd, opts) +} + +func (a crdClientAdapter) Apply(ctx context.Context, name string, patch []byte) (*apiextensionsv1.CustomResourceDefinition, error) { + force := true + return a.inner.ApiextensionsV1().CustomResourceDefinitions().Patch( + ctx, + name, + types.ApplyPatchType, + patch, + metav1.PatchOptions{FieldManager: crdFieldManager, Force: &force}, + ) +} + +func (a crdClientAdapter) Get(ctx context.Context, name string, opts metav1.GetOptions) (*apiextensionsv1.CustomResourceDefinition, error) { + return a.inner.ApiextensionsV1().CustomResourceDefinitions().Get(ctx, name, opts) +} + +// ApplyCRDs installs CRDs according to policy, then waits for each to report +// Established. CRDs are never pruned. timeout bounds the wait. +// +// create-replace uses server-side apply with a patch derived from the object +// YAML (status and server-managed metadata stripped). See +// https://kubernetes.io/docs/reference/using-api/server-side-apply/ +func ApplyCRDs(ctx context.Context, cfg *rest.Config, crds []Object, policy Policy, timeout time.Duration) (CRDStats, error) { + var stats CRDStats + if len(crds) == 0 { + return stats, nil + } + if cfg == nil { + return stats, fmt.Errorf("apply CRDs: cluster configuration is required") + } + switch policy { + case PolicyCreate, PolicyCreateReplace: + default: + return stats, fmt.Errorf("unknown CRD policy %q", policy) + } + cs, err := apiextensionsclient.NewForConfig(cfg) + if err != nil { + return stats, fmt.Errorf("build apiextensions client: %w", err) + } + return applyCRDs(ctx, crdClientAdapter{inner: cs}, crds, policy, timeout) +} + +func applyCRDs(ctx context.Context, client crdClient, crds []Object, policy Policy, timeout time.Duration) (CRDStats, error) { + var stats CRDStats + if timeout <= 0 { + timeout = 60 * time.Second + } + applied := make([]string, 0, len(crds)) + for i := range crds { + crd, err := decodeCRD(crds[i]) + if err != nil { + return stats, err + } + existing, getErr := client.Get(ctx, crd.Name, metav1.GetOptions{}) + switch { + case apierrors.IsNotFound(getErr): + if policy == PolicyCreateReplace { + patch, patchErr := ssaPatchFromObject(crds[i]) + if patchErr != nil { + return stats, patchErr + } + if _, applyErr := client.Apply(ctx, crd.Name, patch); applyErr != nil { + return stats, fmt.Errorf("apply CRD %s: %w", crd.Name, applyErr) + } + } else if _, createErr := client.Create(ctx, crd, metav1.CreateOptions{}); createErr != nil { + return stats, fmt.Errorf("create CRD %s: %w", crd.Name, createErr) + } + stats.Created++ + applied = append(applied, crd.Name) + case getErr != nil: + return stats, fmt.Errorf("get CRD %s: %w", crd.Name, getErr) + case policy == PolicyCreate: + applied = append(applied, existing.Name) + case policy == PolicyCreateReplace: + patch, patchErr := ssaPatchFromObject(crds[i]) + if patchErr != nil { + return stats, patchErr + } + if _, applyErr := client.Apply(ctx, crd.Name, patch); applyErr != nil { + return stats, fmt.Errorf("replace CRD %s: %w", crd.Name, applyErr) + } + stats.Replaced++ + applied = append(applied, crd.Name) + } + } + + deadline := time.Now().Add(timeout) + for _, name := range applied { + remaining := time.Until(deadline) + if remaining <= 0 { + return stats, fmt.Errorf("timed out waiting for CRD %s to become Established", name) + } + if err := waitEstablished(ctx, client, name, remaining); err != nil { + switch { + case errors.Is(err, context.DeadlineExceeded): + return stats, fmt.Errorf("timed out waiting for CRD %s to become Established: %w", name, err) + case errors.Is(err, context.Canceled): + return stats, fmt.Errorf("waiting for CRD %s canceled: %w", name, err) + default: + return stats, fmt.Errorf("wait for CRD %s to become Established: %w", name, err) + } + } + stats.Ready++ + } + return stats, nil +} + +func decodeCRD(o Object) (*apiextensionsv1.CustomResourceDefinition, error) { + var crd apiextensionsv1.CustomResourceDefinition + if err := sigsyaml.Unmarshal(o.Raw, &crd); err != nil { + return nil, fmt.Errorf("%s: decode CRD: %w", o.Path, err) + } + if crd.Name == "" { + return nil, fmt.Errorf("%s: CRD metadata.name is empty", o.Path) + } + return &crd, nil +} + +// ssaPatchFromObject builds a server-side apply body from the object's YAML. +// Status and server-managed metadata are stripped so the patch matches the +// intended document rather than a typed round-trip full of null fields. +func ssaPatchFromObject(o Object) ([]byte, error) { + var obj map[string]any + if err := sigsyaml.Unmarshal(o.Raw, &obj); err != nil { + return nil, fmt.Errorf("%s: decode for apply: %w", o.Path, err) + } + delete(obj, "status") + if meta, ok := obj["metadata"].(map[string]any); ok { + delete(meta, "managedFields") + delete(meta, "resourceVersion") + delete(meta, "uid") + delete(meta, "creationTimestamp") + delete(meta, "generation") + delete(meta, "selfLink") + } + patch, err := json.Marshal(obj) + if err != nil { + return nil, fmt.Errorf("%s: marshal apply patch: %w", o.Path, err) + } + return patch, nil +} + +func waitEstablished(ctx context.Context, client crdClient, name string, timeout time.Duration) error { + return wait.PollUntilContextTimeout(ctx, 200*time.Millisecond, timeout, true, func(ctx context.Context) (bool, error) { + crd, err := client.Get(ctx, name, metav1.GetOptions{}) + if err != nil { + return false, err + } + for _, cond := range crd.Status.Conditions { + if cond.Type == apiextensionsv1.Established && cond.Status == apiextensionsv1.ConditionTrue { + return true, nil + } + } + return false, nil + }) +} diff --git a/internal/extras/crd_test.go b/internal/extras/crd_test.go new file mode 100644 index 0000000..bbb46f9 --- /dev/null +++ b/internal/extras/crd_test.go @@ -0,0 +1,423 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras + +import ( + "context" + "encoding/json" + "errors" + "sync" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/client-go/rest" + + apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" + apierrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +type fakeCRDClient struct { + mu sync.Mutex + objects map[string]*apiextensionsv1.CustomResourceDefinition + creates int + applies int + getCalls int + // establishAfterNGets makes Established true after N Get calls per name. + establishAfterNGets int + getsPerName map[string]int + // failGetAfter, when > 0, makes Get return an API error after that many calls. + failGetAfter int + lastPatch []byte + createErr error + applyErr error + getErr error +} + +func newFakeCRDClient() *fakeCRDClient { + return &fakeCRDClient{ + objects: make(map[string]*apiextensionsv1.CustomResourceDefinition), + getsPerName: make(map[string]int), + } +} + +func (f *fakeCRDClient) Create(_ context.Context, crd *apiextensionsv1.CustomResourceDefinition, _ metav1.CreateOptions) (*apiextensionsv1.CustomResourceDefinition, error) { + f.mu.Lock() + defer f.mu.Unlock() + if f.createErr != nil { + return nil, f.createErr + } + if _, ok := f.objects[crd.Name]; ok { + return nil, apierrors.NewAlreadyExists(schema.GroupResource{Group: "apiextensions.k8s.io", Resource: "customresourcedefinitions"}, crd.Name) + } + cp := crd.DeepCopy() + f.objects[crd.Name] = cp + f.creates++ + return cp.DeepCopy(), nil +} + +func (f *fakeCRDClient) Apply(_ context.Context, name string, patch []byte) (*apiextensionsv1.CustomResourceDefinition, error) { + f.mu.Lock() + defer f.mu.Unlock() + if f.applyErr != nil { + return nil, f.applyErr + } + var obj map[string]any + if err := json.Unmarshal(patch, &obj); err != nil { + return nil, err + } + if _, hasStatus := obj["status"]; hasStatus { + return nil, errors.New("apply patch must not include status") + } + if meta, ok := obj["metadata"].(map[string]any); ok { + if _, has := meta["managedFields"]; has { + return nil, errors.New("apply patch must not include managedFields") + } + if _, has := meta["resourceVersion"]; has { + return nil, errors.New("apply patch must not include resourceVersion") + } + } + f.lastPatch = append([]byte(nil), patch...) + crd := &apiextensionsv1.CustomResourceDefinition{ + ObjectMeta: metav1.ObjectMeta{Name: name}, + } + f.objects[name] = crd + f.applies++ + return crd.DeepCopy(), nil +} + +func (f *fakeCRDClient) Get(_ context.Context, name string, _ metav1.GetOptions) (*apiextensionsv1.CustomResourceDefinition, error) { + f.mu.Lock() + defer f.mu.Unlock() + f.getCalls++ + if f.getErr != nil { + return nil, f.getErr + } + if f.failGetAfter > 0 && f.getCalls > f.failGetAfter { + return nil, errors.New("api unavailable") + } + obj, ok := f.objects[name] + if !ok { + return nil, apierrors.NewNotFound(schema.GroupResource{Group: "apiextensions.k8s.io", Resource: "customresourcedefinitions"}, name) + } + cp := obj.DeepCopy() + f.getsPerName[name]++ + if f.establishAfterNGets <= 0 || f.getsPerName[name] >= f.establishAfterNGets { + cp.Status.Conditions = []apiextensionsv1.CustomResourceDefinitionCondition{{ + Type: apiextensionsv1.Established, + Status: apiextensionsv1.ConditionTrue, + }} + } + return cp, nil +} + +func sampleCRDObject(t *testing.T, name string) Object { + t.Helper() + obj := &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "apiextensions.k8s.io/v1", + "kind": "CustomResourceDefinition", + "metadata": map[string]any{ + "name": name, + "resourceVersion": "99", + "uid": "abc", + "creationTimestamp": "2020-01-01T00:00:00Z", + "managedFields": []any{map[string]any{"manager": "other"}}, + }, + "status": map[string]any{ + "conditions": []any{}, + }, + "spec": map[string]any{ + "group": "example.com", + "scope": "Namespaced", + "names": map[string]any{ + "kind": "Widget", + "plural": "widgets", + }, + "versions": []any{ + map[string]any{ + "name": "v1", + "served": true, + "storage": true, + "schema": map[string]any{ + "openAPIV3Schema": map[string]any{"type": "object"}, + }, + }, + }, + }, + }} + o := Object{Path: name + ".yaml", Obj: obj} + raw, err := o.MarshalYAML() + require.NoError(t, err) + o.Raw = raw + return o +} + +// TestApplyCRDs_CreateIfMissing exercises extras package behavior. +func TestApplyCRDs_CreateIfMissing(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.establishAfterNGets = 1 + existing := &apiextensionsv1.CustomResourceDefinition{ + ObjectMeta: metav1.ObjectMeta{Name: "existing.example.com", ResourceVersion: "1"}, + } + client.objects[existing.Name] = existing + + stats, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "existing.example.com"), + sampleCRDObject(t, "new.example.com"), + }, PolicyCreate, 2*time.Second) + require.NoError(t, err) + assert.Equal(t, 1, client.creates) + assert.Equal(t, 0, client.applies) + assert.Equal(t, 1, stats.Created) + assert.Equal(t, 0, stats.Replaced) + assert.Equal(t, 2, stats.Ready) + assert.Contains(t, client.objects, "new.example.com") +} + +// TestApplyCRDs_CreateReplaceAppliesExisting exercises extras package behavior. +func TestApplyCRDs_CreateReplaceAppliesExisting(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.establishAfterNGets = 1 + client.objects["widgets.example.com"] = &apiextensionsv1.CustomResourceDefinition{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets.example.com", ResourceVersion: "7"}, + } + + stats, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "widgets.example.com"), + }, PolicyCreateReplace, 2*time.Second) + require.NoError(t, err) + assert.Equal(t, 0, client.creates) + assert.Equal(t, 1, client.applies) + assert.Equal(t, 0, stats.Created) + assert.Equal(t, 1, stats.Replaced) + assert.Equal(t, 1, stats.Ready) + require.NotEmpty(t, client.lastPatch) + assert.NotContains(t, string(client.lastPatch), `"status"`) + assert.NotContains(t, string(client.lastPatch), `"managedFields"`) + assert.NotContains(t, string(client.lastPatch), `"resourceVersion"`) +} + +// TestApplyCRDs_CreateReplaceAppliesMissing exercises extras package behavior. +func TestApplyCRDs_CreateReplaceAppliesMissing(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.establishAfterNGets = 1 + + stats, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "new.example.com"), + }, PolicyCreateReplace, 2*time.Second) + require.NoError(t, err) + assert.Equal(t, 0, client.creates) + assert.Equal(t, 1, client.applies) + assert.Equal(t, 1, stats.Created) + assert.Equal(t, 0, stats.Replaced) + assert.Contains(t, client.objects, "new.example.com") +} + +// TestApplyCRDs_WaitsForEstablished exercises extras package behavior. +func TestApplyCRDs_WaitsForEstablished(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.establishAfterNGets = 3 + + _, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "slow.example.com"), + }, PolicyCreate, 2*time.Second) + require.NoError(t, err) + assert.GreaterOrEqual(t, client.getsPerName["slow.example.com"], 3) +} + +// TestApplyCRDs_Timeout exercises extras package behavior. +func TestApplyCRDs_Timeout(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.establishAfterNGets = 1_000_000 + + _, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "never.example.com"), + }, PolicyCreate, 300*time.Millisecond) + require.Error(t, err) + assert.Contains(t, err.Error(), "timed out") +} + +// TestApplyCRDs_WaitAPIError exercises extras package behavior. +func TestApplyCRDs_WaitAPIError(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.objects["broken.example.com"] = &apiextensionsv1.CustomResourceDefinition{ + ObjectMeta: metav1.ObjectMeta{Name: "broken.example.com"}, + } + client.failGetAfter = 1 + + _, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "broken.example.com"), + }, PolicyCreate, time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "wait for CRD") + assert.NotContains(t, err.Error(), "timed out") +} + +// TestApplyCRDs_Canceled exercises extras package behavior. +func TestApplyCRDs_Canceled(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.establishAfterNGets = 1_000_000 + ctx, cancel := context.WithCancel(t.Context()) + cancel() + + _, err := applyCRDs(ctx, client, []Object{ + sampleCRDObject(t, "cancel.example.com"), + }, PolicyCreate, time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "canceled") + assert.NotContains(t, err.Error(), "timed out") +} + +// TestApplyCRDs_EmptyNoop exercises extras package behavior. +func TestApplyCRDs_EmptyNoop(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + stats, err := applyCRDs(t.Context(), client, nil, PolicyCreate, time.Second) + require.NoError(t, err) + assert.Equal(t, 0, client.creates) + assert.Equal(t, CRDStats{}, stats) +} + +// TestSSAPatchFromObject_StripsServerFields exercises extras package behavior. +func TestSSAPatchFromObject_StripsServerFields(t *testing.T) { + t.Parallel() + o := sampleCRDObject(t, "widgets.example.com") + patch, err := ssaPatchFromObject(o) + require.NoError(t, err) + var obj map[string]any + require.NoError(t, json.Unmarshal(patch, &obj)) + _, hasStatus := obj["status"] + assert.False(t, hasStatus) + meta, ok := obj["metadata"].(map[string]any) + require.True(t, ok) + _, hasMF := meta["managedFields"] + assert.False(t, hasMF) + _, hasRV := meta["resourceVersion"] + assert.False(t, hasRV) + assert.Equal(t, "widgets.example.com", meta["name"]) + spec, ok := obj["spec"].(map[string]any) + require.True(t, ok) + assert.Equal(t, "example.com", spec["group"]) +} + +// TestApplyCRDs_ExportedGuards covers the public ApplyCRDs entry points that +// never reach the fake client. +func TestApplyCRDs_ExportedGuards(t *testing.T) { + t.Parallel() + ctx := t.Context() + crd := sampleCRDObject(t, "widgets.example.com") + + stats, err := ApplyCRDs(ctx, nil, nil, PolicyCreate, time.Second) + require.NoError(t, err) + assert.Equal(t, CRDStats{}, stats) + + _, err = ApplyCRDs(ctx, nil, []Object{crd}, PolicyCreate, time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "cluster configuration is required") + + _, err = ApplyCRDs(ctx, &rest.Config{Host: "https://127.0.0.1:1"}, []Object{crd}, Policy("nope"), time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "unknown CRD policy") +} + +// TestApplyCRDs_DecodeAndClientErrors covers decode/get/create/apply failures. +func TestApplyCRDs_DecodeAndClientErrors(t *testing.T) { + t.Parallel() + + t.Run("decode error", func(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + _, err := applyCRDs(t.Context(), client, []Object{{ + Path: "bad.yaml", + Raw: []byte("not: [valid"), + }}, PolicyCreate, time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "decode CRD") + }) + + t.Run("empty name", func(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + _, err := applyCRDs(t.Context(), client, []Object{{ + Path: "noname.yaml", + Raw: []byte(`apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: {} +spec: + group: example.com +`), + }}, PolicyCreate, time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "metadata.name is empty") + }) + + t.Run("get error", func(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.getErr = errors.New("forbidden") + _, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "widgets.example.com"), + }, PolicyCreate, time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "get CRD") + }) + + t.Run("create error", func(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.createErr = errors.New("quota exceeded") + _, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "widgets.example.com"), + }, PolicyCreate, time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "create CRD") + }) + + t.Run("apply error on missing", func(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.applyErr = errors.New("ssa rejected") + _, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "widgets.example.com"), + }, PolicyCreateReplace, time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "apply CRD") + }) + + t.Run("replace error on existing", func(t *testing.T) { + t.Parallel() + client := newFakeCRDClient() + client.objects["widgets.example.com"] = &apiextensionsv1.CustomResourceDefinition{ + ObjectMeta: metav1.ObjectMeta{Name: "widgets.example.com"}, + } + client.applyErr = errors.New("ssa rejected") + _, err := applyCRDs(t.Context(), client, []Object{ + sampleCRDObject(t, "widgets.example.com"), + }, PolicyCreateReplace, time.Second) + require.Error(t, err) + assert.Contains(t, err.Error(), "replace CRD") + }) +} diff --git a/internal/extras/doc.go b/internal/extras/doc.go new file mode 100644 index 0000000..d211e9b --- /dev/null +++ b/internal/extras/doc.go @@ -0,0 +1,18 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Package extras loads raw Kubernetes manifests and CRDs from .deployah/, +// merges Deployah identity metadata, injects manifests into a Helm release +// via a post-renderer, and applies CRDs to the cluster before the release. +package extras diff --git a/internal/extras/load.go b/internal/extras/load.go new file mode 100644 index 0000000..a87a031 --- /dev/null +++ b/internal/extras/load.go @@ -0,0 +1,352 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras + +import ( + "bytes" + "errors" + "fmt" + "io" + "io/fs" + "maps" + "os" + "path/filepath" + "slices" + "strings" + + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + + "deployah.dev/deployah/internal/spec" + + yamlutil "k8s.io/apimachinery/pkg/util/yaml" +) + +// crdAPIVersion is the only CustomResourceDefinition apiVersion Deployah +// accepts. apiextensions.k8s.io/v1beta1 was removed in Kubernetes 1.22. +const crdAPIVersion = "apiextensions.k8s.io/v1" + +// Bundle holds the deploy-ready extras for one environment. +type Bundle struct { + // Manifests are objects from .deployah/manifests/ for the selected environment. + Manifests []Object + // CRDs are CustomResourceDefinition objects from .deployah/crds/. + CRDs []Object +} + +// LoadConfig configures [Load]. +type LoadConfig struct { + // SpecDir is the directory containing deployah.yaml (and .deployah/). + SpecDir string + // Project is the project name for identity annotations/labels. + Project string + // Environment is the runtime environment (e.g. review/pr-123). + Environment string + // DeclaredEnvs are the registry keys used to validate manifests// dirs. + DeclaredEnvs []string + // ReleaseNamespace fills empty metadata.namespace on namespaced objects. + ReleaseNamespace string + // Scope resolves namespaced vs cluster-scoped. Required. + Scope ScopeResolver + // Offline is true when cluster discovery is unavailable (plan --offline + // or missing rest config). Unknown types are then allowed; scope defaults + // to namespaced unless an in-repo CRD declares otherwise. + Offline bool +} + +// Load reads .deployah/manifests and .deployah/crds under SpecDir, validates +// them, merges Deployah identity metadata, and returns a deploy-ready Bundle. +// Missing directories yield an empty Bundle with a nil error. +func Load(cfg LoadConfig) (*Bundle, error) { + if cfg.Scope == nil { + return nil, errors.New("extras: ScopeResolver is required") + } + if cfg.Project == "" { + return nil, errors.New("extras: Project is required") + } + + root := filepath.Join(cfg.SpecDir, spec.DeployahConfigDir) + manifestsRoot := filepath.Join(root, spec.ManifestsDir) + crdsRoot := filepath.Join(root, spec.CRDsDir) + + envKey, _ := spec.MatchEnvKey(cfg.Environment, cfg.DeclaredEnvs) + envSafe := spec.NormalizeEnv(cfg.Environment).K8sSafe + + manifestFiles, err := listManifestFiles(manifestsRoot, cfg.DeclaredEnvs, envKey) + if err != nil { + return nil, err + } + crdFiles, err := listCRDFiles(crdsRoot) + if err != nil { + return nil, err + } + + var manifests []Object + for _, path := range manifestFiles { + objs, loadErr := loadFile(path, true) + if loadErr != nil { + return nil, loadErr + } + manifests = append(manifests, objs...) + } + + var crds []Object + for _, path := range crdFiles { + objs, loadErr := loadFile(path, false) + if loadErr != nil { + return nil, loadErr + } + for i := range objs { + if objs[i].Obj.GetKind() != "CustomResourceDefinition" { + return nil, fmt.Errorf("%s: only CustomResourceDefinition objects are allowed under .deployah/crds/ (found %s)", objs[i].Path, objs[i].Obj.GetKind()) + } + if av := objs[i].Obj.GetAPIVersion(); av != crdAPIVersion { + return nil, fmt.Errorf("%s: CustomResourceDefinition must use apiVersion %s (found %s)", objs[i].Path, crdAPIVersion, av) + } + crds = append(crds, objs[i]) + } + } + + crdScope := scopeFromCRDObjects(crds) + scope := withCRDScope(cfg.Scope, crdScope) + + for i := range manifests { + gvk := manifests[i].GVK() + known, knownErr := scope.Known(gvk) + if knownErr != nil { + return nil, fmt.Errorf("%s: resolve type: %w", manifests[i].Path, knownErr) + } + if !known && !cfg.Offline { + return nil, fmt.Errorf("%s: unknown type %s; add its CRD under .deployah/crds/ or install it on the cluster first", manifests[i].Path, gvk.String()) + } + namespaced, scopeErr := scope.Namespaced(gvk) + if scopeErr != nil { + return nil, fmt.Errorf("%s: resolve scope: %w", manifests[i].Path, scopeErr) + } + if mergeErr := mergeIdentity(&manifests[i], cfg.Project, envSafe, spec.SourceManifests, cfg.ReleaseNamespace, namespaced); mergeErr != nil { + return nil, mergeErr + } + } + for i := range crds { + if mergeErr := mergeIdentity(&crds[i], cfg.Project, "", spec.SourceCRDs, "", false); mergeErr != nil { + return nil, mergeErr + } + } + + if dupErr := checkDuplicateIdentities(manifests); dupErr != nil { + return nil, dupErr + } + if dupErr := checkDuplicateIdentities(crds); dupErr != nil { + return nil, dupErr + } + + return &Bundle{Manifests: manifests, CRDs: crds}, nil +} + +func checkDuplicateIdentities(objs []Object) error { + seen := make(map[string]string, len(objs)) + for i := range objs { + id := objs[i].Identity() + if prev, ok := seen[id.Key()]; ok { + return fmt.Errorf("duplicate object %s in %s and %s", id, prev, objs[i].Path) + } + seen[id.Key()] = objs[i].Path + } + return nil +} + +// withCRDScope returns a resolver that prefers CRD-derived scopes. +func withCRDScope(base ScopeResolver, crdScope map[string]bool) ScopeResolver { + if len(crdScope) == 0 { + return base + } + switch r := base.(type) { + case *TableResolver: + merged := make(map[string]bool, len(r.CRDScope)+len(crdScope)) + maps.Copy(merged, r.CRDScope) + maps.Copy(merged, crdScope) + return &TableResolver{CRDScope: merged} + case *DiscoveryResolver: + merged := make(map[string]bool, len(r.Table.CRDScope)+len(crdScope)) + maps.Copy(merged, r.Table.CRDScope) + maps.Copy(merged, crdScope) + return &DiscoveryResolver{Mapper: r.Mapper, Table: TableResolver{CRDScope: merged}} + default: + return &chainedScope{crd: &TableResolver{CRDScope: crdScope}, next: base} + } +} + +type chainedScope struct { + crd *TableResolver + next ScopeResolver +} + +func (c *chainedScope) Known(gvk schema.GroupVersionKind) (bool, error) { + if known, err := c.crd.Known(gvk); err != nil || known { + return known, err + } + return c.next.Known(gvk) +} + +func (c *chainedScope) Namespaced(gvk schema.GroupVersionKind) (bool, error) { + if c.crd.CRDScope != nil { + if ns, ok := c.crd.CRDScope[crdScopeKey(gvk.Group, gvk.Kind)]; ok { + return ns, nil + } + } + return c.next.Namespaced(gvk) +} + +func listManifestFiles(root string, declaredEnvs []string, envKey string) ([]string, error) { + entries, err := os.ReadDir(root) + if err != nil { + if errors.Is(err, fs.ErrNotExist) { + return nil, nil + } + return nil, fmt.Errorf("read %s: %w", root, err) + } + + var files []string + for _, e := range entries { + name := e.Name() + path := filepath.Join(root, name) + if e.IsDir() { + if !slices.Contains(declaredEnvs, name) { + return nil, fmt.Errorf("%s: unknown environment directory %q (must be a declared environment key)", path, name) + } + if envKey == "" || name != envKey { + continue + } + envFiles, listErr := listYAMLFiles(path) + if listErr != nil { + return nil, listErr + } + files = append(files, envFiles...) + continue + } + keep, fileErr := classifyFile(path, name) + if fileErr != nil { + return nil, fileErr + } + if keep { + files = append(files, path) + } + } + return files, nil +} + +func listCRDFiles(root string) ([]string, error) { + entries, err := os.ReadDir(root) + if err != nil { + if errors.Is(err, fs.ErrNotExist) { + return nil, nil + } + return nil, fmt.Errorf("read %s: %w", root, err) + } + for _, e := range entries { + if e.IsDir() { + return nil, fmt.Errorf("%s: subdirectories are not allowed under .deployah/crds/", filepath.Join(root, e.Name())) + } + } + return listYAMLFiles(root) +} + +func listYAMLFiles(dir string) ([]string, error) { + entries, err := os.ReadDir(dir) + if err != nil { + if errors.Is(err, fs.ErrNotExist) { + return nil, nil + } + return nil, fmt.Errorf("read %s: %w", dir, err) + } + var files []string + for _, e := range entries { + name := e.Name() + path := filepath.Join(dir, name) + if e.IsDir() { + return nil, fmt.Errorf("%s: nested directories are not allowed", path) + } + keep, fileErr := classifyFile(path, name) + if fileErr != nil { + return nil, fileErr + } + if keep { + files = append(files, path) + } + } + return files, nil +} + +// classifyFile returns whether the file should be loaded. Dotfiles and +// README/markdown are skipped; other non-YAML files fail. +func classifyFile(path, name string) (bool, error) { + if strings.HasPrefix(name, ".") { + return false, nil + } + lower := strings.ToLower(name) + if strings.HasPrefix(lower, "readme") || strings.HasSuffix(lower, ".md") { + return false, nil + } + ext := strings.ToLower(filepath.Ext(name)) + if ext == ".yaml" || ext == ".yml" { + return true, nil + } + return false, fmt.Errorf("%s: unsupported file (only .yaml/.yml are loaded; remove it or rename)", path) +} + +// loadFile parses a multi-doc YAML file into Objects. When rejectCRD is true, +// a CustomResourceDefinition document is an error. +func loadFile(path string, rejectCRD bool) ([]Object, error) { + data, err := os.ReadFile(path) // #nosec G304 -- path from extras dir listing under SpecDir + if err != nil { + return nil, fmt.Errorf("read %s: %w", path, err) + } + decoder := yamlutil.NewYAMLOrJSONDecoder(bytes.NewReader(data), 4096) + var out []Object + doc := 0 + for { + var obj unstructured.Unstructured + if decodeErr := decoder.Decode(&obj); decodeErr != nil { + if errors.Is(decodeErr, io.EOF) { + break + } + return nil, fmt.Errorf("%s: parse YAML: %w", path, decodeErr) + } + doc++ + if len(obj.Object) == 0 { + continue + } + apiVersion := obj.GetAPIVersion() + kind := obj.GetKind() + name := obj.GetName() + if apiVersion == "" || kind == "" || name == "" { + return nil, fmt.Errorf("%s: document %d missing required apiVersion, kind, or metadata.name", path, doc) + } + if rejectCRD && kind == "CustomResourceDefinition" { + return nil, fmt.Errorf("%s: CustomResourceDefinition belongs in .deployah/crds/, not .deployah/manifests/", path) + } + raw, marshalErr := sigsyamlMarshal(&obj) + if marshalErr != nil { + return nil, fmt.Errorf("%s: %w", path, marshalErr) + } + out = append(out, Object{Path: path, Raw: raw, Obj: obj.DeepCopy()}) + } + return out, nil +} + +func sigsyamlMarshal(obj *unstructured.Unstructured) ([]byte, error) { + o := &Object{Obj: obj} + return o.MarshalYAML() +} diff --git a/internal/extras/load_test.go b/internal/extras/load_test.go new file mode 100644 index 0000000..1d9a5cf --- /dev/null +++ b/internal/extras/load_test.go @@ -0,0 +1,741 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras_test + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/runtime/schema" + + "deployah.dev/deployah/internal/extras" + "deployah.dev/deployah/internal/spec" +) + +func writeFile(t *testing.T, path, content string) { + t.Helper() + require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o750)) + require.NoError(t, os.WriteFile(path, []byte(content), 0o600)) +} + +// TestLoad_MissingDirsEmptyBundle exercises extras package behavior. +func TestLoad_MissingDirsEmptyBundle(t *testing.T) { + t.Parallel() + dir := t.TempDir() + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.NoError(t, err) + assert.Empty(t, bundle.Manifests) + assert.Empty(t, bundle.CRDs) +} + +// TestLoad_FileSelection exercises extras package behavior. +func TestLoad_FileSelection(t *testing.T) { + t.Parallel() + dir := t.TempDir() + root := filepath.Join(dir, ".deployah", "manifests") + writeFile(t, filepath.Join(root, "ok.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: ok +`) + writeFile(t, filepath.Join(root, "README.md"), "# ignore") + writeFile(t, filepath.Join(root, ".old.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: hidden +`) + writeFile(t, filepath.Join(root, ".gitkeep"), "") + + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: &extras.TableResolver{}, + }) + require.NoError(t, err) + require.Len(t, bundle.Manifests, 1) + assert.Equal(t, "ok", bundle.Manifests[0].Obj.GetName()) +} + +// TestLoad_VisibleNonYAMLFails exercises extras package behavior. +func TestLoad_VisibleNonYAMLFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "notes.txt"), "nope") + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "unsupported file") +} + +// TestLoad_MultiDocAndEmptySkipped exercises extras package behavior. +func TestLoad_MultiDocAndEmptySkipped(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "multi.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: a +--- +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: b +`) + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.NoError(t, err) + require.Len(t, bundle.Manifests, 2) + names := []string{bundle.Manifests[0].Obj.GetName(), bundle.Manifests[1].Obj.GetName()} + assert.ElementsMatch(t, []string{"a", "b"}, names) +} + +// TestLoad_DuplicateIdentityFails exercises extras package behavior. +func TestLoad_DuplicateIdentityFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + cm := ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: shared +` + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "one.yaml"), cm) + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "prod", "two.yaml"), cm) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "duplicate object") + assert.Contains(t, err.Error(), "one.yaml") + assert.Contains(t, err.Error(), "two.yaml") +} + +// TestLoad_CRDInManifestsFails exercises extras package behavior. +func TestLoad_CRDInManifestsFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "bad.yaml"), ` +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.example.com +spec: + group: example.com + scope: Namespaced + names: + kind: Widget + plural: widgets + versions: + - name: v1 + served: true + storage: true +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), ".deployah/crds/") +} + +// TestLoad_UnknownEnvDirFails exercises extras package behavior. +func TestLoad_UnknownEnvDirFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "staging", "x.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: x +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "unknown environment directory") +} + +// TestLoad_EnvSubdirPrefixMatch exercises extras package behavior. +func TestLoad_EnvSubdirPrefixMatch(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "common.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: common +`) + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "review", "pr.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: review-only +`) + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "prod", "prod.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: prod-only +`) + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "review/pr-42", + DeclaredEnvs: []string{"prod", "review"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.NoError(t, err) + names := make([]string, 0, len(bundle.Manifests)) + for _, m := range bundle.Manifests { + names = append(names, m.Obj.GetName()) + } + assert.ElementsMatch(t, []string{"common", "review-only"}, names) +} + +// TestLoad_MergesIdentityAndFillsNamespace exercises extras package behavior. +func TestLoad_MergesIdentityAndFillsNamespace(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "cm.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: data + labels: + app: mine + deployah.dev/project: wrong + deployah.dev/managed-by: impostor + deployah.dev/version: "9" + annotations: + note: keep +`) + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "shop", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: &extras.TableResolver{}, + }) + require.NoError(t, err) + require.Len(t, bundle.Manifests, 1) + obj := bundle.Manifests[0].Obj + assert.Equal(t, "apps", obj.GetNamespace()) + assert.Equal(t, "data", obj.GetName()) + assert.Equal(t, "shop", obj.GetLabels()[spec.LabelProject]) + assert.Equal(t, "prod", obj.GetLabels()[spec.LabelEnvironment]) + assert.Equal(t, "mine", obj.GetLabels()["app"]) + assert.Equal(t, spec.SourceManifests, obj.GetAnnotations()[spec.AnnotationSource]) + assert.Equal(t, "shop", obj.GetAnnotations()[spec.AnnotationProject]) + assert.Equal(t, "keep", obj.GetAnnotations()["note"]) + assert.NotContains(t, obj.GetLabels(), spec.LabelComponent) + assert.NotContains(t, obj.GetLabels(), spec.LabelManagedBy) + assert.NotContains(t, obj.GetLabels(), spec.LabelVersion) +} + +// TestLoad_NamespaceMismatchFails exercises extras package behavior. +func TestLoad_NamespaceMismatchFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "cm.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: data + namespace: other +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "differs from release namespace") +} + +// TestLoad_ClusterScopedRejectsNamespace exercises extras package behavior. +func TestLoad_ClusterScopedRejectsNamespace(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "ns.yaml"), ` +apiVersion: v1 +kind: Namespace +metadata: + name: extra + namespace: oops +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "must not set metadata.namespace") +} + +// TestLoad_CRDsMergedWithoutEnvironment exercises extras package behavior. +func TestLoad_CRDsMergedWithoutEnvironment(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "crds", "widget.yaml"), ` +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.example.com + labels: + deployah.dev/environment: should-strip + deployah.dev/managed-by: impostor + keep: custom +spec: + group: example.com + scope: Namespaced + names: + kind: Widget + plural: widgets + versions: + - name: v1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object +`) + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.NoError(t, err) + require.Len(t, bundle.CRDs, 1) + obj := bundle.CRDs[0].Obj + assert.Equal(t, spec.SourceCRDs, obj.GetAnnotations()[spec.AnnotationSource]) + assert.Equal(t, "demo", obj.GetAnnotations()[spec.AnnotationProject]) + assert.Equal(t, "demo", obj.GetLabels()[spec.LabelProject]) + assert.Equal(t, "custom", obj.GetLabels()["keep"]) + assert.NotContains(t, obj.GetLabels(), spec.LabelEnvironment) + assert.NotContains(t, obj.GetLabels(), spec.LabelManagedBy) +} + +// TestLoad_MissingRequiredFieldsFails exercises extras package behavior. +func TestLoad_MissingRequiredFieldsFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "bad.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: {} +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "missing required") +} + +// TestLoad_DuplicateAfterNamespaceFillFails exercises extras package behavior. +func TestLoad_DuplicateAfterNamespaceFillFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "empty-ns.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: shared +`) + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "filled-ns.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: shared + namespace: apps +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "duplicate object") +} + +// TestLoad_UnknownTypeFails exercises extras package behavior. +func TestLoad_UnknownTypeFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "widget.yaml"), ` +apiVersion: example.com/v1 +kind: Widget +metadata: + name: one +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "unknown type") + assert.Contains(t, err.Error(), ".deployah/crds/") +} + +// TestLoad_OfflineAllowsUnknownType lets plan --offline load custom +// resources without discovery or an in-repo CRD (scope defaults to namespaced). +func TestLoad_OfflineAllowsUnknownType(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "widget.yaml"), ` +apiVersion: example.com/v1 +kind: Widget +metadata: + name: one +`) + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: &extras.TableResolver{}, + Offline: true, + }) + require.NoError(t, err) + require.Len(t, bundle.Manifests, 1) + assert.Equal(t, "apps", bundle.Manifests[0].Obj.GetNamespace()) +} + +// TestLoad_CRDWrongAPIVersionFails rejects non-v1 CRD apiVersions at load. +func TestLoad_CRDWrongAPIVersionFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "crds", "old.yaml"), ` +apiVersion: example.com/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.example.com +spec: + group: example.com +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "apiextensions.k8s.io/v1") + assert.Contains(t, err.Error(), "example.com/v1") +} + +// TestLoad_RequiresScopeAndProject rejects incomplete LoadConfig. +func TestLoad_RequiresScopeAndProject(t *testing.T) { + t.Parallel() + _, err := extras.Load(extras.LoadConfig{Project: "demo"}) + require.Error(t, err) + assert.Contains(t, err.Error(), "ScopeResolver is required") + + _, err = extras.Load(extras.LoadConfig{Scope: &extras.TableResolver{}}) + require.Error(t, err) + assert.Contains(t, err.Error(), "Project is required") +} + +// TestLoad_NonCRDUnderCRDsFails rejects ordinary objects in .deployah/crds/. +func TestLoad_NonCRDUnderCRDsFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "crds", "cm.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: nope +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "only CustomResourceDefinition") +} + +// TestLoad_CRDSubdirForbidden rejects nested dirs under .deployah/crds/. +func TestLoad_CRDSubdirForbidden(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "crds", "nested", "x.yaml"), ` +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.example.com +spec: + group: example.com + scope: Namespaced + names: + kind: Widget + plural: widgets + versions: + - name: v1 + served: true + storage: true +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "subdirectories are not allowed") +} + +// TestLoad_NestedManifestDirFails rejects nesting under an env subdir. +func TestLoad_NestedManifestDirFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "prod", "nested", "x.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: x +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "nested directories are not allowed") +} + +// TestLoad_InvalidYAMLFails surfaces parse errors with the file path. +func TestLoad_InvalidYAMLFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "bad.yaml"), "not: [valid") + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "bad.yaml") +} + +// TestLoad_EmptyReleaseNamespaceFails when a namespaced object needs a fill. +func TestLoad_EmptyReleaseNamespaceFails(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "cm.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: data +`) + _, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "", + Scope: &extras.TableResolver{}, + }) + require.Error(t, err) + assert.Contains(t, err.Error(), "no release namespace") +} + +type rejectAllScope struct{} + +func (rejectAllScope) Known(schema.GroupVersionKind) (bool, error) { return false, nil } +func (rejectAllScope) Namespaced(schema.GroupVersionKind) (bool, error) { + return true, nil +} + +// TestLoad_ChainedScopeFromCustomResolver covers withCRDScope's default branch. +func TestLoad_ChainedScopeFromCustomResolver(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "crds", "widget.yaml"), ` +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.example.com +spec: + group: example.com + scope: Namespaced + names: + kind: Widget + plural: widgets + versions: + - name: v1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object +`) + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "w.yaml"), ` +apiVersion: example.com/v1 +kind: Widget +metadata: + name: one +`) + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: rejectAllScope{}, + }) + require.NoError(t, err) + require.Len(t, bundle.Manifests, 1) + assert.Equal(t, "apps", bundle.Manifests[0].Obj.GetNamespace()) +} + +// TestLoad_DiscoveryResolverMergesCRDScope covers the DiscoveryResolver branch +// of withCRDScope (mapper nil falls back to the table + CRD scope). +func TestLoad_DiscoveryResolverMergesCRDScope(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "crds", "widget.yaml"), ` +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: widgets.example.com +spec: + group: example.com + scope: Namespaced + names: + kind: Widget + plural: widgets + versions: + - name: v1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object +`) + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "w.yaml"), ` +apiVersion: example.com/v1 +kind: Widget +metadata: + name: one +`) + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: &extras.DiscoveryResolver{}, + }) + require.NoError(t, err) + require.Len(t, bundle.Manifests, 1) +} + +// TestLoadFromSpec_Offline loads extras without a rest.Config. +func TestLoadFromSpec_Offline(t *testing.T) { + t.Parallel() + dir := t.TempDir() + specPath := filepath.Join(dir, "deployah.yaml") + writeFile(t, specPath, "apiVersion: deployah.dev/v1-alpha.2\nproject: demo\n") + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "cm.yaml"), ` +apiVersion: v1 +kind: ConfigMap +metadata: + name: data +`) + bundle, err := extras.LoadFromSpec(specPath, &spec.Spec{Project: "demo"}, nil, "prod", "apps", nil) + require.NoError(t, err) + require.Len(t, bundle.Manifests, 1) + assert.NotNil(t, bundle.PostRendererFor()) +} diff --git a/internal/extras/merge.go b/internal/extras/merge.go new file mode 100644 index 0000000..798d713 --- /dev/null +++ b/internal/extras/merge.go @@ -0,0 +1,90 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras + +import ( + "fmt" + "strings" + + "deployah.dev/deployah/internal/spec" +) + +// mergeIdentity applies Deployah identity labels/annotations and fills +// namespace for namespaced objects. Reserved deployah.dev/* keys always win; +// other common keys lose to the manifest. Names are never rewritten. +func mergeIdentity(o *Object, project, environment, source, releaseNamespace string, namespaced bool) error { + if namespaced { + ns := o.Obj.GetNamespace() + if ns == "" { + if releaseNamespace == "" { + return fmt.Errorf("%s: namespaced object has empty metadata.namespace and no release namespace is set", o.Path) + } + o.Obj.SetNamespace(releaseNamespace) + } else if releaseNamespace != "" && ns != releaseNamespace { + return fmt.Errorf("%s: metadata.namespace %q differs from release namespace %q", o.Path, ns, releaseNamespace) + } + } else if o.Obj.GetNamespace() != "" { + // Cluster-scoped objects must not carry a namespace. + return fmt.Errorf("%s: cluster-scoped %s must not set metadata.namespace", o.Path, o.Obj.GetKind()) + } + + // Labels: reserved keys we own overwrite; other deployah.dev/* keys are + // stripped so extras cannot impersonate reserved semantics. Never set + // component on extras. + labels := getMetaStringMap(o.Obj, "labels") + if labels == nil { + labels = map[string]string{} + } + labels[spec.LabelProject] = project + keepEnv := source == spec.SourceManifests && environment != "" + if keepEnv { + labels[spec.LabelEnvironment] = environment + } + for k := range labels { + if !strings.HasPrefix(k, spec.LabelPrefix+"/") { + continue + } + if k == spec.LabelProject { + continue + } + if keepEnv && k == spec.LabelEnvironment { + continue + } + delete(labels, k) + } + o.Obj.SetLabels(labels) + + annotations := getMetaStringMap(o.Obj, "annotations") + if annotations == nil { + annotations = map[string]string{} + } + annotations[spec.AnnotationSource] = source + annotations[spec.AnnotationProject] = project + // Strip any other reserved deployah.dev/* annotation keys the user set + // that we do not own, so they cannot impersonate reserved semantics. + for k := range annotations { + if strings.HasPrefix(k, spec.LabelPrefix+"/") && k != spec.AnnotationSource && k != spec.AnnotationProject { + delete(annotations, k) + } + } + o.Obj.SetAnnotations(annotations) + + raw, err := o.MarshalYAML() + if err != nil { + return err + } + o.Raw = raw + return nil +} diff --git a/internal/extras/object.go b/internal/extras/object.go new file mode 100644 index 0000000..6b71bdb --- /dev/null +++ b/internal/extras/object.go @@ -0,0 +1,98 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras + +import ( + "fmt" + "strings" + + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + + sigsyaml "sigs.k8s.io/yaml" +) + +// Identity uniquely identifies a Kubernetes object for collision checks. +type Identity struct { + APIVersion string + Kind string + Namespace string + Name string +} + +// String returns a human-readable identity for error messages. +func (id Identity) String() string { + ns := id.Namespace + if ns == "" { + ns = "(cluster)" + } + return fmt.Sprintf("%s/%s %s/%s", id.APIVersion, id.Kind, ns, id.Name) +} + +// Key returns a stable map key for identity comparisons. +func (id Identity) Key() string { + return strings.Join([]string{id.APIVersion, id.Kind, id.Namespace, id.Name}, "\x00") +} + +// Object is one Kubernetes document loaded from an extras file. +type Object struct { + Path string + Raw []byte + Obj *unstructured.Unstructured +} + +// Identity returns the object's identity, using an empty namespace for +// cluster-scoped resources that have none set. +func (o *Object) Identity() Identity { + return Identity{ + APIVersion: o.Obj.GetAPIVersion(), + Kind: o.Obj.GetKind(), + Namespace: o.Obj.GetNamespace(), + Name: o.Obj.GetName(), + } +} + +// GVK returns the object's GroupVersionKind. +func (o *Object) GVK() schema.GroupVersionKind { + return o.Obj.GroupVersionKind() +} + +// MarshalYAML serializes the object back to YAML bytes. +func (o *Object) MarshalYAML() ([]byte, error) { + out, err := sigsyaml.Marshal(o.Obj.Object) + if err != nil { + return nil, fmt.Errorf("marshal %s: %w", o.Identity(), err) + } + return out, nil +} + +// getMetaStringMap returns metadata. as map[string]string. +func getMetaStringMap(obj *unstructured.Unstructured, field string) map[string]string { + meta, ok := obj.Object["metadata"].(map[string]any) + if !ok || meta == nil { + return nil + } + raw, ok := meta[field].(map[string]any) + if !ok || raw == nil { + return nil + } + out := make(map[string]string, len(raw)) + for k, v := range raw { + if s, isString := v.(string); isString { + out[k] = s + } + } + return out +} diff --git a/internal/extras/postrenderer.go b/internal/extras/postrenderer.go new file mode 100644 index 0000000..2b9aba0 --- /dev/null +++ b/internal/extras/postrenderer.go @@ -0,0 +1,107 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras + +import ( + "bytes" + "errors" + "fmt" + "io" + + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + + yamlutil "k8s.io/apimachinery/pkg/util/yaml" +) + +// PostRenderer appends literal extra manifests to Helm's rendered stream. +// Extra YAML never passes through the template engine, so `{{ }}` is preserved. +type PostRenderer struct { + Manifests []Object +} + +// Run implements helm.sh/helm/v4/pkg/postrenderer.PostRenderer. +func (p *PostRenderer) Run(renderedManifests *bytes.Buffer) (*bytes.Buffer, error) { + if p == nil || len(p.Manifests) == 0 { + return renderedManifests, nil + } + + generated, err := parseIdentities(renderedManifests.Bytes()) + if err != nil { + return nil, fmt.Errorf("parse rendered manifests: %w", err) + } + + for i := range p.Manifests { + id := p.Manifests[i].Identity() + if path, ok := generated[id.Key()]; ok { + return nil, fmt.Errorf("extra manifest %s collides with generated object %s (from %s)", p.Manifests[i].Path, id, path) + } + generated[id.Key()] = p.Manifests[i].Path + } + + out := new(bytes.Buffer) + if renderedManifests.Len() > 0 { + out.Write(renderedManifests.Bytes()) + if !bytes.HasSuffix(renderedManifests.Bytes(), []byte("\n")) { + out.WriteByte('\n') + } + } + for i := range p.Manifests { + raw := bytes.TrimSpace(p.Manifests[i].Raw) + if len(raw) == 0 { + continue + } + if out.Len() > 0 { + out.WriteString("---\n") + } + out.Write(raw) + if !bytes.HasSuffix(raw, []byte("\n")) { + out.WriteByte('\n') + } + } + return out, nil +} + +func parseIdentities(data []byte) (map[string]string, error) { + out := make(map[string]string) + if len(bytes.TrimSpace(data)) == 0 { + return out, nil + } + decoder := yamlutil.NewYAMLOrJSONDecoder(bytes.NewReader(data), 4096) + idx := 0 + for { + var obj unstructured.Unstructured + if err := decoder.Decode(&obj); err != nil { + if errors.Is(err, io.EOF) { + break + } + return nil, err + } + idx++ + if len(obj.Object) == 0 { + continue + } + id := Identity{ + APIVersion: obj.GetAPIVersion(), + Kind: obj.GetKind(), + Namespace: obj.GetNamespace(), + Name: obj.GetName(), + } + if id.APIVersion == "" || id.Kind == "" || id.Name == "" { + continue + } + out[id.Key()] = fmt.Sprintf("rendered#%d", idx) + } + return out, nil +} diff --git a/internal/extras/postrenderer_test.go b/internal/extras/postrenderer_test.go new file mode 100644 index 0000000..f044867 --- /dev/null +++ b/internal/extras/postrenderer_test.go @@ -0,0 +1,184 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras_test + +import ( + "bytes" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + + "deployah.dev/deployah/internal/extras" +) + +// TestPostRenderer_AppendsAndPreservesBraces exercises extras package behavior. +func TestPostRenderer_AppendsAndPreservesBraces(t *testing.T) { + t.Parallel() + extra := &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "monitoring.coreos.com/v1", + "kind": "PrometheusRule", + "metadata": map[string]any{ + "name": "alerts", + "namespace": "apps", + }, + "spec": map[string]any{ + "groups": []any{ + map[string]any{ + "name": "demo", + "rules": []any{ + map[string]any{ + "alert": "Down", + "annotations": map[string]any{ + "summary": "instance {{ $labels.instance }} is down", + }, + }, + }, + }, + }, + }, + }} + raw, err := (&extras.Object{Obj: extra}).MarshalYAML() + require.NoError(t, err) + + pr := &extras.PostRenderer{Manifests: []extras.Object{{ + Path: "alerts.yaml", + Raw: raw, + Obj: extra, + }}} + + rendered := bytes.NewBufferString(`apiVersion: v1 +kind: ConfigMap +metadata: + name: generated + namespace: apps +`) + out, err := pr.Run(rendered) + require.NoError(t, err) + body := out.String() + assert.Contains(t, body, "name: generated") + assert.Contains(t, body, "kind: PrometheusRule") + assert.Contains(t, body, "instance {{ $labels.instance }} is down") + assert.NotContains(t, body, `{{"{{"}}`) +} + +// TestPostRenderer_CollisionFails exercises extras package behavior. +func TestPostRenderer_CollisionFails(t *testing.T) { + t.Parallel() + extra := &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": map[string]any{ + "name": "shared", + "namespace": "apps", + }, + }} + raw, err := (&extras.Object{Obj: extra}).MarshalYAML() + require.NoError(t, err) + + pr := &extras.PostRenderer{Manifests: []extras.Object{{ + Path: "extra.yaml", + Raw: raw, + Obj: extra, + }}} + rendered := bytes.NewBufferString(`apiVersion: v1 +kind: ConfigMap +metadata: + name: shared + namespace: apps +`) + _, err = pr.Run(rendered) + require.Error(t, err) + assert.Contains(t, err.Error(), "collides") + assert.Contains(t, err.Error(), "extra.yaml") +} + +// TestPostRenderer_NilOrEmptyPassthrough exercises extras package behavior. +func TestPostRenderer_NilOrEmptyPassthrough(t *testing.T) { + t.Parallel() + in := bytes.NewBufferString("apiVersion: v1\nkind: ConfigMap\nmetadata:\n name: x\n") + out, err := (*extras.PostRenderer)(nil).Run(in) + require.NoError(t, err) + assert.Equal(t, in.String(), out.String()) + + out, err = (&extras.PostRenderer{}).Run(in) + require.NoError(t, err) + assert.Equal(t, in.String(), out.String()) +} + +// TestPostRenderer_InvalidRenderedYAML fails when Helm output cannot be parsed. +func TestPostRenderer_InvalidRenderedYAML(t *testing.T) { + t.Parallel() + pr := &extras.PostRenderer{Manifests: []extras.Object{{ + Path: "extra.yaml", + Raw: []byte("apiVersion: v1\nkind: ConfigMap\nmetadata:\n name: x\n"), + Obj: &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": map[string]any{"name": "x"}, + }}, + }}} + _, err := pr.Run(bytes.NewBufferString("not: [valid")) + require.Error(t, err) + assert.Contains(t, err.Error(), "parse rendered manifests") +} + +// TestPostRenderer_EmptyRawSkipped ignores whitespace-only extras. +func TestPostRenderer_EmptyRawSkipped(t *testing.T) { + t.Parallel() + extra := &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": map[string]any{"name": "extra", "namespace": "apps"}, + }} + pr := &extras.PostRenderer{Manifests: []extras.Object{ + {Path: "empty.yaml", Raw: []byte(" \n"), Obj: extra}, + }} + out, err := pr.Run(bytes.NewBufferString("")) + require.NoError(t, err) + assert.Empty(t, out.String()) +} + +// TestPostRenderer_AddsTrailingNewline inserts a newline before extras. +func TestPostRenderer_AddsTrailingNewline(t *testing.T) { + t.Parallel() + extra := &unstructured.Unstructured{Object: map[string]any{ + "apiVersion": "v1", + "kind": "ConfigMap", + "metadata": map[string]any{"name": "extra", "namespace": "apps"}, + }} + raw, err := (&extras.Object{Obj: extra}).MarshalYAML() + require.NoError(t, err) + raw = bytes.TrimRight(raw, "\n") + + pr := &extras.PostRenderer{Manifests: []extras.Object{ + {Path: "extra.yaml", Raw: raw, Obj: extra}, + }} + out, err := pr.Run(bytes.NewBufferString("apiVersion: v1\nkind: ConfigMap\nmetadata:\n name: base\n namespace: apps")) + require.NoError(t, err) + body := out.String() + assert.Contains(t, body, "name: base") + assert.Contains(t, body, "name: extra") + assert.True(t, strings.HasSuffix(body, "\n")) +} + +// TestIdentity_StringClusterScoped uses (cluster) when namespace is empty. +func TestIdentity_StringClusterScoped(t *testing.T) { + t.Parallel() + id := extras.Identity{APIVersion: "v1", Kind: "Namespace", Name: "demo"} + assert.Equal(t, "v1/Namespace (cluster)/demo", id.String()) +} diff --git a/internal/extras/scope.go b/internal/extras/scope.go new file mode 100644 index 0000000..724dab1 --- /dev/null +++ b/internal/extras/scope.go @@ -0,0 +1,293 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras + +import ( + "fmt" + "strings" + + "k8s.io/apimachinery/pkg/api/meta" + "k8s.io/apimachinery/pkg/runtime/schema" + "k8s.io/client-go/discovery" + "k8s.io/client-go/rest" + "k8s.io/client-go/restmapper" + + memcached "k8s.io/client-go/discovery/cached/memory" +) + +// ScopeResolver reports whether a GVK is known and whether it is namespaced. +type ScopeResolver interface { + // Known reports whether gvk is a built-in kind, declared by an in-repo + // CRD, or present in live discovery. + Known(gvk schema.GroupVersionKind) (bool, error) + // Namespaced reports whether gvk is namespaced. When Known is false and + // discovery is unavailable, callers may still use this; unknown kinds + // default to namespaced. + Namespaced(gvk schema.GroupVersionKind) (bool, error) +} + +// builtInScope maps "group/kind" (lowercase; core group is empty so keys look +// like "/configmap") to namespaced. Keys include a small DX allowlist for +// common operator APIs (cert-manager, prometheus-operator) so extras for those +// kinds load offline without embedding their CRDs; arbitrary groups never match +// by Kind alone. +var builtInScope = map[string]bool{ + // core + "/namespace": false, + "/node": false, + "/persistentvolume": false, + "/persistentvolumeclaim": true, + "/pod": true, + "/service": true, + "/configmap": true, + "/secret": true, + "/serviceaccount": true, + "/endpoints": true, + "/limitrange": true, + "/resourcequota": true, + "/event": true, + + // apps + "apps/deployment": true, + "apps/statefulset": true, + "apps/daemonset": true, + "apps/replicaset": true, + "apps/controllerrevision": true, + + // batch + "batch/job": true, + "batch/cronjob": true, + + // networking.k8s.io + "networking.k8s.io/ingress": true, + "networking.k8s.io/networkpolicy": true, + "networking.k8s.io/ingressclass": false, + + // autoscaling + "autoscaling/horizontalpodautoscaler": true, + + // policy + "policy/poddisruptionbudget": true, + "policy/podsecuritypolicy": false, + + // rbac.authorization.k8s.io + "rbac.authorization.k8s.io/role": true, + "rbac.authorization.k8s.io/rolebinding": true, + "rbac.authorization.k8s.io/clusterrole": false, + "rbac.authorization.k8s.io/clusterrolebinding": false, + + // storage.k8s.io + "storage.k8s.io/storageclass": false, + "storage.k8s.io/csidriver": false, + "storage.k8s.io/csinode": false, + "storage.k8s.io/volumeattachment": false, + + // scheduling.k8s.io + "scheduling.k8s.io/priorityclass": false, + + // apiextensions.k8s.io + "apiextensions.k8s.io/customresourcedefinition": false, + + // admissionregistration.k8s.io + "admissionregistration.k8s.io/mutatingwebhookconfiguration": false, + "admissionregistration.k8s.io/validatingwebhookconfiguration": false, + + // apiregistration.k8s.io + "apiregistration.k8s.io/apiservice": false, + + // node.k8s.io + "node.k8s.io/runtimeclass": false, + + // flowcontrol.apiserver.k8s.io + "flowcontrol.apiserver.k8s.io/flowschema": false, + "flowcontrol.apiserver.k8s.io/prioritylevelconfiguration": false, + + // coordination.k8s.io + "coordination.k8s.io/lease": true, + + // discovery.k8s.io + "discovery.k8s.io/endpointslice": true, + + // cert-manager (DX allowlist) + "cert-manager.io/certificate": true, + "cert-manager.io/issuer": true, + "cert-manager.io/clusterissuer": false, + "cert-manager.io/certificaterequest": true, + "acme.cert-manager.io/challenge": true, + "acme.cert-manager.io/order": true, + + // prometheus-operator (DX allowlist) + "monitoring.coreos.com/prometheusrule": true, + "monitoring.coreos.com/servicemonitor": true, + "monitoring.coreos.com/podmonitor": true, + "monitoring.coreos.com/probe": true, + "monitoring.coreos.com/alertmanagerconfig": true, +} + +// TableResolver resolves scope from a built-in group/kind table and optional +// CRD-provided scopes. Unknown kinds are not Known; Namespaced defaults +// them to namespaced when called anyway. +type TableResolver struct { + // CRDScope maps "group/kind" (lowercase) to namespaced, from .deployah/crds. + CRDScope map[string]bool +} + +// Known implements ScopeResolver. +func (r *TableResolver) Known(gvk schema.GroupVersionKind) (bool, error) { + key := crdScopeKey(gvk.Group, gvk.Kind) + if r != nil && r.CRDScope != nil { + if _, ok := r.CRDScope[key]; ok { + return true, nil + } + } + _, ok := builtInScope[key] + return ok, nil +} + +// Namespaced implements ScopeResolver. +func (r *TableResolver) Namespaced(gvk schema.GroupVersionKind) (bool, error) { + key := crdScopeKey(gvk.Group, gvk.Kind) + if r != nil && r.CRDScope != nil { + if ns, ok := r.CRDScope[key]; ok { + return ns, nil + } + } + if ns, ok := builtInScope[key]; ok { + return ns, nil + } + return true, nil +} + +// DiscoveryResolver wraps a RESTMapper and falls back to TableResolver. +type DiscoveryResolver struct { + Mapper meta.RESTMapper + Table TableResolver +} + +// NewDiscoveryResolver builds a ScopeResolver from a rest.Config. When cfg +// is nil, it returns a table-only resolver. +func NewDiscoveryResolver(cfg *rest.Config, crdScope map[string]bool) (ScopeResolver, error) { + table := TableResolver{CRDScope: crdScope} + if cfg == nil { + return &table, nil + } + disco, err := discovery.NewDiscoveryClientForConfig(cfg) + if err != nil { + return nil, fmt.Errorf("build discovery client: %w", err) + } + mapper := restmapper.NewDeferredDiscoveryRESTMapper(memcached.NewMemCacheClient(disco)) + return &DiscoveryResolver{Mapper: mapper, Table: table}, nil +} + +// Known implements ScopeResolver. +func (r *DiscoveryResolver) Known(gvk schema.GroupVersionKind) (bool, error) { + if r.Mapper != nil { + if _, err := r.Mapper.RESTMapping(gvk.GroupKind(), gvk.Version); err == nil { + return true, nil + } + } + return r.Table.Known(gvk) +} + +// Namespaced implements ScopeResolver. +func (r *DiscoveryResolver) Namespaced(gvk schema.GroupVersionKind) (bool, error) { + if r.Mapper != nil { + mapping, err := r.Mapper.RESTMapping(gvk.GroupKind(), gvk.Version) + if err == nil && mapping != nil { + return mapping.Scope.Name() == meta.RESTScopeNameNamespace, nil + } + } + return r.Table.Namespaced(gvk) +} + +func crdScopeKey(group, kind string) string { + return strings.ToLower(group) + "/" + strings.ToLower(kind) +} + +// scopeFromCRDObjects extracts group/kind -> namespaced from CRD Objects. +func scopeFromCRDObjects(crds []Object) map[string]bool { + out := make(map[string]bool) + for i := range crds { + group, _ := unstructuredNestedString(crds[i].Obj.Object, "spec", "group") + kind, _ := unstructuredNestedString(crds[i].Obj.Object, "spec", "names", "kind") + scope, _ := unstructuredNestedString(crds[i].Obj.Object, "spec", "scope") + if group == "" || kind == "" { + continue + } + out[crdScopeKey(group, kind)] = !strings.EqualFold(scope, "Cluster") + } + return out +} + +// GroupVersionsFromCRDs returns the set of "group/version" strings declared by +// the given CRD objects (every entry under spec.versions). Used to skip +// required-API checks for APIs this deploy is about to install. +func GroupVersionsFromCRDs(crds []Object) map[string]struct{} { + out := make(map[string]struct{}) + for i := range crds { + group, ok := unstructuredNestedString(crds[i].Obj.Object, "spec", "group") + if !ok || group == "" { + continue + } + versions, hasVersions := unstructuredNestedSlice(crds[i].Obj.Object, "spec", "versions") + if !hasVersions { + continue + } + for _, v := range versions { + vm, isMap := v.(map[string]any) + if !isMap { + continue + } + name, isString := vm["name"].(string) + if !isString || name == "" { + continue + } + out[group+"/"+name] = struct{}{} + } + } + return out +} + +func unstructuredNestedString(obj map[string]any, fields ...string) (string, bool) { + cur := any(obj) + for _, f := range fields { + m, ok := cur.(map[string]any) + if !ok { + return "", false + } + cur, ok = m[f] + if !ok { + return "", false + } + } + s, ok := cur.(string) + return s, ok +} + +func unstructuredNestedSlice(obj map[string]any, fields ...string) ([]any, bool) { + cur := any(obj) + for _, f := range fields { + m, ok := cur.(map[string]any) + if !ok { + return nil, false + } + cur, ok = m[f] + if !ok { + return nil, false + } + } + s, ok := cur.([]any) + return s, ok +} diff --git a/internal/extras/scope_test.go b/internal/extras/scope_test.go new file mode 100644 index 0000000..d45d617 --- /dev/null +++ b/internal/extras/scope_test.go @@ -0,0 +1,248 @@ +// Copyright 2025 The Deployah Authors +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +package extras_test + +import ( + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + "k8s.io/apimachinery/pkg/api/meta" + "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "k8s.io/apimachinery/pkg/runtime/schema" + + "deployah.dev/deployah/internal/extras" +) + +// TestTableResolver_BuiltInAndDefault exercises extras package behavior. +func TestTableResolver_BuiltInAndDefault(t *testing.T) { + t.Parallel() + r := &extras.TableResolver{} + ns, err := r.Namespaced(schema.GroupVersionKind{Group: "", Version: "v1", Kind: "ConfigMap"}) + require.NoError(t, err) + assert.True(t, ns) + + ns, err = r.Namespaced(schema.GroupVersionKind{Group: "", Version: "v1", Kind: "Namespace"}) + require.NoError(t, err) + assert.False(t, ns) + + known, err := r.Known(schema.GroupVersionKind{Group: "apps", Version: "v1", Kind: "Deployment"}) + require.NoError(t, err) + assert.True(t, known) + + // Kind alone is not enough: wrong group is unknown. + known, err = r.Known(schema.GroupVersionKind{Group: "evil.io", Version: "v1", Kind: "Deployment"}) + require.NoError(t, err) + assert.False(t, known) + + known, err = r.Known(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "Widget"}) + require.NoError(t, err) + assert.False(t, known) + + // Namespaced still defaults unknown kinds to namespaced when called. + ns, err = r.Namespaced(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "Widget"}) + require.NoError(t, err) + assert.True(t, ns) +} + +// TestTableResolver_OperatorAllowlistUsesRealGroups checks group-aware +// allowlist matching for common operator kinds. +func TestTableResolver_OperatorAllowlistUsesRealGroups(t *testing.T) { + t.Parallel() + r := &extras.TableResolver{} + known, err := r.Known(schema.GroupVersionKind{Group: "cert-manager.io", Version: "v1", Kind: "Certificate"}) + require.NoError(t, err) + assert.True(t, known) + + known, err = r.Known(schema.GroupVersionKind{Group: "evil.io", Version: "v1", Kind: "Certificate"}) + require.NoError(t, err) + assert.False(t, known) + + known, err = r.Known(schema.GroupVersionKind{Group: "monitoring.coreos.com", Version: "v1", Kind: "PrometheusRule"}) + require.NoError(t, err) + assert.True(t, known) +} + +// TestGroupVersionsFromCRDs extracts group/version pairs from CRD YAML. +func TestGroupVersionsFromCRDs(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "crds", "cert.yaml"), ` +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: certificates.cert-manager.io +spec: + group: cert-manager.io + scope: Namespaced + names: + kind: Certificate + plural: certificates + versions: + - name: v1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object +`) + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "default", + Scope: &extras.TableResolver{}, + }) + require.NoError(t, err) + gvs := extras.GroupVersionsFromCRDs(bundle.CRDs) + _, ok := gvs["cert-manager.io/v1"] + assert.True(t, ok) +} + +// TestTableResolver_KnownFromCRDScope exercises extras package behavior. +func TestTableResolver_KnownFromCRDScope(t *testing.T) { + t.Parallel() + r := &extras.TableResolver{CRDScope: map[string]bool{ + "example.com/widget": true, + }} + known, err := r.Known(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "Widget"}) + require.NoError(t, err) + assert.True(t, known) +} + +// TestTableResolver_CRDScopeOverridesDefault exercises extras package behavior. +func TestTableResolver_CRDScopeOverridesDefault(t *testing.T) { + t.Parallel() + r := &extras.TableResolver{CRDScope: map[string]bool{ + "example.com/clusterwidget": false, + }} + ns, err := r.Namespaced(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "ClusterWidget"}) + require.NoError(t, err) + assert.False(t, ns) +} + +// TestLoad_UsesCRDScopeForCustomResources exercises extras package behavior. +func TestLoad_UsesCRDScopeForCustomResources(t *testing.T) { + t.Parallel() + dir := t.TempDir() + writeFile(t, filepath.Join(dir, ".deployah", "crds", "widget.yaml"), ` +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: clusterwidgets.example.com +spec: + group: example.com + scope: Cluster + names: + kind: ClusterWidget + plural: clusterwidgets + versions: + - name: v1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object +`) + writeFile(t, filepath.Join(dir, ".deployah", "manifests", "cw.yaml"), ` +apiVersion: example.com/v1 +kind: ClusterWidget +metadata: + name: one +`) + bundle, err := extras.Load(extras.LoadConfig{ + SpecDir: dir, + Project: "demo", + Environment: "prod", + DeclaredEnvs: []string{"prod"}, + ReleaseNamespace: "apps", + Scope: &extras.TableResolver{}, + }) + require.NoError(t, err) + require.Len(t, bundle.Manifests, 1) + assert.Empty(t, bundle.Manifests[0].Obj.GetNamespace()) +} + +// TestNewDiscoveryResolver_NilConfig returns a table-only resolver. +func TestNewDiscoveryResolver_NilConfig(t *testing.T) { + t.Parallel() + scope, err := extras.NewDiscoveryResolver(nil, map[string]bool{"example.com/widget": true}) + require.NoError(t, err) + known, err := scope.Known(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "Widget"}) + require.NoError(t, err) + assert.True(t, known) +} + +// TestDiscoveryResolver_MapperHitAndFallback covers mapper hits and table +// fallback. +func TestDiscoveryResolver_MapperHitAndFallback(t *testing.T) { + t.Parallel() + mapper := meta.NewDefaultRESTMapper([]schema.GroupVersion{{Group: "example.com", Version: "v1"}}) + mapper.Add(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "Widget"}, meta.RESTScopeNamespace) + mapper.Add(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "ClusterWidget"}, meta.RESTScopeRoot) + + r := &extras.DiscoveryResolver{ + Mapper: mapper, + Table: extras.TableResolver{CRDScope: map[string]bool{"other.io/thing": true}}, + } + + known, err := r.Known(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "Widget"}) + require.NoError(t, err) + assert.True(t, known) + + ns, err := r.Namespaced(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "Widget"}) + require.NoError(t, err) + assert.True(t, ns) + + ns, err = r.Namespaced(schema.GroupVersionKind{Group: "example.com", Version: "v1", Kind: "ClusterWidget"}) + require.NoError(t, err) + assert.False(t, ns) + + known, err = r.Known(schema.GroupVersionKind{Group: "other.io", Version: "v1", Kind: "Thing"}) + require.NoError(t, err) + assert.True(t, known, "unknown to mapper still known via table CRDScope") +} + +// TestGroupVersionsFromCRDs_MalformedSkipped ignores incomplete CRD objects. +func TestGroupVersionsFromCRDs_MalformedSkipped(t *testing.T) { + t.Parallel() + crds := []extras.Object{ + {Obj: &unstructured.Unstructured{Object: map[string]any{ + "spec": map[string]any{"group": "example.com"}, + }}}, + {Obj: &unstructured.Unstructured{Object: map[string]any{ + "spec": map[string]any{ + "group": "example.com", + "versions": []any{"v1", map[string]any{"name": ""}}, + }, + }}}, + {Obj: &unstructured.Unstructured{Object: map[string]any{ + "spec": map[string]any{ + "group": "ok.io", + "versions": []any{ + map[string]any{"name": "v1"}, + map[string]any{"name": "v2beta1"}, + }, + }, + }}}, + } + gvs := extras.GroupVersionsFromCRDs(crds) + assert.Equal(t, map[string]struct{}{ + "ok.io/v1": {}, + "ok.io/v2beta1": {}, + }, gvs) +} diff --git a/internal/helm/generate.go b/internal/helm/generate.go index 7a40a6f..6eefb20 100644 --- a/internal/helm/generate.go +++ b/internal/helm/generate.go @@ -294,9 +294,13 @@ func MapSpecToChartValues(m *spec.Spec, desiredEnvironment string, resolved *spe componentValues := map[string]any{ "commonLabels": map[string]string{ - "deployah.dev/project": m.Project, - "deployah.dev/component": componentName, - "deployah.dev/environment": spec.NormalizeEnv(desiredEnvironment).K8sSafe, + spec.LabelProject: m.Project, + spec.LabelComponent: componentName, + spec.LabelEnvironment: spec.NormalizeEnv(desiredEnvironment).K8sSafe, + }, + "commonAnnotations": map[string]string{ + spec.AnnotationSource: spec.SourceSpec, + spec.AnnotationProject: m.Project, }, } diff --git a/internal/helm/generate_test.go b/internal/helm/generate_test.go index a3062a9..08d677b 100644 --- a/internal/helm/generate_test.go +++ b/internal/helm/generate_test.go @@ -610,6 +610,11 @@ func TestMapSpecToChartValues_Profiles(t *testing.T) { assert.Equal(t, "web", labels["tier"]) assert.Equal(t, "shop", labels["deployah.dev/project"]) + annotations, ok := web["commonAnnotations"].(map[string]string) + require.True(t, ok) + assert.Equal(t, spec.SourceSpec, annotations[spec.AnnotationSource]) + assert.Equal(t, "shop", annotations[spec.AnnotationProject]) + tolerations, ok := web["tolerations"].([]any) require.True(t, ok) require.Len(t, tolerations, 1) diff --git a/internal/helm/helm.go b/internal/helm/helm.go index 7081baf..637f91c 100644 --- a/internal/helm/helm.go +++ b/internal/helm/helm.go @@ -13,6 +13,7 @@ import ( "helm.sh/helm/v4/pkg/chart/v2/loader" "helm.sh/helm/v4/pkg/cli" "helm.sh/helm/v4/pkg/kube" + "helm.sh/helm/v4/pkg/postrenderer" "helm.sh/helm/v4/pkg/release" "helm.sh/helm/v4/pkg/storage/driver" "k8s.io/apimachinery/pkg/labels" @@ -161,6 +162,16 @@ func NewClient(opts ...Option) (*Client, error) { return c, nil } +// Namespace returns the release namespace Helm will use for installs and +// offline renders (from WithNamespace, HELM_NAMESPACE, or the kubeconfig +// context default). +func (c *Client) Namespace() string { + if c == nil || c.settings == nil { + return "" + } + return c.settings.Namespace() +} + // IsReachable reports whether the configured Kubernetes cluster is reachable. // Also works around helm/helm#32183: Helm v4.2.0 panics on a second // IsReachable call after the first one fails (typed-nil cached in @@ -176,10 +187,11 @@ func (c *Client) IsReachable() error { // InstallApp installs or upgrades the app using the embedded chart, using // resolved (if non-nil) for platform-resolved FQDN/TLS values. When dryRun // is true, it renders client-side via [Client.RenderManifests] instead of -// touching the cluster. -func (c *Client) InstallApp(ctx context.Context, manifest *spec.Spec, environment string, dryRun bool, resolved *spec.ResolvedSpec) error { +// touching the cluster. postRenderer, when non-nil, is applied to rendered +// manifests before they are installed or upgraded. +func (c *Client) InstallApp(ctx context.Context, manifest *spec.Spec, environment string, dryRun bool, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) error { if dryRun { - _, cleanup, err := c.RenderManifests(ctx, manifest, environment, resolved) + _, cleanup, err := c.RenderManifests(ctx, manifest, environment, resolved, postRenderer) if cleanup != nil { defer cleanup() } @@ -230,6 +242,7 @@ func (c *Client) InstallApp(ctx context.Context, manifest *spec.Spec, environmen install.WaitStrategy = kube.StatusWatcherStrategy install.RollbackOnFailure = true install.Labels = labels + install.PostRenderer = postRenderer if _, runErr := install.RunWithContext(ctx, ch, values); runErr != nil { return c.wrapHelmError("install", GenerateReleaseName(manifest.Project, environment), runErr) @@ -244,6 +257,7 @@ func (c *Client) InstallApp(ctx context.Context, manifest *spec.Spec, environmen upgrade.RollbackOnFailure = true upgrade.WaitStrategy = kube.StatusWatcherStrategy upgrade.Labels = labels + upgrade.PostRenderer = postRenderer _, err = upgrade.RunWithContext(ctx, GenerateReleaseName(manifest.Project, environment), ch, values) if err != nil { return c.wrapHelmError("upgrade", GenerateReleaseName(manifest.Project, environment), err) diff --git a/internal/helm/render.go b/internal/helm/render.go index 7ba4045..5b971ad 100644 --- a/internal/helm/render.go +++ b/internal/helm/render.go @@ -22,6 +22,7 @@ import ( "helm.sh/helm/v4/pkg/action" "helm.sh/helm/v4/pkg/chart/v2/loader" + "helm.sh/helm/v4/pkg/postrenderer" "deployah.dev/deployah/internal/render" "deployah.dev/deployah/internal/spec" @@ -38,7 +39,7 @@ import ( // // Callers must invoke the returned cleanup func; ChartPath is not removed // automatically so callers like deploy can reuse it for the real apply. -func (c *Client) RenderManifests(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) (result *render.RenderResult, cleanup func(), err error) { +func (c *Client) RenderManifests(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) (result *render.RenderResult, cleanup func(), err error) { releaseName := GenerateReleaseName(manifest.Project, environment) ch, chartPath, cleanup, err := c.prepareAndLoadChart(ctx, manifest, environment, resolved) @@ -54,9 +55,9 @@ func (c *Client) RenderManifests(ctx context.Context, manifest *spec.Spec, envir // Not found -> fresh install. Any other history error is treated // the same way InstallApp does: fall through to an install attempt, // which will surface a clearer error if something else is wrong. - result, err = c.renderInstall(ctx, releaseName, ch, values, labels) + result, err = c.renderInstall(ctx, releaseName, ch, values, labels, postRenderer) } else { - result, err = c.renderUpgrade(ctx, releaseName, ch, values, labels) + result, err = c.renderUpgrade(ctx, releaseName, ch, values, labels, postRenderer) } if err != nil { cleanup() @@ -73,7 +74,7 @@ func (c *Client) RenderManifests(ctx context.Context, manifest *spec.Spec, envir // fresh install (IsUpgrade false, Revision 1) even when a release already // exists, so it can't be diffed against a prior release like // [Client.RenderManifests] can; use that instead when cluster access is fine. -func (c *Client) RenderOffline(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) (result *render.RenderResult, cleanup func(), err error) { +func (c *Client) RenderOffline(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) (result *render.RenderResult, cleanup func(), err error) { releaseName := GenerateReleaseName(manifest.Project, environment) ch, chartPath, cleanup, err := c.prepareAndLoadChart(ctx, manifest, environment, resolved) @@ -83,7 +84,7 @@ func (c *Client) RenderOffline(ctx context.Context, manifest *spec.Spec, environ values, labels := renderInputs(manifest, environment) - result, err = c.renderInstall(ctx, releaseName, ch, values, labels) + result, err = c.renderInstall(ctx, releaseName, ch, values, labels, postRenderer) if err != nil { cleanup() return nil, nil, err @@ -179,7 +180,7 @@ func restoreCapabilitiesForDryRun(cfg *action.Configuration) func() { } } -func (c *Client) renderInstall(ctx context.Context, releaseName string, ch *chart.Chart, values map[string]any, labels map[string]string) (*render.RenderResult, error) { +func (c *Client) renderInstall(ctx context.Context, releaseName string, ch *chart.Chart, values map[string]any, labels map[string]string, postRenderer postrenderer.PostRenderer) (*render.RenderResult, error) { // See restoreCapabilitiesForDryRun: a client-side dry-run install swaps // cfg.KubeClient, cfg.Releases, and cfg.Capabilities. Restore them once // this render is done so only this call is affected. @@ -192,6 +193,7 @@ func (c *Client) renderInstall(ctx context.Context, releaseName string, ch *char install.DryRunStrategy = action.DryRunClient install.DisableOpenAPIValidation = true install.Labels = labels + install.PostRenderer = postRenderer rel, runErr := install.RunWithContext(ctx, ch, values) if runErr != nil { @@ -211,7 +213,7 @@ func (c *Client) renderInstall(ctx context.Context, releaseName string, ch *char }, nil } -func (c *Client) renderUpgrade(ctx context.Context, releaseName string, ch *chart.Chart, values map[string]any, labels map[string]string) (*render.RenderResult, error) { +func (c *Client) renderUpgrade(ctx context.Context, releaseName string, ch *chart.Chart, values map[string]any, labels map[string]string, postRenderer postrenderer.PostRenderer) (*render.RenderResult, error) { // Upgrade's dry-run path doesn't swap KubeClient/Releases as of Helm // v4.2.1; restoring defensively guards against a future version adding // the short-circuit Install already has. Capabilities is deliberately @@ -224,6 +226,7 @@ func (c *Client) renderUpgrade(ctx context.Context, releaseName string, ch *char upgrade.DryRunStrategy = action.DryRunClient upgrade.DisableOpenAPIValidation = true upgrade.Labels = labels + upgrade.PostRenderer = postRenderer rel, runErr := upgrade.RunWithContext(ctx, releaseName, ch, values) if runErr != nil { diff --git a/internal/plan/build.go b/internal/plan/build.go index 2806700..7e9e590 100644 --- a/internal/plan/build.go +++ b/internal/plan/build.go @@ -18,6 +18,8 @@ import ( "context" "fmt" + "helm.sh/helm/v4/pkg/postrenderer" + "deployah.dev/deployah/internal/render" "deployah.dev/deployah/internal/spec" @@ -31,7 +33,7 @@ import ( // internal/session and tests can inject a minimal fake. type BuildClient interface { historyClient - RenderManifests(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) (*render.RenderResult, func(), error) + RenderManifests(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) (*render.RenderResult, func(), error) } // BuildPlan renders manifest for environment via client and diffs the @@ -43,9 +45,10 @@ type BuildClient interface { // The caller must invoke the returned cleanup func once done with // result.ChartPath (same contract as [helm.Client.RenderManifests]). On // error, cleanup is still returned when a chart was prepared and must be -// called. -func BuildPlan(ctx context.Context, client BuildClient, manifest *spec.Spec, environment, clusterContext string, resolved *spec.ResolvedSpec) (*Plan, *render.RenderResult, func(), error) { - result, cleanup, err := client.RenderManifests(ctx, manifest, environment, resolved) +// called. postRenderer, when non-nil, is forwarded to RenderManifests so +// extras appear in the diff. +func BuildPlan(ctx context.Context, client BuildClient, manifest *spec.Spec, environment, clusterContext string, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) (*Plan, *render.RenderResult, func(), error) { + result, cleanup, err := client.RenderManifests(ctx, manifest, environment, resolved, postRenderer) if cleanup == nil { cleanup = func() {} } diff --git a/internal/session/interfaces.go b/internal/session/interfaces.go index 2a22049..0e42c4c 100644 --- a/internal/session/interfaces.go +++ b/internal/session/interfaces.go @@ -18,6 +18,7 @@ import ( "context" "time" + "helm.sh/helm/v4/pkg/postrenderer" "k8s.io/apimachinery/pkg/labels" "deployah.dev/deployah/internal/render" @@ -35,18 +36,21 @@ type HelmClient interface { // InstallApp installs or upgrades an application using Helm. When resolved // is non-nil, TLS and hostname values are sourced from it rather than the - // raw spec. - InstallApp(ctx context.Context, manifest *spec.Spec, environment string, dryRun bool, resolved *spec.ResolvedSpec) error + // raw spec. postRenderer, when non-nil, is applied to the rendered + // manifests before they are sent to the cluster. + InstallApp(ctx context.Context, manifest *spec.Spec, environment string, dryRun bool, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) error // RenderManifests renders the chart for manifest/environment client-side, // without mutating the cluster or Helm's release history. The caller must // run the returned cleanup func once done with the result's ChartPath. - RenderManifests(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) (*render.RenderResult, func(), error) + // postRenderer, when non-nil, is applied to the rendered manifests. + RenderManifests(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) (*render.RenderResult, func(), error) // RenderOffline renders the chart for manifest/environment as a fresh // install, without any Kubernetes API access. The caller must run the // returned cleanup func once done with the result's ChartPath. - RenderOffline(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) (*render.RenderResult, func(), error) + // postRenderer, when non-nil, is applied to the rendered manifests. + RenderOffline(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) (*render.RenderResult, func(), error) // DeleteRelease uninstalls a Helm release. When wait is true the call // blocks until all resources are fully removed using the legacy polling diff --git a/internal/session/session_test.go b/internal/session/session_test.go index 16bb9b6..572661b 100644 --- a/internal/session/session_test.go +++ b/internal/session/session_test.go @@ -26,6 +26,7 @@ import ( "github.com/stretchr/testify/assert" "github.com/stretchr/testify/mock" "github.com/stretchr/testify/require" + "helm.sh/helm/v4/pkg/postrenderer" "k8s.io/apimachinery/pkg/labels" "k8s.io/client-go/kubernetes" "k8s.io/client-go/kubernetes/fake" @@ -80,14 +81,14 @@ func (m *MockHelmClient) IsReachable() error { } // InstallApp implements [HelmClient]. -func (m *MockHelmClient) InstallApp(ctx context.Context, manifest *spec.Spec, environment string, dryRun bool, resolved *spec.ResolvedSpec) error { - args := m.Called(ctx, manifest, environment, dryRun, resolved) +func (m *MockHelmClient) InstallApp(ctx context.Context, manifest *spec.Spec, environment string, dryRun bool, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) error { + args := m.Called(ctx, manifest, environment, dryRun, resolved, postRenderer) return args.Error(0) } // RenderManifests implements [HelmClient]. -func (m *MockHelmClient) RenderManifests(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) (*render.RenderResult, func(), error) { - args := m.Called(ctx, manifest, environment, resolved) +func (m *MockHelmClient) RenderManifests(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) (*render.RenderResult, func(), error) { + args := m.Called(ctx, manifest, environment, resolved, postRenderer) if err := args.Error(2); err != nil { return nil, func() {}, err } @@ -106,8 +107,8 @@ func (m *MockHelmClient) RenderManifests(ctx context.Context, manifest *spec.Spe } // RenderOffline implements [HelmClient]. -func (m *MockHelmClient) RenderOffline(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) (*render.RenderResult, func(), error) { - args := m.Called(ctx, manifest, environment, resolved) +func (m *MockHelmClient) RenderOffline(ctx context.Context, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec, postRenderer postrenderer.PostRenderer) (*render.RenderResult, func(), error) { + args := m.Called(ctx, manifest, environment, resolved, postRenderer) if err := args.Error(2); err != nil { return nil, func() {}, err } @@ -777,7 +778,7 @@ func TestIntegrationWithMocks(t *testing.T) { mockHelm := &MockHelmClient{} testManifest := &spec.Spec{Project: "test-project"} - mockHelm.On("InstallApp", mock.Anything, testManifest, "production", false, mock.Anything).Return(nil) + mockHelm.On("InstallApp", mock.Anything, testManifest, "production", false, mock.Anything, mock.Anything).Return(nil) sess := New(WithHelmFactory(func(s *Session) (HelmClient, error) { return mockHelm, nil @@ -789,7 +790,7 @@ func TestIntegrationWithMocks(t *testing.T) { helmClient, err := cluster.Helm() assert.NoError(t, err) - err = helmClient.InstallApp(t.Context(), testManifest, "production", false, nil) + err = helmClient.InstallApp(t.Context(), testManifest, "production", false, nil, nil) assert.NoError(t, err) mockHelm.AssertExpectations(t) }) diff --git a/internal/spec/constants.go b/internal/spec/constants.go index fbdfe71..c6afef3 100644 --- a/internal/spec/constants.go +++ b/internal/spec/constants.go @@ -185,6 +185,32 @@ const ( // ManagedByValue is the value used for the managed-by label ManagedByValue = "deployah" + + // AnnotationSource is the annotation key recording which Deployah layer + // produced a managed object (spec, manifests, or crds). + AnnotationSource = LabelPrefix + "/source" + + // AnnotationProject is the annotation key for project identification on + // Deployah-managed objects. Same string as LabelProject; used as an + // annotation so CRDs (which carry no environment label) still identify + // the owning project. + AnnotationProject = LabelProject + + // SourceSpec is the AnnotationSource value for chart-generated objects. + SourceSpec = "spec" + + // SourceManifests is the AnnotationSource value for .deployah/manifests. + SourceManifests = "manifests" + + // SourceCRDs is the AnnotationSource value for .deployah/crds. + SourceCRDs = "crds" + + // ManifestsDir is the subdirectory under DeployahConfigDir for extra + // Kubernetes manifests. + ManifestsDir = "manifests" + + // CRDsDir is the subdirectory under DeployahConfigDir for CRDs. + CRDsDir = "crds" ) // ResourcePresetMappings defines the resource specifications for each preset diff --git a/internal/spec/loader.go b/internal/spec/loader.go index 9b497e3..15bcdb5 100644 --- a/internal/spec/loader.go +++ b/internal/spec/loader.go @@ -75,6 +75,14 @@ func environmentRegistry(environments map[string]Environment, platform *Platform return slices.Sorted(maps.Keys(environments)), "the spec" } +// DeclaredEnvironments returns the sorted registry of environment names that +// may be deployed to. The platform config owns the registry when present; +// otherwise the spec's environments map is used. +func DeclaredEnvironments(environments map[string]Environment, platform *PlatformConfig) []string { + names, _ := environmentRegistry(environments, platform) + return names +} + // ResolveEnvironment selects the target environment and returns its name and // developer-owned overrides. Which names are valid (the registry) is owned by // the platform config when one exists; the spec's environments map supplies diff --git a/internal/testing/plan_scenarios.go b/internal/testing/plan_scenarios.go index 22016cb..bc84bb6 100644 --- a/internal/testing/plan_scenarios.go +++ b/internal/testing/plan_scenarios.go @@ -28,6 +28,7 @@ import ( "github.com/stretchr/testify/require" "gopkg.in/yaml.v3" + "deployah.dev/deployah/internal/extras" "deployah.dev/deployah/internal/helm" "deployah.dev/deployah/internal/k8s" "deployah.dev/deployah/internal/plan" @@ -247,10 +248,15 @@ func renderManifestFile(t *testing.T, dir, filename string) manifestSide { resolved = resolvedSpec } - client, err := helm.NewClient() + // Pin the release namespace so goldens stay stable regardless of + // HELM_NAMESPACE or the ambient kubeconfig context. + client, err := helm.NewClient(helm.WithNamespace("default")) require.NoError(t, err) - result, cleanup, err := client.RenderOffline(ctx, manifest, envName, resolved) + bundle, loadErr := extras.LoadFromSpec(specPath, manifest, platform, envName, client.Namespace(), nil) + require.NoError(t, loadErr) + + result, cleanup, err := client.RenderOffline(ctx, manifest, envName, resolved, bundle.PostRendererFor()) if cleanup != nil { t.Cleanup(cleanup) } diff --git a/internal/testing/scenario_discovery.go b/internal/testing/scenario_discovery.go index 3c30db3..abdce97 100644 --- a/internal/testing/scenario_discovery.go +++ b/internal/testing/scenario_discovery.go @@ -24,6 +24,8 @@ import ( "strings" "gopkg.in/yaml.v3" + + "deployah.dev/deployah/internal/spec" ) // errorScenarioIndicators are name prefixes that mark a scenario as @@ -52,12 +54,17 @@ func DiscoverScenarios(scenariosDir string) ([]TestScenario, error) { return nil } + // .deployah holds extras fixtures, not nested scenarios. + if info.Name() == spec.DeployahConfigDir || strings.HasPrefix(info.Name(), ".") { + return filepath.SkipDir + } + // plan-* directories hold plan-config.yaml scenarios (see // plan_scenarios.go), not render/golden-file scenarios: skip them // here so they never get treated as a render scenario missing its // expected/ directory. if strings.HasPrefix(info.Name(), "plan-") { - return nil + return filepath.SkipDir } manifestPath := filepath.Join(path, "deployah.yaml") diff --git a/internal/testing/schema_validate.go b/internal/testing/schema_validate.go index 1dd4497..c73d440 100644 --- a/internal/testing/schema_validate.go +++ b/internal/testing/schema_validate.go @@ -24,18 +24,24 @@ import ( "k8s.io/client-go/kubernetes/scheme" ) -// unregisteredSchemeKinds lists kinds the embedded chart can emit that -// [validateAgainstScheme] cannot check against [scheme.Scheme]: +// unregisteredSchemeKinds lists kinds the chart or .deployah/ extras can +// emit that [validateAgainstScheme] cannot check against [scheme.Scheme]: // -// - ServiceMonitor, PodMonitor: Prometheus Operator CRDs, not core -// Kubernetes types. No scenario renders them today. +// - ServiceMonitor, PodMonitor, PrometheusRule: Prometheus Operator CRDs. +// - ClusterWidget: fixture CRD used by extras scenarios. // - HorizontalPodAutoscaler: [helm.Client.RenderOffline] has no live // cluster, so Capabilities.KubeVersion falls back to Helm's pre-1.23 // default, making the chart select the removed autoscaling/v2beta1 API // instead of autoscaling/v2. This affects `deployah plan --offline` for // any autoscaling component; the real fix belongs in the render // pipeline (a modern default KubeVersion), not here. -var unregisteredSchemeKinds = []string{"ServiceMonitor", "PodMonitor", "HorizontalPodAutoscaler"} +var unregisteredSchemeKinds = []string{ + "ServiceMonitor", + "PodMonitor", + "PrometheusRule", + "ClusterWidget", + "HorizontalPodAutoscaler", +} // schemeDecoder is a strict (unknown-field-rejecting) decoder against the // client-go built-in scheme, shared across scenarios. diff --git a/internal/testing/types.go b/internal/testing/types.go index 27bdce3..9727e39 100644 --- a/internal/testing/types.go +++ b/internal/testing/types.go @@ -30,6 +30,7 @@ import ( "gopkg.in/yaml.v3" "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" + "deployah.dev/deployah/internal/extras" "deployah.dev/deployah/internal/helm" "deployah.dev/deployah/internal/k8s" "deployah.dev/deployah/internal/spec" @@ -120,7 +121,7 @@ func (suite *IntegrationTestSuite) RunScenarioTest(t *testing.T, scenario TestSc t.Chdir(testDir) - manifest, environment, resolved, err := suite.loadAndResolve(t, scenario) + manifest, environment, resolved, platform, err := suite.loadAndResolve(t, scenario) if scenario.ExpectError || len(scenario.ExpectedErrors) > 0 { require.Error(t, err) @@ -131,7 +132,7 @@ func (suite *IntegrationTestSuite) RunScenarioTest(t *testing.T, scenario TestSc } require.NoError(t, err) - renderedManifests, err := suite.renderChart(t, manifest, environment, resolved) + renderedManifests, err := suite.renderChart(t, testDir, manifest, environment, resolved, platform) require.NoError(t, err) // Every rendered object must decode cleanly into its typed @@ -180,12 +181,17 @@ func (suite *IntegrationTestSuite) setupScenarioEnvironment(t *testing.T, scenar require.NoError(t, err) } + extrasSrc := filepath.Join(suite.ScenariosDir, scenario.ScenarioDir, spec.DeployahConfigDir) + if _, statErr := os.Stat(extrasSrc); statErr == nil { + require.NoError(t, copyDir(extrasSrc, filepath.Join(testDir, spec.DeployahConfigDir))) + } + return testDir } // loadAndResolve loads the manifest (and optional platform file), resolves the // environment, and runs [spec.Resolve] when a platform is present. -func (suite *IntegrationTestSuite) loadAndResolve(t *testing.T, scenario TestScenario) (*spec.Spec, string, *spec.ResolvedSpec, error) { +func (suite *IntegrationTestSuite) loadAndResolve(t *testing.T, scenario TestScenario) (*spec.Spec, string, *spec.ResolvedSpec, *spec.PlatformConfig, error) { t.Helper() ctx := context.Background() @@ -193,28 +199,28 @@ func (suite *IntegrationTestSuite) loadAndResolve(t *testing.T, scenario TestSce if scenario.PlatformFile != "" { loaded, err := spec.LoadPlatform(filepath.Base(scenario.PlatformFile)) if err != nil { - return nil, "", nil, err + return nil, "", nil, nil, err } platform = loaded } manifest, err := spec.Load(ctx, scenario.ManifestFile, scenario.Environment, platform) if err != nil { - return nil, "", nil, err + return nil, "", nil, platform, err } envName, _, err := spec.ResolveEnvironment(manifest.Environments, platform, scenario.Environment) if err != nil { - return nil, "", nil, err + return nil, "", nil, platform, err } if platform == nil { - return manifest, envName, nil, nil + return manifest, envName, nil, nil, nil } envIdentity := spec.NormalizeEnv(envName) resolved, _, err := spec.Resolve(manifest, platform, envIdentity, spec.SubstitutionReport{}) if err != nil { - return nil, "", nil, err + return nil, "", nil, platform, err } // Materialize self-signed TLS certs offline (no cluster access), @@ -222,24 +228,34 @@ func (suite *IntegrationTestSuite) loadAndResolve(t *testing.T, scenario TestSce // expose scenario fails render with "certificate not materialized // before render". if tlsErr := k8s.MaterializeSelfSignedTLS(ctx, nil, "", resolved); tlsErr != nil { - return nil, "", nil, tlsErr + return nil, "", nil, platform, tlsErr } - return manifest, envName, resolved, nil + return manifest, envName, resolved, platform, nil } // renderChart renders manifest/environment through Helm's real template // engine via [helm.Client.RenderOffline] (no cluster access, matching // `deployah plan --offline`), returning the resulting Kubernetes objects. -func (suite *IntegrationTestSuite) renderChart(t *testing.T, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec) ([]unstructured.Unstructured, error) { +// Extra manifests from .deployah/ under testDir are appended via the Helm +// post-renderer. +func (suite *IntegrationTestSuite) renderChart(t *testing.T, testDir string, manifest *spec.Spec, environment string, resolved *spec.ResolvedSpec, platform *spec.PlatformConfig) ([]unstructured.Unstructured, error) { t.Helper() - client, err := helm.NewClient() + // Pin the release namespace so goldens stay stable regardless of + // HELM_NAMESPACE or the ambient kubeconfig context. + client, err := helm.NewClient(helm.WithNamespace("default")) if err != nil { return nil, fmt.Errorf("create helm client: %w", err) } - result, cleanup, err := client.RenderOffline(context.Background(), manifest, environment, resolved) + specPath := filepath.Join(testDir, "deployah.yaml") + bundle, loadErr := extras.LoadFromSpec(specPath, manifest, platform, environment, client.Namespace(), nil) + if loadErr != nil { + return nil, fmt.Errorf("load extras: %w", loadErr) + } + + result, cleanup, err := client.RenderOffline(context.Background(), manifest, environment, resolved, bundle.PostRendererFor()) if cleanup != nil { t.Cleanup(cleanup) } @@ -431,3 +447,28 @@ func copyFile(src, dst string) error { } return dstCloseErr } + +// copyDir recursively copies src into dst. dst is created if missing. +func copyDir(src, dst string) error { + if err := os.MkdirAll(dst, 0o750); err != nil { + return err + } + entries, err := os.ReadDir(src) + if err != nil { + return err + } + for _, e := range entries { + from := filepath.Join(src, e.Name()) + to := filepath.Join(dst, e.Name()) + if e.IsDir() { + if copyErr := copyDir(from, to); copyErr != nil { + return copyErr + } + continue + } + if copyErr := copyFile(from, to); copyErr != nil { + return copyErr + } + } + return nil +} diff --git a/scenarios/autoscaling-hpa/expected/deployment-autoscaling-hpa-production-api.yaml b/scenarios/autoscaling-hpa/expected/deployment-autoscaling-hpa-production-api.yaml index aa97422..d9aad5c 100644 --- a/scenarios/autoscaling-hpa/expected/deployment-autoscaling-hpa-production-api.yaml +++ b/scenarios/autoscaling-hpa/expected/deployment-autoscaling-hpa-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: autoscaling-hpa + deployah.dev/source: spec labels: app.kubernetes.io/instance: autoscaling-hpa-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/autoscaling-hpa/expected/horizontalpodautoscaler-autoscaling-hpa-production-api.yaml b/scenarios/autoscaling-hpa/expected/horizontalpodautoscaler-autoscaling-hpa-production-api.yaml index a279a86..2218401 100644 --- a/scenarios/autoscaling-hpa/expected/horizontalpodautoscaler-autoscaling-hpa-production-api.yaml +++ b/scenarios/autoscaling-hpa/expected/horizontalpodautoscaler-autoscaling-hpa-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: autoscaling/v2beta1 kind: HorizontalPodAutoscaler metadata: + annotations: + deployah.dev/project: autoscaling-hpa + deployah.dev/source: spec labels: app.kubernetes.io/instance: autoscaling-hpa-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/autoscaling-hpa/expected/service-autoscaling-hpa-production-api.yaml b/scenarios/autoscaling-hpa/expected/service-autoscaling-hpa-production-api.yaml index c32b23f..9d7130d 100644 --- a/scenarios/autoscaling-hpa/expected/service-autoscaling-hpa-production-api.yaml +++ b/scenarios/autoscaling-hpa/expected/service-autoscaling-hpa-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: autoscaling-hpa + deployah.dev/source: spec labels: app.kubernetes.io/instance: autoscaling-hpa-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/basic-web-service/expected/deployment-basic-web-service-dev.yaml b/scenarios/basic-web-service/expected/deployment-basic-web-service-dev.yaml index 956267f..7206028 100644 --- a/scenarios/basic-web-service/expected/deployment-basic-web-service-dev.yaml +++ b/scenarios/basic-web-service/expected/deployment-basic-web-service-dev.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: basic-web-service + deployah.dev/source: spec labels: app.kubernetes.io/instance: basic-web-service-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/basic-web-service/expected/service-basic-web-service-dev.yaml b/scenarios/basic-web-service/expected/service-basic-web-service-dev.yaml index c021417..4203775 100644 --- a/scenarios/basic-web-service/expected/service-basic-web-service-dev.yaml +++ b/scenarios/basic-web-service/expected/service-basic-web-service-dev.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: basic-web-service + deployah.dev/source: spec labels: app.kubernetes.io/instance: basic-web-service-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/command-args-resources/expected/deployment-command-args-resources-production-processor.yaml b/scenarios/command-args-resources/expected/deployment-command-args-resources-production-processor.yaml index aaa41ba..d034f19 100644 --- a/scenarios/command-args-resources/expected/deployment-command-args-resources-production-processor.yaml +++ b/scenarios/command-args-resources/expected/deployment-command-args-resources-production-processor.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: command-args-resources + deployah.dev/source: spec labels: app.kubernetes.io/instance: command-args-resources-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/command-args-resources/expected/service-command-args-resources-production-processor.yaml b/scenarios/command-args-resources/expected/service-command-args-resources-production-processor.yaml index 8a9c3aa..3299001 100644 --- a/scenarios/command-args-resources/expected/service-command-args-resources-production-processor.yaml +++ b/scenarios/command-args-resources/expected/service-command-args-resources-production-processor.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: command-args-resources + deployah.dev/source: spec labels: app.kubernetes.io/instance: command-args-resources-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/env-substitution/expected/deployment-env-substitution-dev-api.yaml b/scenarios/env-substitution/expected/deployment-env-substitution-dev-api.yaml index b9423e9..c4f793b 100644 --- a/scenarios/env-substitution/expected/deployment-env-substitution-dev-api.yaml +++ b/scenarios/env-substitution/expected/deployment-env-substitution-dev-api.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: env-substitution + deployah.dev/source: spec labels: app.kubernetes.io/instance: env-substitution-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/env-substitution/expected/service-env-substitution-dev-api.yaml b/scenarios/env-substitution/expected/service-env-substitution-dev-api.yaml index 8b55cf1..317476d 100644 --- a/scenarios/env-substitution/expected/service-env-substitution-dev-api.yaml +++ b/scenarios/env-substitution/expected/service-env-substitution-dev-api.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: env-substitution + deployah.dev/source: spec labels: app.kubernetes.io/instance: env-substitution-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-apex-certmanager/expected/deployment-expose-apex-certmanager-production-api.yaml b/scenarios/expose-apex-certmanager/expected/deployment-expose-apex-certmanager-production-api.yaml index 2ec2de3..74d88e7 100644 --- a/scenarios/expose-apex-certmanager/expected/deployment-expose-apex-certmanager-production-api.yaml +++ b/scenarios/expose-apex-certmanager/expected/deployment-expose-apex-certmanager-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: expose-apex-certmanager + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-apex-certmanager-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-apex-certmanager/expected/ingress-expose-apex-certmanager-production-api.yaml b/scenarios/expose-apex-certmanager/expected/ingress-expose-apex-certmanager-production-api.yaml index e37e002..20444bf 100644 --- a/scenarios/expose-apex-certmanager/expected/ingress-expose-apex-certmanager-production-api.yaml +++ b/scenarios/expose-apex-certmanager/expected/ingress-expose-apex-certmanager-production-api.yaml @@ -3,6 +3,8 @@ kind: Ingress metadata: annotations: cert-manager.io/cluster-issuer: letsencrypt-prod + deployah.dev/project: expose-apex-certmanager + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-apex-certmanager-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-apex-certmanager/expected/service-expose-apex-certmanager-production-api.yaml b/scenarios/expose-apex-certmanager/expected/service-expose-apex-certmanager-production-api.yaml index e0b43bb..e09a92a 100644 --- a/scenarios/expose-apex-certmanager/expected/service-expose-apex-certmanager-production-api.yaml +++ b/scenarios/expose-apex-certmanager/expected/service-expose-apex-certmanager-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: expose-apex-certmanager + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-apex-certmanager-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-secretname/expected/deployment-expose-secretname-production-api.yaml b/scenarios/expose-secretname/expected/deployment-expose-secretname-production-api.yaml index 935d2f0..e2984cf 100644 --- a/scenarios/expose-secretname/expected/deployment-expose-secretname-production-api.yaml +++ b/scenarios/expose-secretname/expected/deployment-expose-secretname-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: expose-secretname + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-secretname-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-secretname/expected/ingress-expose-secretname-production-api.yaml b/scenarios/expose-secretname/expected/ingress-expose-secretname-production-api.yaml index 2f01493..9fbc8da 100644 --- a/scenarios/expose-secretname/expected/ingress-expose-secretname-production-api.yaml +++ b/scenarios/expose-secretname/expected/ingress-expose-secretname-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: networking.k8s.io/v1 kind: Ingress metadata: + annotations: + deployah.dev/project: expose-secretname + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-secretname-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-secretname/expected/service-expose-secretname-production-api.yaml b/scenarios/expose-secretname/expected/service-expose-secretname-production-api.yaml index 1b06a88..0d60885 100644 --- a/scenarios/expose-secretname/expected/service-expose-secretname-production-api.yaml +++ b/scenarios/expose-secretname/expected/service-expose-secretname-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: expose-secretname + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-secretname-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-selfsigned/expected/deployment-expose-selfsigned-production-web.yaml b/scenarios/expose-selfsigned/expected/deployment-expose-selfsigned-production-web.yaml index c8d74f1..60cb830 100644 --- a/scenarios/expose-selfsigned/expected/deployment-expose-selfsigned-production-web.yaml +++ b/scenarios/expose-selfsigned/expected/deployment-expose-selfsigned-production-web.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: expose-selfsigned + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-selfsigned-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-selfsigned/expected/ingress-expose-selfsigned-production-web.yaml b/scenarios/expose-selfsigned/expected/ingress-expose-selfsigned-production-web.yaml index b4e67f3..a86c720 100644 --- a/scenarios/expose-selfsigned/expected/ingress-expose-selfsigned-production-web.yaml +++ b/scenarios/expose-selfsigned/expected/ingress-expose-selfsigned-production-web.yaml @@ -1,6 +1,9 @@ apiVersion: networking.k8s.io/v1 kind: Ingress metadata: + annotations: + deployah.dev/project: expose-selfsigned + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-selfsigned-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-selfsigned/expected/secret-web.selfsigned.example.com-tls.yaml b/scenarios/expose-selfsigned/expected/secret-web.selfsigned.example.com-tls.yaml index 09a640d..12be1fd 100644 --- a/scenarios/expose-selfsigned/expected/secret-web.selfsigned.example.com-tls.yaml +++ b/scenarios/expose-selfsigned/expected/secret-web.selfsigned.example.com-tls.yaml @@ -4,6 +4,9 @@ data: tls.key: (masked, generated fresh on every render) kind: Secret metadata: + annotations: + deployah.dev/project: expose-selfsigned + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-selfsigned-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/expose-selfsigned/expected/service-expose-selfsigned-production-web.yaml b/scenarios/expose-selfsigned/expected/service-expose-selfsigned-production-web.yaml index ca82186..2bb3f87 100644 --- a/scenarios/expose-selfsigned/expected/service-expose-selfsigned-production-web.yaml +++ b/scenarios/expose-selfsigned/expected/service-expose-selfsigned-production-web.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: expose-selfsigned + deployah.dev/source: spec labels: app.kubernetes.io/instance: expose-selfsigned-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/extras-env-and-crds/.deployah/crds/clusterwidget.yaml b/scenarios/extras-env-and-crds/.deployah/crds/clusterwidget.yaml new file mode 100644 index 0000000..d3ac226 --- /dev/null +++ b/scenarios/extras-env-and-crds/.deployah/crds/clusterwidget.yaml @@ -0,0 +1,21 @@ +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: clusterwidgets.example.com +spec: + group: example.com + scope: Cluster + names: + kind: ClusterWidget + plural: clusterwidgets + singular: clusterwidget + versions: + - name: v1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object diff --git a/scenarios/extras-env-and-crds/.deployah/manifests/common-cm.yaml b/scenarios/extras-env-and-crds/.deployah/manifests/common-cm.yaml new file mode 100644 index 0000000..da300bc --- /dev/null +++ b/scenarios/extras-env-and-crds/.deployah/manifests/common-cm.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: common-extra +data: + scope: all-environments diff --git a/scenarios/extras-env-and-crds/.deployah/manifests/prod/prod-extra.yaml b/scenarios/extras-env-and-crds/.deployah/manifests/prod/prod-extra.yaml new file mode 100644 index 0000000..e32d358 --- /dev/null +++ b/scenarios/extras-env-and-crds/.deployah/manifests/prod/prod-extra.yaml @@ -0,0 +1,6 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: prod-extra +data: + scope: prod-only diff --git a/scenarios/extras-env-and-crds/.deployah/manifests/widget.yaml b/scenarios/extras-env-and-crds/.deployah/manifests/widget.yaml new file mode 100644 index 0000000..699155b --- /dev/null +++ b/scenarios/extras-env-and-crds/.deployah/manifests/widget.yaml @@ -0,0 +1,5 @@ +apiVersion: example.com/v1 +kind: ClusterWidget +metadata: + name: shared-widget +spec: {} diff --git a/scenarios/extras-env-and-crds/deployah.yaml b/scenarios/extras-env-and-crds/deployah.yaml new file mode 100644 index 0000000..b818ff4 --- /dev/null +++ b/scenarios/extras-env-and-crds/deployah.yaml @@ -0,0 +1,12 @@ +# $schema: ../../internal/spec/schema/v1-alpha.2/manifest.json +apiVersion: v1-alpha.2 +project: extras-env-and-crds +components: + web: + image: nginx:latest + port: 8080 + environments: [dev, prod] + resourcePreset: small +environments: + dev: {} + prod: {} diff --git a/scenarios/extras-env-and-crds/expected-dev/clusterwidget-shared-widget.yaml b/scenarios/extras-env-and-crds/expected-dev/clusterwidget-shared-widget.yaml new file mode 100644 index 0000000..0504c41 --- /dev/null +++ b/scenarios/extras-env-and-crds/expected-dev/clusterwidget-shared-widget.yaml @@ -0,0 +1,11 @@ +apiVersion: example.com/v1 +kind: ClusterWidget +metadata: + annotations: + deployah.dev/project: extras-env-and-crds + deployah.dev/source: manifests + labels: + deployah.dev/environment: dev + deployah.dev/project: extras-env-and-crds + name: shared-widget +spec: {} diff --git a/scenarios/extras-env-and-crds/expected-dev/configmap-common-extra.yaml b/scenarios/extras-env-and-crds/expected-dev/configmap-common-extra.yaml new file mode 100644 index 0000000..f921900 --- /dev/null +++ b/scenarios/extras-env-and-crds/expected-dev/configmap-common-extra.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +data: + scope: all-environments +kind: ConfigMap +metadata: + annotations: + deployah.dev/project: extras-env-and-crds + deployah.dev/source: manifests + labels: + deployah.dev/environment: dev + deployah.dev/project: extras-env-and-crds + name: common-extra + namespace: default diff --git a/scenarios/extras-env-and-crds/expected-dev/deployment-extras-env-and-crds-dev-web.yaml b/scenarios/extras-env-and-crds/expected-dev/deployment-extras-env-and-crds-dev-web.yaml new file mode 100644 index 0000000..9405391 --- /dev/null +++ b/scenarios/extras-env-and-crds/expected-dev/deployment-extras-env-and-crds-dev-web.yaml @@ -0,0 +1,81 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployah.dev/project: extras-env-and-crds + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: extras-env-and-crds-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: web + deployah.dev/component: web + deployah.dev/environment: dev + deployah.dev/project: extras-env-and-crds + helm.sh/chart: web-0.1.0 + name: extras-env-and-crds-dev-web + namespace: default +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: extras-env-and-crds-dev + app.kubernetes.io/name: web + strategy: + type: RollingUpdate + template: + metadata: + annotations: null + labels: + app.kubernetes.io/instance: extras-env-and-crds-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: web + deployah.dev/component: web + deployah.dev/environment: dev + deployah.dev/project: extras-env-and-crds + helm.sh/chart: web-0.1.0 + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: extras-env-and-crds-dev + app.kubernetes.io/name: web + topologyKey: kubernetes.io/hostname + weight: 1 + containers: + - image: docker.io/library/nginx:latest + imagePullPolicy: Always + livenessProbe: + failureThreshold: 6 + periodSeconds: 10 + tcpSocket: + port: http + timeoutSeconds: 3 + name: web + ports: + - containerPort: 8080 + name: http + protocol: TCP + readinessProbe: + failureThreshold: 3 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + startupProbe: + failureThreshold: 36 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + restartPolicy: Always + serviceAccountName: default diff --git a/scenarios/extras-env-and-crds/expected-dev/service-extras-env-and-crds-dev-web.yaml b/scenarios/extras-env-and-crds/expected-dev/service-extras-env-and-crds-dev-web.yaml new file mode 100644 index 0000000..dd678ac --- /dev/null +++ b/scenarios/extras-env-and-crds/expected-dev/service-extras-env-and-crds-dev-web.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + deployah.dev/project: extras-env-and-crds + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: extras-env-and-crds-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: web + deployah.dev/component: web + deployah.dev/environment: dev + deployah.dev/project: extras-env-and-crds + helm.sh/chart: web-0.1.0 + name: extras-env-and-crds-dev-web + namespace: default +spec: + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + selector: + app.kubernetes.io/instance: extras-env-and-crds-dev + app.kubernetes.io/name: web + sessionAffinity: None + type: ClusterIP diff --git a/scenarios/extras-env-and-crds/expected-prod/clusterwidget-shared-widget.yaml b/scenarios/extras-env-and-crds/expected-prod/clusterwidget-shared-widget.yaml new file mode 100644 index 0000000..2637817 --- /dev/null +++ b/scenarios/extras-env-and-crds/expected-prod/clusterwidget-shared-widget.yaml @@ -0,0 +1,11 @@ +apiVersion: example.com/v1 +kind: ClusterWidget +metadata: + annotations: + deployah.dev/project: extras-env-and-crds + deployah.dev/source: manifests + labels: + deployah.dev/environment: prod + deployah.dev/project: extras-env-and-crds + name: shared-widget +spec: {} diff --git a/scenarios/extras-env-and-crds/expected-prod/configmap-common-extra.yaml b/scenarios/extras-env-and-crds/expected-prod/configmap-common-extra.yaml new file mode 100644 index 0000000..064216b --- /dev/null +++ b/scenarios/extras-env-and-crds/expected-prod/configmap-common-extra.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +data: + scope: all-environments +kind: ConfigMap +metadata: + annotations: + deployah.dev/project: extras-env-and-crds + deployah.dev/source: manifests + labels: + deployah.dev/environment: prod + deployah.dev/project: extras-env-and-crds + name: common-extra + namespace: default diff --git a/scenarios/extras-env-and-crds/expected-prod/configmap-prod-extra.yaml b/scenarios/extras-env-and-crds/expected-prod/configmap-prod-extra.yaml new file mode 100644 index 0000000..31f0b74 --- /dev/null +++ b/scenarios/extras-env-and-crds/expected-prod/configmap-prod-extra.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +data: + scope: prod-only +kind: ConfigMap +metadata: + annotations: + deployah.dev/project: extras-env-and-crds + deployah.dev/source: manifests + labels: + deployah.dev/environment: prod + deployah.dev/project: extras-env-and-crds + name: prod-extra + namespace: default diff --git a/scenarios/extras-env-and-crds/expected-prod/deployment-extras-env-and-crds-prod-web.yaml b/scenarios/extras-env-and-crds/expected-prod/deployment-extras-env-and-crds-prod-web.yaml new file mode 100644 index 0000000..fcf628f --- /dev/null +++ b/scenarios/extras-env-and-crds/expected-prod/deployment-extras-env-and-crds-prod-web.yaml @@ -0,0 +1,81 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployah.dev/project: extras-env-and-crds + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: extras-env-and-crds-prod + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: web + deployah.dev/component: web + deployah.dev/environment: prod + deployah.dev/project: extras-env-and-crds + helm.sh/chart: web-0.1.0 + name: extras-env-and-crds-prod-web + namespace: default +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: extras-env-and-crds-prod + app.kubernetes.io/name: web + strategy: + type: RollingUpdate + template: + metadata: + annotations: null + labels: + app.kubernetes.io/instance: extras-env-and-crds-prod + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: web + deployah.dev/component: web + deployah.dev/environment: prod + deployah.dev/project: extras-env-and-crds + helm.sh/chart: web-0.1.0 + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: extras-env-and-crds-prod + app.kubernetes.io/name: web + topologyKey: kubernetes.io/hostname + weight: 1 + containers: + - image: docker.io/library/nginx:latest + imagePullPolicy: Always + livenessProbe: + failureThreshold: 6 + periodSeconds: 10 + tcpSocket: + port: http + timeoutSeconds: 3 + name: web + ports: + - containerPort: 8080 + name: http + protocol: TCP + readinessProbe: + failureThreshold: 3 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + startupProbe: + failureThreshold: 36 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + restartPolicy: Always + serviceAccountName: default diff --git a/scenarios/extras-env-and-crds/expected-prod/service-extras-env-and-crds-prod-web.yaml b/scenarios/extras-env-and-crds/expected-prod/service-extras-env-and-crds-prod-web.yaml new file mode 100644 index 0000000..116fb49 --- /dev/null +++ b/scenarios/extras-env-and-crds/expected-prod/service-extras-env-and-crds-prod-web.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + deployah.dev/project: extras-env-and-crds + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: extras-env-and-crds-prod + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: web + deployah.dev/component: web + deployah.dev/environment: prod + deployah.dev/project: extras-env-and-crds + helm.sh/chart: web-0.1.0 + name: extras-env-and-crds-prod-web + namespace: default +spec: + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + selector: + app.kubernetes.io/instance: extras-env-and-crds-prod + app.kubernetes.io/name: web + sessionAffinity: None + type: ClusterIP diff --git a/scenarios/extras-manifest/.deployah/manifests/cm-with-reserved-labels.yaml b/scenarios/extras-manifest/.deployah/manifests/cm-with-reserved-labels.yaml new file mode 100644 index 0000000..c41987a --- /dev/null +++ b/scenarios/extras-manifest/.deployah/manifests/cm-with-reserved-labels.yaml @@ -0,0 +1,16 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: reserved-labels + labels: + app: keep-me + deployah.dev/project: wrong + deployah.dev/component: should-strip + deployah.dev/managed-by: impostor + deployah.dev/version: "99" + annotations: + note: keep + deployah.dev/source: forged + deployah.dev/fakekey: strip-me +data: + key: value diff --git a/scenarios/extras-manifest/.deployah/manifests/networkpolicy.yaml b/scenarios/extras-manifest/.deployah/manifests/networkpolicy.yaml new file mode 100644 index 0000000..ba406f2 --- /dev/null +++ b/scenarios/extras-manifest/.deployah/manifests/networkpolicy.yaml @@ -0,0 +1,11 @@ +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: web-deny-all +spec: + podSelector: + matchLabels: + app.kubernetes.io/name: web + policyTypes: + - Ingress + - Egress diff --git a/scenarios/extras-manifest/.deployah/manifests/prometheusrule.yaml b/scenarios/extras-manifest/.deployah/manifests/prometheusrule.yaml new file mode 100644 index 0000000..9c1ef93 --- /dev/null +++ b/scenarios/extras-manifest/.deployah/manifests/prometheusrule.yaml @@ -0,0 +1,12 @@ +apiVersion: monitoring.coreos.com/v1 +kind: PrometheusRule +metadata: + name: web-alerts +spec: + groups: + - name: web + rules: + - alert: WebDown + expr: up == 0 + annotations: + summary: instance {{ $labels.instance }} is down diff --git a/scenarios/extras-manifest/deployah.yaml b/scenarios/extras-manifest/deployah.yaml new file mode 100644 index 0000000..a63c367 --- /dev/null +++ b/scenarios/extras-manifest/deployah.yaml @@ -0,0 +1,11 @@ +# $schema: ../../internal/spec/schema/v1-alpha.2/manifest.json +apiVersion: v1-alpha.2 +project: extras-manifest +components: + web: + image: nginx:latest + port: 8080 + environments: [dev] + resourcePreset: small +environments: + dev: {} diff --git a/scenarios/extras-manifest/expected/configmap-reserved-labels.yaml b/scenarios/extras-manifest/expected/configmap-reserved-labels.yaml new file mode 100644 index 0000000..5f1e9d7 --- /dev/null +++ b/scenarios/extras-manifest/expected/configmap-reserved-labels.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +data: + key: value +kind: ConfigMap +metadata: + annotations: + deployah.dev/project: extras-manifest + deployah.dev/source: manifests + note: keep + labels: + app: keep-me + deployah.dev/environment: dev + deployah.dev/project: extras-manifest + name: reserved-labels + namespace: default diff --git a/scenarios/extras-manifest/expected/deployment-extras-manifest-dev-web.yaml b/scenarios/extras-manifest/expected/deployment-extras-manifest-dev-web.yaml new file mode 100644 index 0000000..c9150af --- /dev/null +++ b/scenarios/extras-manifest/expected/deployment-extras-manifest-dev-web.yaml @@ -0,0 +1,81 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployah.dev/project: extras-manifest + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: extras-manifest-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: web + deployah.dev/component: web + deployah.dev/environment: dev + deployah.dev/project: extras-manifest + helm.sh/chart: web-0.1.0 + name: extras-manifest-dev-web + namespace: default +spec: + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: extras-manifest-dev + app.kubernetes.io/name: web + strategy: + type: RollingUpdate + template: + metadata: + annotations: null + labels: + app.kubernetes.io/instance: extras-manifest-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: web + deployah.dev/component: web + deployah.dev/environment: dev + deployah.dev/project: extras-manifest + helm.sh/chart: web-0.1.0 + spec: + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - podAffinityTerm: + labelSelector: + matchLabels: + app.kubernetes.io/instance: extras-manifest-dev + app.kubernetes.io/name: web + topologyKey: kubernetes.io/hostname + weight: 1 + containers: + - image: docker.io/library/nginx:latest + imagePullPolicy: Always + livenessProbe: + failureThreshold: 6 + periodSeconds: 10 + tcpSocket: + port: http + timeoutSeconds: 3 + name: web + ports: + - containerPort: 8080 + name: http + protocol: TCP + readinessProbe: + failureThreshold: 3 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + resources: + limits: {} + requests: + cpu: 500m + ephemeral-storage: 50Mi + memory: 512Mi + startupProbe: + failureThreshold: 36 + periodSeconds: 5 + tcpSocket: + port: http + timeoutSeconds: 3 + restartPolicy: Always + serviceAccountName: default diff --git a/scenarios/extras-manifest/expected/networkpolicy-web-deny-all.yaml b/scenarios/extras-manifest/expected/networkpolicy-web-deny-all.yaml new file mode 100644 index 0000000..29224c3 --- /dev/null +++ b/scenarios/extras-manifest/expected/networkpolicy-web-deny-all.yaml @@ -0,0 +1,18 @@ +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + annotations: + deployah.dev/project: extras-manifest + deployah.dev/source: manifests + labels: + deployah.dev/environment: dev + deployah.dev/project: extras-manifest + name: web-deny-all + namespace: default +spec: + podSelector: + matchLabels: + app.kubernetes.io/name: web + policyTypes: + - Ingress + - Egress diff --git a/scenarios/extras-manifest/expected/prometheusrule-web-alerts.yaml b/scenarios/extras-manifest/expected/prometheusrule-web-alerts.yaml new file mode 100644 index 0000000..3b46278 --- /dev/null +++ b/scenarios/extras-manifest/expected/prometheusrule-web-alerts.yaml @@ -0,0 +1,19 @@ +apiVersion: monitoring.coreos.com/v1 +kind: PrometheusRule +metadata: + annotations: + deployah.dev/project: extras-manifest + deployah.dev/source: manifests + labels: + deployah.dev/environment: dev + deployah.dev/project: extras-manifest + name: web-alerts + namespace: default +spec: + groups: + - name: web + rules: + - alert: WebDown + annotations: + summary: instance {{ $labels.instance }} is down + expr: up == 0 diff --git a/scenarios/extras-manifest/expected/service-extras-manifest-dev-web.yaml b/scenarios/extras-manifest/expected/service-extras-manifest-dev-web.yaml new file mode 100644 index 0000000..c0de5a5 --- /dev/null +++ b/scenarios/extras-manifest/expected/service-extras-manifest-dev-web.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: Service +metadata: + annotations: + deployah.dev/project: extras-manifest + deployah.dev/source: spec + labels: + app.kubernetes.io/instance: extras-manifest-dev + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: web + deployah.dev/component: web + deployah.dev/environment: dev + deployah.dev/project: extras-manifest + helm.sh/chart: web-0.1.0 + name: extras-manifest-dev-web + namespace: default +spec: + ports: + - name: http + port: 80 + protocol: TCP + targetPort: http + selector: + app.kubernetes.io/instance: extras-manifest-dev + app.kubernetes.io/name: web + sessionAffinity: None + type: ClusterIP diff --git a/scenarios/health-check-http/expected/deployment-health-check-http-dev-api.yaml b/scenarios/health-check-http/expected/deployment-health-check-http-dev-api.yaml index ce2853b..7ce2c19 100644 --- a/scenarios/health-check-http/expected/deployment-health-check-http-dev-api.yaml +++ b/scenarios/health-check-http/expected/deployment-health-check-http-dev-api.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: health-check-http + deployah.dev/source: spec labels: app.kubernetes.io/instance: health-check-http-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/health-check-http/expected/service-health-check-http-dev-api.yaml b/scenarios/health-check-http/expected/service-health-check-http-dev-api.yaml index 66f082b..d49c138 100644 --- a/scenarios/health-check-http/expected/service-health-check-http-dev-api.yaml +++ b/scenarios/health-check-http/expected/service-health-check-http-dev-api.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: health-check-http + deployah.dev/source: spec labels: app.kubernetes.io/instance: health-check-http-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/health-disabled/expected/deployment-health-disabled-dev-api.yaml b/scenarios/health-disabled/expected/deployment-health-disabled-dev-api.yaml index 65edf55..20a4a48 100644 --- a/scenarios/health-disabled/expected/deployment-health-disabled-dev-api.yaml +++ b/scenarios/health-disabled/expected/deployment-health-disabled-dev-api.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: health-disabled + deployah.dev/source: spec labels: app.kubernetes.io/instance: health-disabled-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/health-disabled/expected/service-health-disabled-dev-api.yaml b/scenarios/health-disabled/expected/service-health-disabled-dev-api.yaml index 74f48fa..520f119 100644 --- a/scenarios/health-disabled/expected/service-health-disabled-dev-api.yaml +++ b/scenarios/health-disabled/expected/service-health-disabled-dev-api.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: health-disabled + deployah.dev/source: spec labels: app.kubernetes.io/instance: health-disabled-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-component/expected/deployment-multi-component-production-api.yaml b/scenarios/multi-component/expected/deployment-multi-component-production-api.yaml index 24e0c27..652281d 100644 --- a/scenarios/multi-component/expected/deployment-multi-component-production-api.yaml +++ b/scenarios/multi-component/expected/deployment-multi-component-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: multi-component + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-component-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-component/expected/deployment-multi-component-production-web.yaml b/scenarios/multi-component/expected/deployment-multi-component-production-web.yaml index dfc8fc0..495972c 100644 --- a/scenarios/multi-component/expected/deployment-multi-component-production-web.yaml +++ b/scenarios/multi-component/expected/deployment-multi-component-production-web.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: multi-component + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-component-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-component/expected/service-multi-component-production-api.yaml b/scenarios/multi-component/expected/service-multi-component-production-api.yaml index 6d2da5f..3892f3e 100644 --- a/scenarios/multi-component/expected/service-multi-component-production-api.yaml +++ b/scenarios/multi-component/expected/service-multi-component-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: multi-component + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-component-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-component/expected/service-multi-component-production-web.yaml b/scenarios/multi-component/expected/service-multi-component-production-web.yaml index 11e105b..8c55f58 100644 --- a/scenarios/multi-component/expected/service-multi-component-production-web.yaml +++ b/scenarios/multi-component/expected/service-multi-component-production-web.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: multi-component + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-component-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-env/expected-dev/deployment-multi-env-dev-debug-sidecar.yaml b/scenarios/multi-env/expected-dev/deployment-multi-env-dev-debug-sidecar.yaml index cee2d6c..ee894c4 100644 --- a/scenarios/multi-env/expected-dev/deployment-multi-env-dev-debug-sidecar.yaml +++ b/scenarios/multi-env/expected-dev/deployment-multi-env-dev-debug-sidecar.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: multi-env + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-env-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-env/expected-dev/deployment-multi-env-dev-web.yaml b/scenarios/multi-env/expected-dev/deployment-multi-env-dev-web.yaml index 2cc1707..95cc5ae 100644 --- a/scenarios/multi-env/expected-dev/deployment-multi-env-dev-web.yaml +++ b/scenarios/multi-env/expected-dev/deployment-multi-env-dev-web.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: multi-env + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-env-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-env/expected-dev/service-multi-env-dev-debug-sidecar.yaml b/scenarios/multi-env/expected-dev/service-multi-env-dev-debug-sidecar.yaml index 06dbe71..6c3de08 100644 --- a/scenarios/multi-env/expected-dev/service-multi-env-dev-debug-sidecar.yaml +++ b/scenarios/multi-env/expected-dev/service-multi-env-dev-debug-sidecar.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: multi-env + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-env-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-env/expected-dev/service-multi-env-dev-web.yaml b/scenarios/multi-env/expected-dev/service-multi-env-dev-web.yaml index 64233c2..7964186 100644 --- a/scenarios/multi-env/expected-dev/service-multi-env-dev-web.yaml +++ b/scenarios/multi-env/expected-dev/service-multi-env-dev-web.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: multi-env + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-env-dev app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-env/expected-production/deployment-multi-env-production-web.yaml b/scenarios/multi-env/expected-production/deployment-multi-env-production-web.yaml index 0848a7c..b693583 100644 --- a/scenarios/multi-env/expected-production/deployment-multi-env-production-web.yaml +++ b/scenarios/multi-env/expected-production/deployment-multi-env-production-web.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: multi-env + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-env-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/multi-env/expected-production/service-multi-env-production-web.yaml b/scenarios/multi-env/expected-production/service-multi-env-production-web.yaml index 9ce93b0..cd8ce62 100644 --- a/scenarios/multi-env/expected-production/service-multi-env-production-web.yaml +++ b/scenarios/multi-env/expected-production/service-multi-env-production-web.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: multi-env + deployah.dev/source: spec labels: app.kubernetes.io/instance: multi-env-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/plan-after-failed-upgrade/previous.yaml b/scenarios/plan-after-failed-upgrade/previous.yaml index d16f3b7..59636e5 100644 --- a/scenarios/plan-after-failed-upgrade/previous.yaml +++ b/scenarios/plan-after-failed-upgrade/previous.yaml @@ -20,6 +20,8 @@ metadata: - manager: helm operation: Update annotations: + deployah.dev/project: plan-after-failed-upgrade + deployah.dev/source: spec meta.helm.sh/release-name: plan-after-failed-upgrade-dev meta.helm.sh/release-namespace: default kubectl.kubernetes.io/last-applied-configuration: "{}" @@ -110,6 +112,8 @@ metadata: generation: 1 creationTimestamp: "2024-01-01T00:00:00Z" annotations: + deployah.dev/project: plan-after-failed-upgrade + deployah.dev/source: spec meta.helm.sh/release-name: plan-after-failed-upgrade-dev meta.helm.sh/release-namespace: default labels: diff --git a/scenarios/plan-extras-fresh-install/.deployah/manifests/networkpolicy.yaml b/scenarios/plan-extras-fresh-install/.deployah/manifests/networkpolicy.yaml new file mode 100644 index 0000000..6a04ade --- /dev/null +++ b/scenarios/plan-extras-fresh-install/.deployah/manifests/networkpolicy.yaml @@ -0,0 +1,10 @@ +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: plan-extras-deny +spec: + podSelector: + matchLabels: + app.kubernetes.io/name: web + policyTypes: + - Ingress diff --git a/scenarios/plan-extras-fresh-install/deployah.yaml b/scenarios/plan-extras-fresh-install/deployah.yaml new file mode 100644 index 0000000..0bc695f --- /dev/null +++ b/scenarios/plan-extras-fresh-install/deployah.yaml @@ -0,0 +1,11 @@ +# $schema: ../../internal/spec/schema/v1-alpha.2/manifest.json +apiVersion: v1-alpha.2 +project: plan-extras-fresh-install +components: + web: + image: nginx:latest + port: 8080 + environments: [dev] + resourcePreset: small +environments: + dev: {} diff --git a/scenarios/plan-extras-fresh-install/plan-config.yaml b/scenarios/plan-extras-fresh-install/plan-config.yaml new file mode 100644 index 0000000..d86d3cd --- /dev/null +++ b/scenarios/plan-extras-fresh-install/plan-config.yaml @@ -0,0 +1,18 @@ +# Fresh install: generated resources plus the extra NetworkPolicy appear as adds. +freshInstall: true + +changes: + - action: add + kind: Deployment + name: plan-extras-fresh-install-dev-web + - action: add + kind: NetworkPolicy + name: plan-extras-deny + - action: add + kind: Service + name: plan-extras-fresh-install-dev-web + +summary: + add: 3 + change: 0 + destroy: 0 diff --git a/scenarios/profile-basic/expected/deployment-profile-basic-production-web.yaml b/scenarios/profile-basic/expected/deployment-profile-basic-production-web.yaml index 04258cc..b3e7146 100644 --- a/scenarios/profile-basic/expected/deployment-profile-basic-production-web.yaml +++ b/scenarios/profile-basic/expected/deployment-profile-basic-production-web.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: profile-basic + deployah.dev/source: spec labels: app.kubernetes.io/instance: profile-basic-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/profile-basic/expected/service-profile-basic-production-web.yaml b/scenarios/profile-basic/expected/service-profile-basic-production-web.yaml index 242efba..7d57d14 100644 --- a/scenarios/profile-basic/expected/service-profile-basic-production-web.yaml +++ b/scenarios/profile-basic/expected/service-profile-basic-production-web.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: profile-basic + deployah.dev/source: spec labels: app.kubernetes.io/instance: profile-basic-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/profile-merge/expected/deployment-profile-merge-production-api.yaml b/scenarios/profile-merge/expected/deployment-profile-merge-production-api.yaml index 37cb94e..d5fad9b 100644 --- a/scenarios/profile-merge/expected/deployment-profile-merge-production-api.yaml +++ b/scenarios/profile-merge/expected/deployment-profile-merge-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: apps/v1 kind: Deployment metadata: + annotations: + deployah.dev/project: profile-merge + deployah.dev/source: spec labels: app.kubernetes.io/instance: profile-merge-production app.kubernetes.io/managed-by: Helm diff --git a/scenarios/profile-merge/expected/service-profile-merge-production-api.yaml b/scenarios/profile-merge/expected/service-profile-merge-production-api.yaml index 4a15f16..af090fb 100644 --- a/scenarios/profile-merge/expected/service-profile-merge-production-api.yaml +++ b/scenarios/profile-merge/expected/service-profile-merge-production-api.yaml @@ -1,6 +1,9 @@ apiVersion: v1 kind: Service metadata: + annotations: + deployah.dev/project: profile-merge + deployah.dev/source: spec labels: app.kubernetes.io/instance: profile-merge-production app.kubernetes.io/managed-by: Helm