Skip to content

wip: auto-bootstrap plural on crossplane#3791

Open
zreigz wants to merge 1 commit into
masterfrom
lukasz/prod-4581-auto-bootstrap-plural-on-crossplane-created-eksaksgke
Open

wip: auto-bootstrap plural on crossplane#3791
zreigz wants to merge 1 commit into
masterfrom
lukasz/prod-4581-auto-bootstrap-plural-on-crossplane-created-eksaksgke

Conversation

@zreigz

@zreigz zreigz commented Jul 1, 2026

Copy link
Copy Markdown
Member

Test Plan

Test environment: https://console.your-env.onplural.sh/

Checklist

  • I have added a meaningful title and summary to convey the impact of this PR to a user.
  • If required, I have updated the Plural documentation accordingly.
  • I have added tests to cover my changes.
  • I have deployed the agent to a test environment and verified that it works as expected (required only when changing agent code).

Plural Flow: console

@zreigz zreigz added the enhancement New feature or request label Jul 1, 2026
@linear

linear Bot commented Jul 1, 2026

Copy link
Copy Markdown

PROD-4581

@zreigz zreigz changed the title auto-bootstrap plural on crossplane wip: auto-bootstrap plural on crossplane Jul 1, 2026
@soffi-ai

soffi-ai Bot commented Jul 1, 2026

Copy link
Copy Markdown

Soffi AI Summary

This PR introduces automatic bootstrapping of Plural-managed clusters provisioned via Crossplane, specifically targeting AWS EKS clusters created through the Crossplane AWS provider. Previously, clusters created through Crossplane had no automated path to registration with the Plural Console — this change closes that gap by adding a new PluralCrossplaneCluster CRD and a dedicated controller that watches Crossplane AWS cluster resources and bootstraps them into the Plural fleet.

What was added:

  • A new PluralCrossplaneCluster CRD (api/v1alpha1/pluralcrossplanecluster_types.go) that acts as a declarative trigger: users define a PluralCrossplaneCluster resource referencing a Crossplane-provisioned EKS cluster, and the operator handles the rest.
  • A new reconciler (internal/controller/pluralcrossplanecluster_controller.go) that watches PluralCrossplaneCluster resources and drives the bootstrap lifecycle.
  • A crossplane internal package with provider-agnostic cluster discovery (cluster.go), AWS EKS-specific credential/kubeconfig extraction (aws_eks.go), and AWS cluster auth (aws_cluster_auth.go) logic — with test coverage for scheme registration and auth extraction.
  • Snapshots of the Crossplane AWS provider CRDs (_awsclusters.yaml, _awsclusterauths.yaml) used as reference/test fixtures.
  • Registration of the new controller and Crossplane scheme into the operator's main entrypoint (cmd/agent/main.go, cmd/agent/kubernetes.go).

Dependency changes: The crossplane-runtime library (v1.20.0) was added, along with bumps to several existing dependencies (docker/cli, containerd/stargz-snapshotter/estargz, various go-openapi/swag sub-packages) to satisfy the new transitive dependency tree.

Commits

Commit Summary
7d25dba Introduces the PluralCrossplaneCluster CRD, its reconciler, and the internal crossplane package containing AWS EKS cluster discovery and auth extraction logic. Registers the new controller and Crossplane scheme into the deployment-operator agent. Updates go.mod/go.sum to add crossplane-runtime v1.20.0 and bump related transitive dependencies.

Deploy in Soffi


Updated: 2026-07-01 13:39 UTC

@socket-security

Copy link
Copy Markdown

@socket-security

Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action Severity Alert  (click "▶" to expand/collapse)
Warn High
Obfuscated code: golang github.com/crossplane/crossplane-runtime is 90.0% likely obfuscated

Confidence: 0.90

Location: Package overview

From: go/deployment-operator/go.modgolang/github.com/crossplane/crossplane-runtime@v1.20.0

ℹ Read more on: This package | This alert | What is obfuscated code?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore golang/github.com/crossplane/crossplane-runtime@v1.20.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report

@greptile-apps

greptile-apps Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds a new PluralCrossplaneCluster CRD and controller that auto-bootstraps a Plural-managed cluster from a Crossplane-provisioned one, targeting AWS EKS (both crossplane-contrib and upbound providers). Once the Crossplane cluster is ready, the controller syncs the cluster to the Console API and deploys the Plural agent via Helm into the target cluster.

  • Introduces PluralCrossplaneCluster type with full RBAC binding support, console token secret ref, and flexible agent Helm configuration.
  • Adds a crossplane package with provider dispatch, AWS EKS mirror types, connection secret resolution (including ClusterAuth-based fallback), and scheme registration.
  • Contains a reconciliation bug where the spec hash is persisted and the resource is marked Ready=True even when the Helm agent deployment returns an "in-progress" error, preventing the controller from ever retrying the install.

Confidence Score: 3/5

The controller can permanently skip deploying the Plural agent into the target cluster if Helm happens to be busy during the first reconciliation pass.

When deployAgent returns a Helm "in progress" error, the code falls through and writes Status.SHA + Status.ID and sets Ready=True. The next reconcile sees HasID() && !changed and exits early, so the agent install is never retried — the cluster appears bootstrapped in Console but the agent is absent. This affects the primary purpose of the controller.

go/deployment-operator/internal/controller/pluralcrossplanecluster_controller.go — specifically the Helm "in progress" fallthrough around line 143.

Important Files Changed

Filename Overview
go/deployment-operator/internal/controller/pluralcrossplanecluster_controller.go New controller for PluralCrossplaneCluster; contains a bug where the spec hash is persisted even when the Helm agent deployment returns an "in progress" error, preventing future reconciliation retries.
go/deployment-operator/internal/crossplane/cluster.go Well-structured ManagedCluster interface, connection secret resolution, and kubeconfig extraction utilities; logic is clean and tested.
go/deployment-operator/internal/crossplane/aws_eks.go Local mirror types for AWS EKS Crossplane clusters (crossplane-contrib and upbound v1beta1/v1beta2) with proper DeepCopy implementations; no issues found.
go/deployment-operator/internal/crossplane/aws_cluster_auth.go Hydrates connection secret config from a ClusterAuth resource when the cluster itself has none; unfiltered cluster-wide List could cause false matches at scale.
go/deployment-operator/internal/crossplane/provider.go Provider dispatch with explicit ErrUnsupportedProvider sentinel; extensible structure for future GKE/AKS support.
go/deployment-operator/internal/crossplane/scheme.go Registers all three AWS EKS GVKs (crossplane-contrib v1beta1, upbound v1beta1, upbound v1beta2) into a runtime Scheme; straightforward and tested.
go/deployment-operator/api/v1alpha1/pluralcrossplanecluster_types.go CRD type definitions and helper methods for PluralCrossplaneCluster; clean implementation with proper attribute conversion for Console API.
go/deployment-operator/cmd/agent/kubernetes.go Registers PluralCrossplaneClusterController with the manager; consistent with existing controller registration pattern in the file.
go/deployment-operator/cmd/agent/main.go Adds crossplane scheme to the global scheme via AddToScheme; minimal, correct change.

Reviews (1): Last reviewed commit: "plural crossplane cluster" | Re-trigger Greptile

Comment on lines +143 to +154
}
return ctrl.Result{}, err
}

if err = in.deployAgent(pluralCrossplaneCluster, kubeconfig, deployToken); err != nil {
if !strings.Contains(err.Error(), "another operation (install/upgrade/rollback) is in progress") {
logger.Error(err, "failed to deploy agent")
return reconcile.Result{}, err
}
}

pluralCrossplaneCluster.Status.SHA = lo.ToPtr(sha)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 SHA saved when Helm is still in-progress, blocking future retries

When deployAgent returns the "another operation is in progress" error, the code falls through to lines 150–153, saving the spec hash to Status.SHA and marking ReadyConditionType as True. On the next reconciliation, the HasID() && !changed early-exit fires and the controller returns immediately — the agent install is never retried. The cluster will appear Ready in Console even though the Helm chart was never successfully deployed.

Comment on lines +37 to +41
list := &unstructured.UnstructuredList{}
list.SetGroupVersionKind(authGV.WithKind("ClusterAuthList"))
if err := c.List(ctx, list); err != nil {
return nil, false, err
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Unfiltered cluster-wide list of ClusterAuth objects

c.List without any ListOptions fetches every ClusterAuth across all namespaces. This is not wrong today, but if many clusters exist the first matching ClusterAuth whose clusterName/clusterNameRef happens to equal the target name wins — even if it lives in an unrelated namespace. Passing an explicit namespace option (or the cluster's own namespace) would scope the search and avoid false matches.

Suggested change
list := &unstructured.UnstructuredList{}
list.SetGroupVersionKind(authGV.WithKind("ClusterAuthList"))
if err := c.List(ctx, list); err != nil {
return nil, false, err
}
list := &unstructured.UnstructuredList{}
list.SetGroupVersionKind(authGV.WithKind("ClusterAuthList"))
if err := c.List(ctx, list, k8sClient.InNamespace(corev1.NamespaceAll)); err != nil {
return nil, false, err
}

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant