Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -212,8 +212,8 @@ test-unit: vet ## Run unit tests.

# E2E_TIMEOUT is the timeout for e2e tests.
E2E_TIMEOUT ?= 1h
# E2E_GINKGO_LABEL_FILTER is ginkgo label query for selecting tests. See
# https://onsi.github.io/ginkgo/#spec-labels. Default runs Platform:AWS and Platform:Generic tests; excludes Feature:Proxy, Feature:Upgrade, and Provider:Bitwarden.
# E2E_GINKGO_LABEL_FILTER is ginkgo label query for selecting tests. See https://onsi.github.io/ginkgo/#spec-labels.
# Default runs Platform:AWS and Platform:Generic tests; excludes Feature:Proxy, Feature:Upgrade, and Provider:Bitwarden.
E2E_GINKGO_LABEL_FILTER ?= Platform: isSubsetOf {AWS,Generic} && !(Feature: containsAny {Proxy, Upgrade}) && !(Provider: containsAny Bitwarden)
.PHONY: test-e2e
test-e2e: ## Run e2e tests against a cluster.
Expand Down
38 changes: 30 additions & 8 deletions test/e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ make test-e2e E2E_GINKGO_LABEL_FILTER=""
| Key | Values | Meaning |
|-----|--------|---------|
| `Platform` | `AWS`, `GCP`, `Generic` | Cluster or portability requirement |
| `Provider` | `AWS`, `Bitwarden` | External secret backend integration |
| `Provider` | `AWS`, `Bitwarden`, `Vault` | External secret backend integration |
| `API` | `Bitwarden` | Direct HTTP tests against bitwarden-sdk-server |
| `Feature` | see below | Optional capability or functional area |

Expand All @@ -34,7 +34,8 @@ make test-e2e E2E_GINKGO_LABEL_FILTER=""
| `CustomLabels` | Custom and managed label lifecycle |
| `NetworkPolicy` | Static and custom network policy naming |
| `Proxy` | Proxy egress network policy (requires cluster-wide OpenShift proxy) |
| `TrustedCABundle` | trustedCABundle ConfigMap mounting and validation |
| `TrustedCABundle` | trustedCABundle ConfigMap mounting/validation and Vault TLS failure→recovery |
| `ExternalSecretsTemplating` | ExternalSecret template merge (Kubernetes + Vault → dockerconfigjson) |
| `Upgrade` | Post-upgrade migration checks (temporary) |

## Default filter
Expand All @@ -51,10 +52,6 @@ This runs portable tests plus AWS provider tests, and **API:Bitwarden** health/a

### Secrets you provision

```bash
hack/e2e-setup-secrets.sh setup
```

| Secret | Namespace | Keys | Required when filter includes |
|--------|-----------|------|-------------------------------|
| `aws-creds` | `kube-system` | `aws_access_key_id`, `aws_secret_access_key` | `Provider:AWS` (`Platform:AWS` or `Platform:GCP`) |
Expand All @@ -73,6 +70,7 @@ The bitwarden-sdk-server plugin uses **`bitwarden-tls-certs`** (TLS materials fo
| Requirement | Required when filter includes |
|-------------|-------------------------------|
| Cluster-wide OpenShift proxy (`proxy.config.openshift.io/cluster`) | `Feature:Proxy` |
| OpenShift with `redhat-operators` catalog (`openshift-marketplace`) | Main e2e Describe (`e2e_test.go`) — root `BeforeAll` installs Red Hat cert-manager Operator via OLM if not already present |

If a prerequisite is missing, the affected spec **fails** with a message pointing here — it does not skip.

Expand All @@ -86,9 +84,11 @@ If a prerequisite is missing, the affected spec **fails** with a message pointin
| `Platform:GCP && Provider:AWS` | GCP cluster using AWS Secrets Manager |
| `Provider:AWS` | Any AWS Secrets Manager integration (`Platform:AWS` or `Platform:GCP`) |
| `Provider:Bitwarden` | Bitwarden provider sync and API Secrets API |
| `Provider:Vault` | Vault HTTPS + trustedCABundle failure/recovery (uses suite-installed Red Hat cert-manager Operator) |
| `API:Bitwarden` | bitwarden-sdk-server HTTP API (deploys plugin + `bitwarden-tls-certs` automatically) |
| `API:Bitwarden \|\| Provider:Bitwarden` | All Bitwarden HTTP and provider tests (requires `bitwarden-creds` for Secrets API / provider sync) |
| `Feature:TrustedCABundle` | Trusted CA bundle suite |
| `Feature:TrustedCABundle` | Trusted CA bundle suite (mount/validation + Vault TLS path when combined with `Provider:Vault`) |
| `Feature:ExternalSecretsTemplating` | ExternalSecret templating merge (Kubernetes + Vault dockerconfig) |
Comment on lines +87 to +91

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Keep the executable default filter aligned with the Vault documentation.

The README and PR objective say Provider:Vault is excluded by default, but Makefile Line 217 excludes only Bitwarden. Vault specs that satisfy the platform predicate can therefore run during make test-e2e, unexpectedly requiring cert-manager/OLM prerequisites.

Update the Makefile filter to exclude {Bitwarden, Vault}, or correct this documentation if Vault is intended to run by default.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/README.md` around lines 87 - 91, The executable default e2e filter
must match the documented exclusion of Vault. Update the default filter in the
Makefile target governing make test-e2e to exclude both Bitwarden and Vault,
preserving the existing behavior for other suites and avoiding changes to the
README unless Vault is intentionally meant to run by default.

| `Feature:Proxy` | Proxy egress network policy |
| `Feature:Upgrade` | Post-upgrade network policy migration check |
| `Feature:NetworkPolicy` | Static and custom network policy naming |
Expand All @@ -110,6 +110,9 @@ make test-e2e E2E_GINKGO_LABEL_FILTER="Feature:NetworkPolicy || Feature:Proxy"

# AWS integration only (any platform label that uses AWS SM)
make test-e2e E2E_GINKGO_LABEL_FILTER="Provider:AWS"

# Vault TLS + trustedCABundle (uses cert-manager from root BeforeAll)
make test-e2e E2E_GINKGO_LABEL_FILTER="Provider:Vault"
```

## Specs by label
Expand Down Expand Up @@ -147,10 +150,29 @@ File: `trusted_ca_bundle_test.go`

| Feature | Describe |
|---------|----------|
| `TrustedCABundle` | Trusted CA Bundle |
| `TrustedCABundle` | Trusted CA Bundle (mount / `SSL_CERT_DIR` / Degraded / watch-label restore) |

The **Custom Network Policy Naming** spec adds a dummy egress port to `ExternalSecretsConfig` (if not already present — entries cannot be removed due to CEL immutability), verifies the operator creates `eso-user-e2e-test-custom-np` in the operand namespace (`external-secrets`), and leaves the CR entry in place.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Move the Custom Network Policy paragraph to its owning section.

This paragraph appears immediately under trusted_ca_bundle_test.go, although the feature table identifies Custom Network Policy Naming as an e2e_test.go NetworkPolicy spec. Move it near the NetworkPolicy entry to keep the documentation mapping accurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/README.md` at line 155, Move the “Custom Network Policy Naming”
paragraph from beneath trusted_ca_bundle_test.go to the section containing the
NetworkPolicy entry for e2e_test.go. Preserve the paragraph text unchanged and
keep it associated with the NetworkPolicy spec documentation.


### `Provider:Vault` + `Feature:TrustedCABundle`

File: `e2e_test.go` — **Vault Secret Manager**

- **BeforeAll:** cert-manager CA (`isCA`) + server Certificate for Vault HTTPS; Vault deploy/init
- Uses Red Hat cert-manager Operator installed by the root `BeforeAll` of the main e2e Describe (`ensureCertManagerOperatorReady`)
- Asserts SecretStore `Ready=False` / `InvalidProviderConfig` when `trustedCABundle` points at a valid but non-matching CA (`vault-e2e-sample-ca`), then Ready after switching to the Vault CA ConfigMap (`vault-server-ca` from the isCA Certificate)
- After trustedCABundle switch: SecretStore Ready → PushSecret → ExternalSecret → verify synced Secret
- Excluded from the default label filter (longer Vault HTTPS setup)
- Does not uninstall cert-manager on teardown (shared for other suites)

### `Provider:Vault` + `Feature:ExternalSecretsTemplating`

File: `e2e_test.go` — **Vault Secret Manager** (same Context)

- Creates a namespaced Kubernetes SecretStore (`kubernetes-backend`) with dedicated `eso-secret-reader` SA/Role/RoleBinding (no existing e2e Kubernetes-provider fixture)
- Pushes a dockerconfigjson Secret to Vault, reads another from Kubernetes, merges both via ExternalSecret `target.template` into `merged-registry-pull-secret`
- Asserts the target Secret is `kubernetes.io/dockerconfigjson` and contains both registry auth hosts

### `Provider:Bitwarden`

File: `bitwarden_es_test.go` — **Bitwarden Provider**
Expand Down
184 changes: 184 additions & 0 deletions test/e2e/cert_manager_helpers_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
//go:build e2e
// +build e2e

/*
Copyright 2026.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package e2e

import (
"context"
"fmt"
"strings"
"time"

corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/client-go/dynamic"
"k8s.io/client-go/kubernetes"

. "github.com/onsi/ginkgo/v2"

"github.com/openshift/external-secrets-operator/test/utils"
)

const (
certManagerOperatorNamespace = "cert-manager-operator"
certManagerOperandNamespace = "cert-manager"
certManagerOperatorManifest = "testdata/cert-manager/operator.yaml"

certManagerOperatorPodPrefix = "cert-manager-operator-controller-manager-"
certManagerCAInjectorPodPrefix = "cert-manager-cainjector-"
certManagerWebhookPodPrefix = "cert-manager-webhook-"

certManagerInstallTimeout = 10 * time.Minute
)

var certificateGVR = schema.GroupVersionResource{
Group: "cert-manager.io",
Version: "v1",
Resource: "certificates",
}

// ensureCertManagerOperatorReady installs the Red Hat cert-manager Operator via OLM if needed,
// then waits until the operator, operand pods, and Certificate API are ready.
// Idempotent: skips install when the Certificate API is already usable and operands are Ready.
func ensureCertManagerOperatorReady(ctx context.Context, clientset *kubernetes.Clientset, dynamicClient dynamic.Interface) error {
By("Ensuring Red Hat cert-manager Operator is ready")

if err := certManagerCertificateCRDInstalled(ctx, dynamicClient); err == nil {
if err := waitForCertManagerOperandPods(ctx, clientset, 2*time.Minute); err == nil {
By("cert-manager Certificate API and operand pods already ready")
return nil
}
}

By("Applying cert-manager Operator OLM manifests (Namespace, OperatorGroup, Subscription)")
if err := utils.ApplyManifestFromReader(ctx, dynamicClient, testassets.ReadFile, certManagerOperatorManifest); err != nil {
return fmt.Errorf("apply cert-manager operator manifests: %w", err)
}

By(fmt.Sprintf("Waiting for cert-manager operator pod in namespace %s", certManagerOperatorNamespace))
if err := waitForReadyPodsByNamePrefixes(ctx, clientset, certManagerOperatorNamespace, []string{certManagerOperatorPodPrefix}, certManagerInstallTimeout); err != nil {
return fmt.Errorf("wait for cert-manager operator pod: %w", err)
}

By(fmt.Sprintf("Waiting for cert-manager operand pods in namespace %s", certManagerOperandNamespace))
if err := waitForCertManagerOperandPods(ctx, clientset, certManagerInstallTimeout); err != nil {
return fmt.Errorf("wait for cert-manager operand pods: %w", err)
}

By("Waiting for cert-manager Certificate API")
if err := wait.PollUntilContextTimeout(ctx, 5*time.Second, 2*time.Minute, true, func(ctx context.Context) (bool, error) {
if err := certManagerCertificateCRDInstalled(ctx, dynamicClient); err != nil {
return false, nil
}
return true, nil
}); err != nil {
return fmt.Errorf("cert-manager Certificate API not available after install: %w", err)
}

By("Red Hat cert-manager Operator is ready")
return nil
}

// certManagerCertificateCRDInstalled returns nil when the cert-manager Certificate API is available.
func certManagerCertificateCRDInstalled(ctx context.Context, dynamicClient dynamic.Interface) error {
_, err := dynamicClient.Resource(certificateGVR).Namespace("default").List(ctx, metav1.ListOptions{Limit: 1})
if err != nil {
return fmt.Errorf("cert-manager Certificate API unavailable: %w", err)
}
return nil
}

// waitForCertManagerOperandPods waits until controller, cainjector, and webhook pods are Ready.
func waitForCertManagerOperandPods(ctx context.Context, clientset kubernetes.Interface, timeout time.Duration) error {
return wait.PollUntilContextTimeout(ctx, 5*time.Second, timeout, true, func(ctx context.Context) (bool, error) {
pods, err := clientset.CoreV1().Pods(certManagerOperandNamespace).List(ctx, metav1.ListOptions{})
if err != nil {
if strings.Contains(err.Error(), "not found") {
return false, nil
}
return false, err
}

var hasController, hasCAInjector, hasWebhook bool
for i := range pods.Items {
pod := &pods.Items[i]
if pod.Status.Phase != corev1.PodRunning || !isPodReadyForCertManager(pod) {
continue
}
name := pod.Name
switch {
case strings.HasPrefix(name, certManagerCAInjectorPodPrefix):
hasCAInjector = true
case strings.HasPrefix(name, certManagerWebhookPodPrefix):
hasWebhook = true
case strings.HasPrefix(name, "cert-manager-") &&
!strings.HasPrefix(name, certManagerCAInjectorPodPrefix) &&
!strings.HasPrefix(name, certManagerWebhookPodPrefix) &&
!strings.Contains(name, "startupapicheck"):
// Main controller Deployment pods: cert-manager-<replicaset>-<pod>
hasController = true
}
}
return hasController && hasCAInjector && hasWebhook, nil
})
}

func waitForReadyPodsByNamePrefixes(ctx context.Context, clientset kubernetes.Interface, namespace string, prefixes []string, timeout time.Duration) error {
return wait.PollUntilContextTimeout(ctx, 5*time.Second, timeout, true, func(ctx context.Context) (bool, error) {
pods, err := clientset.CoreV1().Pods(namespace).List(ctx, metav1.ListOptions{})
if err != nil {
return false, err
}
matched := make(map[string]bool, len(prefixes))
for _, prefix := range prefixes {
matched[prefix] = false
}
for i := range pods.Items {
pod := &pods.Items[i]
if pod.Status.Phase != corev1.PodRunning || !isPodReadyForCertManager(pod) {
continue
}
for _, prefix := range prefixes {
if strings.HasPrefix(pod.Name, prefix) {
matched[prefix] = true
}
}
}
for _, ok := range matched {
if !ok {
return false, nil
}
}
return true, nil
})
}

func isPodReadyForCertManager(pod *corev1.Pod) bool {
ready := map[string]bool{
"Ready": false,
"ContainersReady": false,
}
for _, cond := range pod.Status.Conditions {
if _, ok := ready[string(cond.Type)]; ok && cond.Status == corev1.ConditionTrue {
ready[string(cond.Type)] = true
}
}
return ready["Ready"] && ready["ContainersReady"]
}
Loading