Skip to content

[CASCL-1386] Add e2e tests for autoscaling cluster evict-legacy-nodes - #3230

Draft
L3n41c wants to merge 1 commit into
mainfrom
lenaic/CASCL-1386-evict-legacy-nodes-e2e-tests
Draft

[CASCL-1386] Add e2e tests for autoscaling cluster evict-legacy-nodes#3230
L3n41c wants to merge 1 commit into
mainfrom
lenaic/CASCL-1386-evict-legacy-nodes-e2e-tests

Conversation

@L3n41c

@L3n41c L3n41c commented Jul 7, 2026

Copy link
Copy Markdown
Member

What does this PR do?

Adds TestEvictLegacyNodes to the autoscaling e2e suite (test/e2e/tests/autoscaling_suite/), giving the kubectl datadog autoscaling cluster evict-legacy-nodes subcommand its first end-to-end coverage. The subtests run sequentially on the shared EKS cluster, in a deliberately ordered flow:

  1. Refuses when Karpenter not installed — asserts the precondition error (run with --dry-run so a violated precondition can never destructively drain the cluster).
  2. Validation errors — table-driven checks for mutually-exclusive --all/--target, missing target, non-positive --eviction-timeout, and the rejected fargate target (all fail in validate() before any cluster call).
  3. Install — default install (Karpenter + Datadog-managed NodePools).
  4. Dry-run does not mutate--all --dry-run produces a plan, cordons nothing, and leaves the legacy managed node groups intact.
  5. Drain all legacy node groups — real --all eviction; verifies the EKS managed node groups are drained to zero and every workload pod reschedules onto a Datadog-managed Karpenter node.
  6. Evict is idempotent — a second --all run is a no-op (Nothing to evict).
  7. Uninstall — restores the clean post-suite state.

Motivation

evict-legacy-nodes (CASCL-1386) was merged across ~a dozen PRs with thorough unit coverage but no e2e coverage, unlike its siblings install / uninstall / update. This closes that gap and validates the orchestrator end-to-end (preconditions, dry-run safety, the real cordon→drain→scale-to-zero→reschedule path, and idempotency) against a live cluster.

Additional Notes

  • The test reuses the existing suite helpers (runKubectlDatadog, testInstall, testUninstall, waitForAllPodsRunning, waitForPendingPods) and the shared anti-affinity workload; a new runEvict helper centralizes the CLI prefix. It reuses the production constant aws.LabelEKSNodegroup so the test stays coupled to the label the drain code actually keys off.
  • The single-method-with-subtests shape is intentional: the ordering is load-bearing (dry-run must run while the node groups still have nodes; the destructive drain must run after it).
  • Known limitation: the drain subtest scales the EKS managed node groups to zero and does not restore them. This is safe because the e2e framework provisions an ephemeral cluster per run; a reused cluster would start the suite with no managed-node-group capacity. Documented inline.
  • No CI/build-tag changes needed — the e2e_autoscaling job's changes rule already covers test/e2e/tests/autoscaling_suite/**/*, and make e2e-autoscaling-tests builds bin/kubectl-datadog first.

Jira: CASCL-1386

Minimum Agent Versions

Are there minimum versions of the Datadog Agent and/or Cluster Agent required?

  • Agent: n/a
  • Cluster Agent: n/a

Describe your test plan

  • go vet and gofmt pass on the new file.
  • The suite cannot run locally (requires EKS/AWS); it is validated by the e2e_autoscaling GitLab CI job, which builds the plugin and runs make e2e-autoscaling-tests. The new subtests add roughly one extra install + drain + uninstall cycle to the suite runtime.

Checklist

  • PR has at least one valid label: bug, enhancement, refactoring, documentation, tooling, and/or dependencies
  • PR has a milestone or the qa/skip-qa label
  • All commits are signed (see: signing commits)

🤖 Generated with Claude Code

Adds TestEvictLegacyNodes to the autoscaling e2e suite, covering the
evict-legacy-nodes subcommand end-to-end against the shared EKS cluster:
refusal when Karpenter is not installed, flag validation, a dry-run that
mutates nothing, a real --all drain of the EKS managed node groups onto
Datadog-managed Karpenter NodePools, and idempotency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 7, 2026

Copy link
Copy Markdown

Code Coverage

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 46.29% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: e92e2b2 | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant