Add support for vpc.controlPlaneOnPrivateSubnets

[guessi]

Description

Closes #8792.

When eksctl creates a VPC, it currently passes both the public and the private subnets to the EKS API, so the control plane's cross-account ENIs are placed in public subnets as well as private ones. Restricting them to private subnets afterwards requires eksctl utils update-cluster-vpc-config, which calls the EKS API directly and leaves the cluster's CloudFormation stack out of sync with the actual configuration.

This PR adds a vpc.controlPlaneOnPrivateSubnets field so that only the private subnets are passed to the EKS API at cluster creation time. Public subnets are still created and used for NAT gateways and internet-facing load balancers — this does not make the cluster fully private, it only changes which subnets the control plane's ENIs land in.

Key implementation points:

• The field is honoured for eksctl-created VPCs (IPv4 and IPv6, including fully-private clusters) and for pre-existing/imported VPCs.
• It is rejected when combined with vpc.controlPlaneSubnetIDs, since that field already gives explicit control over control plane subnets.
• It is rejected when the configured private subnets do not cover at least two availability zones, which EKS requires. When availabilityZones is set explicitly, distinct zones are counted, since duplicates are permitted elsewhere and collapse into a single private subnet. When subnets are pre-existing, their zones are counted directly. The check is best-effort and is skipped in the two cases where the zones are not yet known at validation time:
  • availabilityZones is left unset, so eksctl selects the zones itself in eks.SetAvailabilityZones, which runs after validation. Auto-selection always yields distinct zones, so there is nothing to check.
  • Subnets are given only by ID, so their AZ is resolved from EC2 after validation runs. Those are allowed through and rejected by the EKS API instead if insufficient.
• Outposts clusters are exempt, since the control plane there is already private-only.
• eksctl utils update-cluster-vpc-config now warns and ignores the field (consistent with how eksctl update nodegroup handles other unsupported fields) and points users at vpc.controlPlaneSubnetIDs instead, since that command talks to the EKS API directly and can't keep the CloudFormation stack in sync.
• Control plane subnet selection is now built through a single helper. autoMode is passed into it rather than derived internally, so the new field keeps applying only to VPCs that eksctl creates — the pre-existing VPC path has never restricted the control plane for Auto Mode clusters, and deriving autoMode there would have changed that behaviour for configurations not using the new field.

See examples/49-control-plane-on-private-subnets.yaml for a usage example, and the updated userdocs/src/usage/cluster-subnets-security-groups.md for documentation.

Checklist

• Added tests that cover your change (if possible)
• Added/modified documentation as required (such as the README.md, or the userdocs directory)
• Manually tested
• Made sure the title of the PR is a good description that can go into the release notes
• (Core team) Added labels for change area (e.g. area/nodegroup) and kind (e.g. kind/improvement)

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

• Backfilled missing tests for code in same general area 🎉
• Refactored something and made the world a better place 🌟

[gustavodiaz7722]

The multi-AZ validation doesn't cover the path the docs advertise. validateControlPlaneOnPrivateSubnets bails out when VPC.Subnets == nil, assuming SetSubnets guarantees one private subnet per requested zone. It doesn't: validateAvailabilityZones only checks count and explicitly permits duplicates, and SetSubnets keys private subnets by zone name, so availabilityZones: [us-west-2a, us-west-2a] collapses to one subnet. I confirmed this passes validation with no complaint. Either count distinct entries in c.AvailabilityZones in that branch (three lines), or drop the docs claim that "eksctl validates this before creating anything." The first is better.

The update-cluster-vpc-config test needs to stop calling AWS. newMockCmd drives the real command with no provider mocking, so once the loader passes, doUpdateClusterVPCConfig reaches STS and EKS. It currently only passes because credentials fail to resolve, and its assertion sits inside if err != nil, so it asserts nothing when the call succeeds. Every other case in that file fails before provider construction. Assert against cmdutils.NewUpdateClusterVPCLoader(...).Load() directly.

The new hard error in update_cluster_vpc.go directly contradicts the logger.Info on the line above it, which promises other fields "will be ignored." Live, they print back to back. The codebase convention for this situation is logger.Warning (see eksctl update nodegroup in configfile.go:1018), and the strictness is selective anyway since create nodegroup -f and upgrade cluster -f still ignore the field silently. I lean toward a warning, because rejecting breaks the documented one-config-file-per-cluster workflow.

[gustavodiaz7722]

@guessi Thanks for raising this PR, overall looks good.

[guessi]

@gustavodiaz7722 thanks for your time to review PR, I've submit a new commit to address mentioned concerns, please review.

[gustavodiaz7722]

The new AZ check rejects configs where eksctl selects the zones

availabilityZones is optional. When it's omitted, eksctl picks the zones itself in
eks.SetAvailabilityZones (pkg/ctl/create/cluster.go:590), which runs well after validation
(cluster.go:88). So the slice is still empty when validateControlPlaneOnPrivateSubnets reads it,
and the simplest form of the feature is now rejected:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: example
  region: us-west-2
vpc:
  controlPlaneOnPrivateSubnets: true

Same file against both revisions of this PR, via create cluster --dry-run:

009491b21 (before) -> ACCEPTED; eksctl selected us-west-2d, us-west-2a, us-west-2b
3fa7d7b0f (after)  -> Error: vpc.controlPlaneOnPrivateSubnets requires at least 2
                      distinct availability zones, got 0 ([])

eksctl had chosen three distinct zones, so that cluster would have created fine. The new
"...without availability zones set" test locks this in as expected behaviour, which is why CI
stays green.

Suggested fix

Gate on a non-empty slice. Auto-selection always yields distinct zones, so there is nothing to
validate on that path:

if c.VPC.Subnets == nil {
    if len(c.AvailabilityZones) > 0 {
        if azs := sets.New(c.AvailabilityZones...); azs.Len() < MinRequiredAvailabilityZones {
            return fmt.Errorf("vpc.controlPlaneOnPrivateSubnets requires at least %d distinct availability zones, got %d (%v)",
                MinRequiredAvailabilityZones, azs.Len(), c.AvailabilityZones)
        }
    }
    return nil
}

plus flipping that test to expect success. The [us-west-2a, us-west-2a] rejection you added still
works after this change; I verified it.

[guessi]

@gustavodiaz7722 Cool, thanks for keeping it at highest standard, will definitely revise my code to get it fixed. Thanks!

[guessi]

@gustavodiaz7722 Thanks, I've adopted your suggested fix in 2d2cbe7.

Another commit, 20d7adc, strengthens the loader-test assertion, with the corresponding docs update.

[guessi]

the PR here is based on v0.229.0, I noticed there have tons of fixed introduced by v0.230.0, let me know if I should rebase/squash the commits, thanks!