1.12.269.0

We are excited to release the April Update of AKS Edge Essentials.

⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

Version Numbers

• AKS Edge Essentials Version: 1.12.269.0

Note: There are multiple 1.12 versions available for upgrade. Please see the available upgrade paths below. For fresh deployments, the version will be 1.12.269.0.

• Azure Linux Version: 3.0.20260401

• K8s (upstream Kubernetes) distribution: 1.31.5 (only available as upgrade from 1.11.247.0), 1.32.9 (available as upgrade from 1.11.247.0 or 1.12.267.0), 1.33.5 (available as upgrade from 1.12.268.0 or fresh install)

• K3s distribution: 1.31.6 (only available as upgrade from 1.11.247.0), 1.32.9 (available as upgrade from 1.11.247.0 or 1.12.267.0), 1.33.5 (available as upgrade from 1.12.268.0 or fresh install)

• Json version schema: 1.16

Available Upgrade/Update Paths

Current version	Upgrade path
1.11.247.0 k8s/k3s 1.30	1.12.267 k8s/k3s 1.31
1.11.247.0 k8s/k3s 1.31	1.12.268 k8s/k3s 1.32
1.12.267 k8s/k3s 1.31	1.12.268 k8s/k3s 1.32
1.12.268 k8s/k3s 1.32	1.12.269 k8s/k3s 1.33

Please note that, unlike prior releases, in this release, upgrading from one Kubernetes distribution to another within the same release version is supported. To support this, we have released multiple upgrade versions of this release, as noted above.

Note: Please ensure 19GB of free disk space before attempting to upgrade to or fresh install an AKS Edge Essentials 1.12 release.

New Features

• Cross-distribution upgrades supported. With this release, you can upgrade Kubernetes versions within the same release version. See the upgrade paths table above for supported combinations. In accordance with the upstream Kubernetes lifecycle, we recommend upgrading to or installing K8s/K3s version 1.33.

• KMS enabled by default on new deployments. The KMS provider, generally available since version 1.11.247.0, is now enabled by default when deploying a new cluster on version 1.12.269.0 and later. No additional configuration is required.

Experimental or Preview Features

• Key manager for Kubernetes (preview)

• Run Windows containerized workloads (preview)

• Create multi-node deployments on a single machine or across multiple machines (preview)

• GPU-PV (preview)

For more information on the full feature set and benefits, see AKS Edge Essentials Overview.

Key Bug Fixes

• Fixed: Deployment on Windows 11 Insider builds could intermittently fail with the error "Linux node created, failed to do first boot initialization". This was caused by a race condition where the cloudinit.success file was checked before it was written. This issue is now resolved.

Known issues

Upgrade

• Cluster upgrades may fail during the "finalizing update" phase of a Trident-based update after the node VM reboots. You may see an error similar to:

  /EFI/AZLB/bootx64.efi doesn't exist
  

  This occurs when the EFI bootloader file is missing or inaccessible after servicing completes.

  Workaround:

  Connect to the Linux node VM from your Windows host and manually re-run the Trident update:

  1. Open an elevated PowerShell window on the host.

  2. List running VMs and find the AKS Edge Essentials node VM ID:

     hcsdiag list

  3. Connect to the VM using the ID from the previous step:

     hcsdiag console <VM-ID>

  4. Inside the VM, check the Trident status and re-run the update:

     trident get                                    # check the current trident status
     trident offline-initialize host-status.yaml    # verify the trident binary is online
     trident run                                    # re-run the update

Deployment

• Always restart your machine after uninstalling AKS Edge Essentials and before reinstalling. Skipping the restart may cause "deployment already exists" errors, "Virtual machine creation failed" errors, or event logging failures.

• If deployment fails due to insufficient memory or disk space, incomplete VHDX files may be left behind. Manually delete these files before retrying, as they can cause errors in subsequent deployments.

• Deployment with an External virtual switch fails on Windows RS5. Use an Internal virtual switch or a newer Windows version instead.

Linux nodes

• After using Remove-AksEdgeNode to remove a node, it may still appear in kubectl get nodes temporarily while the cluster reconciles. This is expected. The command is asynchronous and returns before the cluster state fully updates.

Preview features

• On K3s Windows nodes, the klipper-lb load balancer pulls images from the internet when connected. Ensure network access is available or pre-pull the required images for air-gapped environments.

• In K3s multi-machine clusters, scaling down the cluster can cause it to become unhealthy. To avoid this, do not remove the current etcd leader node. If you need to remove the leader, first rotate leadership to another node by rebooting the current leader, then safely remove it.

1.11.247.0

We are excited to release the September Update of AKS Edge Essentials.

⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

Version Numbers

• AKS Edge Essentials Version: 1.11.247.0

• Azure Linux (previously Mariner) Version: 3.0.20250822

• K8s (upstream Kubernetes) distribution: 1.30.5 (only available as upgrade from 1.10.868.0), 1.31.5

• K3s distribution: 1.30.6 (only available as upgrade from 1.10.868.0), 1.31.6

• Json version schema: 1.16

Available Upgrade/Update Paths

Current version	Upgrade path
1.10.868.0 k8s/k3s 1.29	1.11.247.0 k8s/k3s 1.30
1.10.868.0 k8s/k3s 1.30	1.11.247.0 k8s/k3s 1.30
1.10.868.0 k8s/k3s 1.30	1.11.247.0 k8s/k3s 1.31

Please note that upgrading from one Kubernetes distribution to another within the same release version is not supported.

New Features

• AKS Edge Essentials version 1.11.247.0 uses Azure Linux 3.0, an upgrade from Azure Linux 2.0, used in previous releases.

• AKS Edge Essentials now connects to Azure Arc during deployment to decrease the number of steps required for customers. Arc parameters are required in the aks-edge configuration file during deployment. AKS Edge Essentials deployment will be blocked if Arc parameters are missing or invalid. Please note, this does not impact offline deployments or volume licensing customers.

• October 2025 Update: KMS provider is now generally available on AKS Edge Essentials starting with AKS Edge Essentials version 1.11.247.0.

Experimental or Preview Features

• Key manager for Kubernetes

• Run Windows containerized workloads.

• Create multi-node deployments either on a single machine or across multiple machines.

• GPU-PV is in preview

For more information feature set and benefits, see AKS Edge Essentials Overview.

Key Bug Fixes

• The previous AKS Edge Essential version had a bug that made the deployment incompatible with some non-English Windows Server OS. This has been fixed in version 1.11.247.0

Known issues

Deployment

• New deployment fails when the execution policy is set to AllSigned (Set-ExecutionPolicy –ExecutionPolicy AllSigned) . Recommend setting the policy to 'ByPass'.

• Uninstalling and reinstalling k8s/k3s causes a “deployment already exists” error or the 'Virtual machine creation failed' error, when creating a new deployment, so recommend restarting your machine before deployment .

• Event logging fails when uninstalling and reinstalling an MSI without a reboot in between . Particularly true when switching between using K3S and K8S.

• When a deployment fails due to lack of memory/disk space, it creates an incomplete VHDX file that causes errors in later deployments.

• Deployment with External virtual switch fails when using Windows RS5.

Linux nodes

• When you use the Remove-AksEdgeNode to remove a node, it might still show up in the kubectl get nodes as the cluster tries to reconcile the state of the present nodes. Please note that the command Remove-AksEdgeNode is a asynchronous call and will return before the cluster state is updated to reflect the new state.

Windows nodes

• On a K3s Windows node, load balancer klipper-lb pulls images from the internet when connected .

• Adding a Windows node takes around ~10 minutes to complete .

• No storage provisioner is available for the Windows node .

• NFS subdir external provisioner pod fails on Windows nodes.

1.10.868.0

We are excited to release the March Update of AKS Edge Essentials.

⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

Version Numbers

• AKS Edge Essentials Version: 1.10.868.0

• Azure Linux (previously Mariner) Version: 2.0.20250207

• K8s (upstream Kubernetes) distribution: 1.29.9 (only available as upgrade from 1.9.262.0), 1.30.5

• K3s distribution: 1.29.9 (only available as upgrade from 1.9.262.0), 1.30.6

• Json version schema: 1.15

New Features

• Enable secret encryption on AKS Edge Essentials clusters with the KMS plugin (preview). Following Kubernetes best practices, it's recommended that you encrypt the Kubernetes secret store. The KMS plugin can be enabled in the aks-edge config file during deployment. The KMS plugin is set to disabled by default during the preview period. You can find more information about this feature [here].

• Key manager for Kubernetes on AKS Edge Essentials (preview). Key manager for Kubernetes is an Azure Arc extension that automates the rotation of the signing key used to issue these service account tokens. You can read more about key manager [here]

• AKS Edge Essentials now connects to Azure Arc during deployment to decrease the number of steps required for customers. Arc parameters are newly required in the aks-edge configuration file during deployment. Starting with the next release, AKS Edge Essentials deployment will be blocked if Arc parameters are missing or invalid. This change does not impact offline deployments.

• The Calico CNI + K3s preview has been discontinued. Customers will need to use Flannel with K3s and Calico with K8s, as noted in the documentation.

Experimental or Preview Features

• KMS plugin

• Key manager for Kubernetes

• Run Windows containerized workloads.

• Create multi-node deployments either on a single machine or across multiple machines.

• GPU-PV is in preview

For more information feature set and benefits, see AKS Edge Essentials Overview.

Known issues

Deployment

• New deployment fails when the execution policy is set to AllSigned (Set-ExecutionPolicy –ExecutionPolicy AllSigned) . Recommend setting the policy to 'ByPass'.

• Uninstalling and reinstalling k8s/k3s causes a “deployment already exists” error or the 'Virtual machine creation failed' error, when creating a new deployment, so recommend restarting your machine before deployment .

• Event logging fails when uninstalling and reinstalling an MSI without a reboot in between . Particularly true when switching between using K3S and K8S.

• When a deployment fails due to lack of memory/disk space, it creates an incomplete VHDX file that causes errors in later deployments .

• Deployment with External virtual switch fails when using Windows RS5.

Linux nodes

• When you use the Remove-AksEdgeNode to remove a node, it might still show up in the kubectl get nodes as the cluster tries to reconcile the state of the present nodes. Please note that the command Remove-AksEdgeNode is a asynchronous call and will return before the cluster state is updated to reflect the new state.

Windows nodes

• On a K3s Windows node, load balancer klipper-lb pulls images from the internet when connected .

• Adding a Windows node takes around ~10 minutes to complete .

• No storage provisioner is available for the Windows node .

• NFS subdir external provisioner pod fails on Windows nodes.

1.9.262.0

We are excited to release the November Update of AKS Edge Essentials.

⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

Version Numbers

• AKS-EE Version: 1.9.262.0

• Mariner Version: 2.0.20241029

• K8s (upstream Kubernetes) distribution: 1.28.9, 1.29.4

• K3s distribution: 1.28.5, 1.29.6 

• Json version schema: 1.14

New Features

• We have updated the Mariner and Windows versions to include CVE fixes between this release and the previous release.

Experimental or Preview Features

• Run Windows containerized workloads.

• Create multi-node deployments either on a single machine or across multiple machines.

• GPU-PV is in preview

For more information feature set and benefits, see AKS Edge Essentials Overview.

CVE fixes

• All CVEs that have been patched between Mariner version 2.0.20240731 and Mariner version 2.0.20241029.

Known issues

Update

• When updating from 1.5.203 to 1.6.384.0 on both Linux and Windows, customers may see one of the following non-fatal errors:

  • "Get-Content: Cannot find path 'C:\Windows\System32\akswindows-manifest.json' because it does not exist."

  • "Get-Content: Cannot find path 'C:\Program Files\AksEdge\akswindows-manifest.json' because it does not exist."

This is a non-fatal error and can be ignored.

• Updates from K8S 1.26.3 to 1.26.6 fails when disk pressure is high. As a workaround, extract the tarball /opt/container-images/container-images-tar-file.tar.gz and use ctr to reimport the images that are missing, using the command ctr -n=k8s.io images import {extracted tar files location}

Deployment

• New deployment fails when the execution policy is set to AllSigned (Set-ExecutionPolicy –ExecutionPolicy AllSigned) . Recommend setting the policy to 'ByPass'.

• Uninstalling and reinstalling k8s/k3s causes a “deployment already exists” error or the 'Virtual machine creation failed' error, when creating a new deployment, so recommend restarting your machine before deployment .

• Event logging fails when uninstalling and reinstalling an MSI without a reboot in between . Particularly true when switching between using K3S and K8S.

• When a deployment fails due to lack of memory/disk space, it creates an incomplete VHDX file that causes errors in later deployments .

• Deployment with External virtual switch fails when using Windows RS5.

Linux nodes

• When you use the Remove-AksEdgeNode to remove a node, it might still show up in the kubectl get nodes as the cluster tries to reconcile the state of the present nodes. Please note that the command Remove-AksEdgeNode is a asynchronous call and will return before the cluster state is updated to reflect the new state.

Windows nodes

• On a K3s Windows node, load balancer klipper-lb pulls images from the internet when connected .

• Adding a Windows node takes around ~10 minutes to complete .

• No storage provisioner is available for the Windows node .

• NFS subdir external provisioner pod fails on Windows nodes.

1.8.202.0 Release

We are excited to release the September Update of AKS Edge Essentials.

⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

Version Numbers

• AKS-EE Version: 1.8.202.0

• Mariner Version: 2.0.20240731

• K8s (upstream Kubernetes) distribution: 1.28.9, 1.29.4

• K3s distribution: 1.28.5, 1.29.6

• Json version schema: 1.14

New Features

• Kubernetes versions supported: On K8S 1.28.9 and 1.29.4, on K3s 1.28.5 and 1.29.6

• Update from K3S 1.28.3 (July release) to 1.28.5 (September release), and on K8S from 1.28.3 (July release) to 1.28.9 (September release).

• Upgrade from K3S 1.27.6 (July release) to 1.28.5 (September release), and K3S 1.28.3 (July release) to 1.29.6 (September release)

• Upgrade from K8S from 1.27.6 (July release) to 1.28.9 (September release), and on K8S from 1.28.3 (July release) to 1.29.4 (September release).

• There is a new precheck during install to validate fdatasync latency. If disk latency is higher than 10ms, install will return a non-blocking warning message.

Experimental or Preview Features

• Run Windows containerized workloads.

• Create multi-node deployments either on a single machine or across multiple machines.

• GPU-PV is in preview

For more information feature set and benefits, see AKS Edge Essentials Overview.

Key bug fixes

• The bug where a K3s cluster with multiple network interfaces would fail to start up after a reboot, has been fixed

CVE fixes

• CVE-2021-26701

• All CVEs fixed with K3s 1.28.5 and 1.29.6

• All CVEs fixed with K8s 1.28.9 and 1.29.4

• CVE's included in the Microsoft August 2024 Patch Tuesday Update.

Known issues

Update

• When updating from 1.5.203 to 1.6.384.0 on both Linux and Windows, customers may see one of the following non-fatal errors:

  • "Get-Content: Cannot find path 'C:\Windows\System32\akswindows-manifest.json' because it does not exist."

  • "Get-Content: Cannot find path 'C:\Program Files\AksEdge\akswindows-manifest.json' because it does not exist."

This is a non-fatal error and can be ignored.

• Updates from K8S 1.26.3 to 1.26.6 fails when disk pressure is high. As a workaround, extract the tarball /opt/container-images/container-images-tar-file.tar.gz and use ctr to reimport the images that are missing, using the command ctr -n=k8s.io images import {extracted tar files location}

Deployment

• New deployment fails when the execution policy is set to AllSigned (Set-ExecutionPolicy –ExecutionPolicy AllSigned) . Recommend setting the policy to 'ByPass'.

• Uninstalling and reinstalling k8s/k3s causes a “deployment already exists” error or the 'Virtual machine creation failed' error, when creating a new deployment, so recommend restarting your machine before deployment .

• Event logging fails when uninstalling and reinstalling an MSI without a reboot in between . Particularly true when switching between using K3S and K8S.

• When a deployment fails due to lack of memory/disk space, it creates an incomplete VHDX file that causes errors in later deployments .

• Deployment with External virtual switch fails when using Windows RS5.

Linux nodes

• When you use the Remove-AksEdgeNode to remove a node, it might still show up in the kubectl get nodes as the cluster tries to reconcile the state of the present nodes. Please note that the command Remove-AksEdgeNode is a asynchronous call and will return before the cluster state is updated to reflect the new state.

Windows nodes

• On a K3s Windows node, load balancer klipper-lb pulls images from the internet when connected .

• Adding a Windows node takes around ~10 minutes to complete .

• No storage provisioner is available for the Windows node .

• NFS subdir external provisioner pod fails on Windows nodes.

1.7.639.0 Release

We are excited to release the July Update of AKS Edge Essentials.

⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

Version Numbers

• AKS-EE Version: 1.7.639.0

• Mariner Version: 2.0.20240609

• K8s (upstream Kubernetes) distribution - 1.27.6, 1.28.3

• K3s distribution – 1.27.6, 1.28.3

• Json version schema - 1.13

New Features

• Kubernetes versions supported: On K8S and K3S - 1.27.6 and 1.28.3

• Update from K3S 1.27.6 (Feb release) to 1.27.6 (July release), and on K8S from 1.27.6 (Feb release) to 1.27.6 (July release).

• Upgrade from K3S 1.26.10 (Feb release) to 1.27.6 (July release), and on K8S from 1.26.6 (Feb release) to 1.27.6 (July release).

• If you are using a ScalableCluster with the network adapter disabled/disconnected, we have added a precheck to ensure provided IP addresses for the network adapter are not in the reserved (169.254.0.0-169.254.255.255) range. If this precheck fails, you will see the error message "Please ensure the adapter is not disabled/disconnected".

Experimental or Preview Features

• Run Windows containerized workloads.

• Create multi-node deployments either on a single machine or across multiple machines.

• GPU-PV is in preview. You can find documentation here.

For more information feature set and benefits, see AKS Edge Essentials Overview.

Key bug fixes

• When removing nodes on a multi-machine cluster, manual validation that the removed node no longer served as the leader was required, as documented here. We have added a check to test this automatically.

• Get-ClusterId can be unreliable when the service is unreachable. We have added retry logic to the command to increase the success rate of this command when the service is unreachable.

Known issues

Update

• When updating from 1.5.203 to 1.6.384.0 on both Linux and Windows, customers may see one of the following non-fatal errors:

  • "Get-Content: Cannot find path 'C:\Windows\System32\akswindows-manifest.json' because it does not exist."

  • "Get-Content: Cannot find path 'C:\Program Files\AksEdge\akswindows-manifest.json' because it does not exist."

This is a non-fatal error and can be ignored.

• Updates from K8S 1.26.3 to 1.26.6 fails when disk pressure is high. As a workaround, extract the tarball /opt/container-images/container-images-tar-file.tar.gz and use ctr to reimport the images that are missing, using the command ctr -n=k8s.io images import {extracted tar files location}

Deployment

• New deployment fails when the execution policy is set to AllSigned (Set-ExecutionPolicy –ExecutionPolicy AllSigned). Recommend setting the policy to 'ByPass'.

• Uninstalling and reinstalling k8s/k3s causes a “deployment already exists” error or the 'Virtual machine creation failed' error, when creating a new deployment, so recommend restarting your machine before deployment .

• Event logging fails when uninstalling and reinstalling an MSI without a reboot in between . Particularly true when switching between using K3S and K8S.

• When a deployment fails due to lack of memory/disk space, it creates an incomplete VHDX file that causes errors in later deployments .

• Deployment with External virtual switch fails when using Windows RS5.

Linux nodes

• When you use the Remove-AksEdgeNode to remove a node, it might still show up in the kubectl get nodes as the cluster tries to reconcile the state of the present nodes. Please note that the command Remove-AksEdgeNode is a asynchronous call and will return before the cluster state is updated to reflect the new state.

Windows nodes

• On a K3s Windows node, load balancer klipper-lb pulls images from the internet when connected .

• Adding a Windows node takes around ~10 minutes to complete .

• No storage provisioner is available for the Windows node .

• NFS subdir external provisioner pod fails on Windows nodes.

1.6.384.0 Release

We are excited to release the February Update of AKS Edge Essentials.
⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

Version Numbers

• AKS-EE Version: 1.6.384.0
• Mariner Version: 2.0.20240123
• K8s (upstream Kubernetes) distribution - 1.26.6, 1.27.6
• K3s distribution – 1.26.10, 1.27.6
• Json version schema - 1.11

New Features

• Kubernetes versions supported: On K8S - 1.26.6 and 1. 27.6, On K3S 1.26.10 and 1.27.6
• Update from K3S 1.26.6 to 1.26.10, and on K8S from 1.26.6 (Nov release) to 1.26.6 (Jan release).
• Upgrade from K3s 1.25.11(Nov release) to 1.26.10(Jan release) and on K8s from 1.25.11(Nov release) to 1.26.6(Jan release)
• We have added json config validation for location field and improved error handling to create a smoother deployment process.

Experimental or Preview Features

• Run Windows containerized workloads.
• Create multi-node deployments either on a single machine or across multiple machines.
• GPU-PV is in preview

For more information feature set and benefits, see AKS Edge Essentials Overview.

Key bug fixes

• Updated journald config to ensure the journal log does not grow beyond the 250-300MB range in order to preserve log space
• Improved install and update experience by optimizing to reduce the free disk space requirement

Known issues

Update

• When updating from 1.5.203 to 1.6.384.0 on both Linux and Windows, customers may see one of the following non-fatal errors:
  • "Get-Content: Cannot find path 'C:\Windows\System32\akswindows-manifest.json' because it does not exist."
  • "Get-Content: Cannot find path 'C:\Program Files\AksEdge\akswindows-manifest.json' because it does not exist."
    This is a non-fatal error and can be ignored.
• Updates from K8S 1.26.3 to 1.26.6 fails when disk pressure is high. As a workaround, extract the tarball /opt/container-images/container-images-tar-file.tar.gz and use ctr to reimport the images that are missing, using the command ctr -n=k8s.io images import {extracted tar files location}

Deployment

• New deployment fails when the execution policy is set to AllSigned (Set-ExecutionPolicy –ExecutionPolicy AllSigned) . Recommend setting the policy to 'ByPass'.

• Uninstalling and reinstalling k8s/k3s causes a “deployment already exists” error or the 'Virtual machine creation failed' error, when creating a new deployment, so recommend restarting your machine before deployment .

• Event logging fails when uninstalling and reinstalling an MSI without a reboot in between . Particularly true when switching between using K3S and K8S.

• When a deployment fails due to lack of memory/disk space, it creates an incomplete VHDX file that causes errors in later deployments .

• Deployment with External virtual switch fails when using Windows RS5.

Linux nodes

• When you use the Remove-AksEdgeNode to remove a node, it might still show up in the kubectl get nodes as the cluster tries to reconcile the state of the present nodes. Please note that the command Remove-AksEdgeNode is a asynchronous call and will return before the cluster state is updated to reflect the new state.

Windows nodes

• On a K3s Windows node, load balancer klipper-lb pulls images from the internet when connected .
• Adding a Windows node takes around ~10 minutes to complete .
• No storage provisioner is available for the Windows node .
• NFS subdir external provisioner pod fails on Windows nodes.

1.5.203 release

We are excited to release the October Update of AKS Edge Essentials.
⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

Version Numbers

• AKS-EE Version: 1.5.203.0
• Mariner Version: 2.0.20231004
• K8s (upstream Kubernetes) distribution - 1.26.6, 1.25.11
• K3s distribution – 1.26.6, 1.25.11
• Json version schema - 1.9

New Features

• Kubernetes versions supported: On K8S - 1.25.11 and 1. 26.6, On K3S 1.25.11 and 1.26.6
• Update from K3S 1.25.8 to 1.25.11, and on K8S from 1.25.7 to 1.25.11 and from 1.26.3 to 1.26.6

Experimental or Preview Features

• Run Windows containerized workloads.
• Create multi-node deployments either on a single machine or across multiple machines.

For more information feature set and benefits, see AKS Edge Essentials Overview.

Key bug fixes

• Fixed bugs related to network switch impacting deployment experiences.
• Fixed issues with resource group requirements during Arc connectivity.
• Added checks in place, that prevents the windows node update if the machine does not have enough disk space to extract the new windows vhdx.
• Added checks in place to identify scenarios where there are multiple Azure resource types with the same name.
• WssdAgent updated to add subnet information if the switch does not have subnet information.
• Improve deployment experience, by cleaning up artifacts from a previous deployment before starting the new deployment
• Fixed the issue where single machine clusters were ignoring the DNS Server configuration parameter.

Known issues

Update

• Updates from K8S 1.26.3 to 1.26.6 fails when disk pressure is high. As a workaround, extract the tarball /opt/container-images/container-images-tar-file.tar.gz and use ctr to reimport the images that are missing, using the command ctr -n=k8s.io images import {extracted tar files location}

Deployment​

• Deploying a K8s Linux node on a 4GB RAM host machine fails.
• New deployment fails when the execution policy is set to AllSigned (Set-ExecutionPolicy –ExecutionPolicy AllSigned)​. Recommend setting the policy to 'ByPass'.
• Uninstalling and reinstalling k8s/k3s causes a “deployment already exists” error or the 'Virtual machine creation failed' error, when creating a new deployment, so recommend restarting your machine before deployment​.
• Event logging fails when uninstalling and reinstalling an MSI without a reboot in between​. Particularly true when switching between using K3S and K8S.
• When a deployment fails due to lack of memory/disk space, it creates an incomplete VHDX file that causes errors in later deployments​.
• Deployment with External virtual switch fails when using Windows RS5.

Linux nodes

• When you use the Remove-AksEdgeNode to remove a node, it might still show up in the kubectl get nodes as the cluster tries to reconcile the state of the present nodes. Please note that the command Remove-AksEdgeNode is a asynchronous call and will return before the cluster state is updated to reflect the new state.

Windows nodes​

• On a K3s Windows node, load balancer klipper-lb pulls images from the internet when connected​.
• Adding a Windows node takes around ~10 minutes to complete​.
• K3s Windows node with Calico as the CNI is not yet supported​.
• No storage provisioner is available for the Windows node​.
• NFS subdir external provisioner pod fails on Windows nodes.​

1.4.109.0 September Release

We are excited to release the September Update of AKS Edge Essentials.
⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

New Features

• Kubernetes 1.26 support. We are now supporting two Kubernetes versions, 1.25 and 1.26.
• Log Partition Size Customizability. Now the log partition size can be set between 1-10GB

Version Numbers

• AKS-EE Version: 1.4.109.0
• Mariner Version: 2.0.20230904
• K8s (upstream Kubernetes) distribution - 1.26.3, 1.25.7
• K3s distribution – 1.26.6+k3s, 1.25.8+k3s
• Json version schema - 1.9

Experimental or Preview Features

• Run Windows containerized workloads
• Create multi-node deployments either on a single machine or across multiple machines.
• Create a cluster with multiple machines

For more information feature set and benefits, see AKS Edge Essentials Overview.

Known issues

Update​

• OS upgrade (Windows 10 to Windows 11) will result in AKS Edge Essentials failing to work properly​
• "Unable to Start HCS VM" exception caught when running command "Start-AksEdgeControlPlaneUpdate -firstControlPlane True" If this error is encountered, the following commands can be used to recover the cluster
  import-module "C:\Program Files\WindowsPowerShell\Modules\1.3.186.0" -Force $nodeConfig = Get-NodeConfig -NodeType ([NodeType]::Linux) $ServiceIpRange = ParseServiceIpRange -ServiceIpRange $nodeConfig.ServiceIpRange $scriptArgs = @{ firstControlPlane = $firstControlPlane networkPlugin = $(Get-ClusterNetworkPlugin) controlPlane = $(Get-LinuxNodeHasControlPlane) controlPlaneEndpointIp = $(Get-ClusterControlPlaneEndpoint).ControlPlaneEndpointIp serviceIPRangeStart = $($ServiceIpRange.Start) serviceIPRangeEnd = $($ServiceIpRange.End) } Update-PostScriptLinuxVM @scriptArgs

Deployment​

• Deploying a K8s Linux node on a 4GB RAM host machine fails
• New deployment fails when the execution policy is set to AllSigned (Set-ExecutionPolicy –ExecutionPolicy AllSigned)​
• Uninstalling and reinstalling k8s/k3s causes a “deployment already exists” error when creating a new deployment, so a restart is necessary​
• Event logging fails when uninstalling and reinstalling an MSI without a reboot in between​
• When a deployment fails due to lack of memory/disk space, it creates an incomplete VHDX file that causes errors in later deployments​
• Deployment with External virtual switch fails when using Windows RS5

Linux nodes

• When you use the Remove-AksEdgeNode to remove a node, it might still show up in the kubectl get nodes as the cluster tries to reconcile the state of the present nodes. Please note that the command Remove-AksEdgeNode is a asynchronous call and will return before the cluster state is updated to reflect the new state.

Windows nodes​

• On a K3s Windows node, load balancer klipper-lb pulls images from the internet when connected​
• Adding a Windows node takes around ~10 minutes to complete​
• K3s Windows node with Calico as the CNI is not yet supported​
• No storage provisioner is available for the Windows node​
• NFS subdir external provisioner pod fails on Windows nodes​
• Multi-Node update Fails to recreate Windows nodes

Arc Connection

• Connecting a cluster that has Windows nodes to Arc may fail due to "logcollector*" pod getting stuck in Imageloopbackoff state.

1.3.186.0 August Release

We are excited to release the August Update of AKS Edge Essentials.
⚠️ WARNING: Note that some of the features in this release are not supported in GA yet.

New Features

• Kubernetes 1.25 support (1.24 is now deprecated)
• Kubernetes 1.24 to 1.25 upgrade
  - Existing deployments on AKS-EE 1.2.414.0 and Kubernetes minor version 1.24 can upgrade to Kubernetes minor version 1.25 by running Set-AksEdgeUpgrade -acceptUpgrade $True See documentation for more details
  • Note: Existing deployments with Windows node or Scalable Cluster must FIRST run an additional command in order to upgrade from Kubernetes version 1.24 to 1.25. For these deployments, running Set-AksEdgeUpgrade without setting this reg key will result in failure. reg add hklm\software\microsoft\aksedge /v acceptUpgrade /t REG_SZ /d True /f

Version Numbers

• K8s (upstream Kubernetes) distribution - 1.25.7
• K3s distribution - 1.25.8+k3s
• Json version schema - 1.8

Experimental or Preview Features

• Run Windows containerized workloads
• Create multi-node deployments either on a single machine or across multiple machines.
• Create a cluster with multiple machines

For more information feature set and benefits, see AKS Edge Essentials Overview.

Fixed issues

• Previously, update would fail if there was not enough available disk space. Now, update disk space has been optimized and warning has been added to prevent updates if sufficient disk space is not available.
• Fixed: When disk space begins to run out, precached images may be removed by kubelet/containerd, causing Kubernetes to fail​
• Fixed: [BUG] Windows Node status "NotReady" after restarting Host machine #127
• Fixed: IP address '192.168.254.1' is not free, While doing offline installation #131
• Various bug fixes and stability improvements

Known issues

Update​

• OS upgrade (Windows 10 to Windows 11) will result in AKS Edge Essentials failing to work properly​
• "Unable to Start HCS VM" exception caught when running command "Start-AksEdgeControlPlaneUpdate -firstControlPlane True" If this error is encountered, the following commands can be used to recover the cluster
  import-module "C:\Program Files\WindowsPowerShell\Modules\1.3.186.0" -Force $nodeConfig = Get-NodeConfig -NodeType ([NodeType]::Linux) $ServiceIpRange = ParseServiceIpRange -ServiceIpRange $nodeConfig.ServiceIpRange $scriptArgs = @{ firstControlPlane = $firstControlPlane networkPlugin = $(Get-ClusterNetworkPlugin) controlPlane = $(Get-LinuxNodeHasControlPlane) controlPlaneEndpointIp = $(Get-ClusterControlPlaneEndpoint).ControlPlaneEndpointIp serviceIPRangeStart = $($ServiceIpRange.Start) serviceIPRangeEnd = $($ServiceIpRange.End) } Update-PostScriptLinuxVM @scriptArgs

Deployment​

• Deploying a K8s Linux node on a 4GB RAM host machine fails
• New deployment fails when the execution policy is set to AllSigned (Set-ExecutionPolicy –ExecutionPolicy AllSigned)​
• Uninstalling and reinstalling k8s/k3s causes a “deployment already exists” error when creating a new deployment, so a restart is necessary​
• Event logging fails when uninstalling and reinstalling an MSI without a reboot in between​
• When a deployment fails due to lack of memory/disk space, it creates an incomplete VHDX file that causes errors in later deployments​
• Deployment with External virtual switch fails when using Windows RS5

Linux nodes

• When you use the Remove-AksEdgeNode to remove a node, it might still show up in the kubectl get nodes as the cluster tries to reconcile the state of the present nodes. Please note that the command Remove-AksEdgeNode is a asynchronous call and will return before the cluster state is updated to reflect the new state.

Windows nodes​

• On a K3s Windows node, load balancer klipper-lb pulls images from the internet when connected​
• Adding a Windows node takes around ~10 minutes to complete​
• K3s Windows node with Calico as the CNI is not yet supported​
• No storage provisioner is available for the Windows node​
• NFS subdir external provisioner pod fails on Windows nodes​
• Multi-Node update Fails to recreate Windows nodes