Skip to content

Releases: Azure/AKS

Release 2024-01-23

Choose a tag to compare

@shashankbarsin shashankbarsin released this 03 Feb 03:18
f91c376

Release 2024-01-23

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Kubernetes 1.25 was deprecated on January 14, 2024 and support transitions to platform support policy. Please upgrade to Kubernetes version 1.26 or above.
  • Starting with Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • All current AKS API versions silently ignore unknown fields. An unknown field is a field that isn't part of the AKS API. AKS API version 2024-01-01, 2024-01-02-preview and all subsequent API versions will change this behavior. Unknown fields in a request will result in the request being rejected with an error stating that the unknown field is not understood. This change only impacts new API versions and won't impact you unless you update to use an API version 2024-01-01 or later. Existing API calls (via Azure Resource Manager templates or otherwise) will continue to function as-is.

Release notes

  • Features

  • Preview features

    • Istio revision 1.19 is now available with Istio-based service mesh add-on. More information on performing canary upgrade for the new minor revision of Istio can be found here. Default revision of the Istio service mesh add-on for new clusters has been updated to 1.18. Istio 1.17 version is no longer supported.
    • Istio based service mesh addon now supports plugin CA to allow users to provide their own certificates and keys for signing workload certificates. More information can be found here.
    • When troubleshooting AKS nodes, for developers not having access to Kubernetes API but having access to node ARM API, node IP and node name information are now made available in this API. More information on accessing the nodes using the private IPs can be found here.
    • The application routing add-on can now manage multiple public and internal NGINX ingress controllers. Advanced ingress controller configuration is possible via a Custom Resource Definition (CRD).
    • AKS extension in VS Code has been updated to 1.4.1.
  • Bug Fixes

    • Fixed an issue that was previously preventing AKS Infiniband support for Standard_HB120-16rs_v3 SKU.
    • Fixed nodeAffinity in calico-node DaemonSet to prevent scheduling on virtual kubelet nodes.
    • Added appgw.ingress.azure.io api-group to ingress-appgw-cr ClusterRole to address missing api-group permissions error in Application Gateway Ingress Controller addon container.
  • Behavioral Change

    • Network observability addon updated with following:
      • increased limits for CPU (500m) and Memory (300Mi).
      • Fixed issue of networking observability agent crashing issue on Windows node pool of AKS clusters version >= 1.28.
      • Introduced a new init-kappie init container as part of kappie-agent DaemonSet.
      • api-resources nodes and namespaces added to kappie-cluster-reader ClusterRole.
    • Starting this month, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy addon will now no longer support the validation for constraint template. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI.
  • Component Updates

Release 2024-01-08

Choose a tag to compare

@charleswool charleswool released this 12 Jan 00:13
4298fb7

Release 2024-01-08

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • CIS Kubernetes V1.27 Benchmark is published which covers AKS 1.21.x through AKS 1.27.x.
  • Kubernetes 1.25 is being deprecated on January 14, 2024 and support will transition to our platform support policy. Please upgrade to Kubernetes version 1.26 or above.
  • Starting January 2024, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy Add-On will now no longer support the validation for constraint template. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI).
  • Starting with Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • Changes to reduce the kube-reserved memory reservation and eviction threshold will not be available in 1.28 as previously shared due to a release issue. These optimizations will be releasing with AKS Kubernetes minor version 1.29, which previews in January 2024. See release calendar.

Release notes

  • Preview features

  • Bug Fixes

    • PUT managedCluster operations on API versions (older than 2023-09-01) that didn't support serviceMeshProfile resulted in "invalid mode" error response to the API requests. This issue has now been fixed.
    • A wrong MCR URL for KEDA image in Air Gapped Cloud was previously used resulting in potential failures in enabling the KEDA addon. This issue has now been fixed.
  • Behavioral Change

    • Starting with the 2024-01-01 and 2024-01-02-preview APIs, we will begin to reject unknown fields in the request payloads. See #4060 for more details.
    • The memory limit for Azure Key Vault provider for Secrets Store CSI Driver is now increased from 200 Mi to 300Mi.
    • Expander flag is removed from AutoscalerProfile from 2023-11-01-preview API since it may cause confusion with existing Expander.
  • Component Updates

Release 2023-11-28

Choose a tag to compare

@kevinkrp93 kevinkrp93 released this 09 Dec 21:15
d18937f

Release 2023-11-28

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Kubernetes 1.25 is being deprecated on January 14, 2024 and support will transition to our platform support policy. Please upgrade to Kubernetes version 1.26 or above.
  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from December 1st, 2023. We recommend updating your cluster with AKS-managed Azure AD before December 1st, 2023. This way you can manage the API server downtime during non-business hours.
  • Starting January 2024, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy Add-On will now no longer support this. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI.
  • Starting Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • Staring with the 2024-01-01 and 2024-01-02-preview APIs, we will begin to reject unknown fields in the request payloads.
  • Changes to reduce the kube-reserved memory reservation and eviction threshold will not be available in 1.28 as previously shared due to a release issue. These optimizations will be releasing with AKS Kubernetes minor version 1.29, which previews in January 2024.

Release notes

Release 2023-11-05

Choose a tag to compare

@bmoore-msft bmoore-msft released this 13 Nov 17:38
067d290

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Kubernetes 1.25 is being deprecated on January 14, 2024 and support will transition to our platform support policy. Please upgrade to Kubernetes version 1.26 or above.
  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from December 1st, 2023. We recommend updating your cluster with AKS-managed Azure AD before December 1st, 2023. This way you can manage the API server downtime during non-business hours.
  • Starting January 2024, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy Add-On will now no longer support this. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI.
  • Windows containerd v1.7 will be the default container runtime for k8s v1.28+ on AKS Windows nodes. Windows Host Process (HPC) containers is GA in Windows containerd v1.7 and it has some breaking changes.
  • Starting Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.

Release notes

  • Features
    • Kubernetes 1.28 is GA
    • Added kubernetes patch versions 1.25.15, 1.26.10, 1.27.7
    • KEDA addon is GA
  • Preview Features
    • Cluster network settings can be updated to enable Kubenet -> CNI Overlay migration - available in the CLI
  • Bug Fixes
    • Incorporated fix for irqbalance #275 a node image upgrade from 202310.19.2 will resolve the unbalanced IRQs
    • Under some conditions it was possible to set max_surge=0 which may interfere with upgrades. Now max_surge must be > 0. See Customize node surge upgrade for more information about the setting.
    • Fixed an issue where PUT operations on managedClusters or agentPools see long latency in the overall operation due to an internal network issue.
    • PATCH operations were allowed on managedClusters in a non-terminal provisioningState. This could cause an eTag mismatch and inconsistent results or failures. PATCH operations will now be block for managedClusters in a non-terminal provisioningState.
  • Behavioral Change
    • Changes to reduce the kube-reserved memory reservation and eviction threshold will not be available in 1.28 as previously shared due to a release issue. These optimizations will be releasing with AKS 1.29, which previews in January 2024.
  • Component Updates
    • Update the aks-app-routing-operator to version 0.0.7 which includes notable changes in version 0.0.6.
      • This update has 3 CVE fixes for the nginx ingress controller.
      • The following changes are also included:
        • The AJP protocol is no longer supported.
        • The whitelist-source-range annotation has been renamed to allowlist-source-range. Both are currently supported but it is recommended to move to the new annotation allowlist-source-range.
      • The custom-http-errors annotation now only supports errors between 400 and 599.
    • Azure Monitor Metrics November release to v.6.8.1
    • Update gatekeeper to v3.13.3 and policy addon 1.2.1
      • Azure Policy Changes
        • Introduce warn for policies, available in select upcoming built-in policy experiences
        • Show an exempt ComplianceReasonCode in the portal for exempt policies.
    • Update Azure Disk CSI driver version to v1.29.1 on AKS 1.28, to v1.28.4 on AKS 1.27, to v1.26.7 on AKS 1.26 and 1.25
    • Update Azure File CSI driver version to v1.29.1 on AKS 1.28, to v1.28.6 on AKS 1.27, to v1.26.9 on AKS 1.26 and 1.25
    • Update Azure Blob CSI driver version to v1.23.1 on AKS 1.28, to v1.22.3 on AKS 1.27, to v1.21.5 on AKS 1.26 and 1.25
    • Update cloud-controller-manager image to v1.27.11, v1.26.17, v1.25.22 (release notes)
    • Update to dropgz v0.0.15 to include azure-ipam v0.0.6
    • Azure Linux image has been updated to Azure Linux - 202311.07.0.
    • AKS Ubuntu 22.04 image has been updated to AKSUbuntu-2204-202311.07.0.

Release 2023-10-29

Choose a tag to compare

@kaysieyu kaysieyu released this 03 Nov 20:05
f869ed2

Release 2023-10-29

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Kubernetes 1.25 is being deprecated at the end of January 2024 and support will transition to our platform support policy.
  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from December 1st, 2023. We recommend updating your cluster with AKS-managed Azure AD before December 1st, 2023. This way you can manage the API server downtime during non-business hours.
  • Starting January 2024, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy Add-On will now no longer support this. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI).
  • Windows containerd v1.7 will be the default container runtime for k8s v1.28+ on AKS Windows nodes. Windows Host Process (HPC) containers is GA in Windows containerd v1.7 and it has some breaking changes.
  • Starting Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • AKS sent out an advisory regarding CVE-2023-29332 on September 13, 2023, which impacts AKS agent nodes. Recommended mitigation is to upgrade AKS cluster and AKS node image. If impacted clusters are not upgraded, AKS will apply mitigation on customer's next cluster update operation including node OS updates and node rolling upgrades, which may cause workload disruption.

Release notes

  • Preview Features
  • Bug Fixes
    • Corrected issue where on tainted/dedicated system pools the Vertical Pod Autoscaler (VPA) deployment could end up on non-system pools.
    • Fix for issue where a Certificate Authority bundle mismatch could produce an update on the image version of the VPA webhook.
    • Fix for possible deadlock scenario between Container Network Service and Azure CNI where pod IPs would not release on pod delete and new pods would not get an IP.
    • Fix for Windows NPM crashes in k8s 1.28 with Containerd 1.7. Bug was a result of Windows NPM DaemonSet referencing a file that did not exist in its current directory.Containerd 1.7.
    • Fix for fleet clusters, so they will now be correctly set to NRG-Lockdown RestrictionLevel Restricted, instead of Unspecified. Additionally, fleet clusters within one of the undesired Unspecified states will be fixed on reconcile.
    • Fix to prevent conflict between Open Service Mesh and AKS Admission Enforcer.
    • Fix to improve response time and reduce long mc and agentpool operation latency.
  • Behavioral Change
  • Component Updates
    • Microsoft Defender for Cloud publisher image has been updated to 1.0.68 (now distroless)
    • Microsoft Defender for Cloud OldFileCleaner image has been updated to 1.4.68
    • Azure Linux image has been updated to Azure Linux - 202310.26.0.
    • AKS Ubuntu 22.04 image has been updated to AKSUbuntu-2204-202310.26.0.

Release 2023-10-22

Choose a tag to compare

@abubinski abubinski released this 30 Oct 18:30
5369fcb

Release 2023-10-22

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Kubernetes 1.25 is being deprecated at the end of January 2024 and support will transition to our platform support policy.
  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from December 1st, 2023. We recommend updating your cluster with AKS-managed Azure AD before December 1st, 2023. This way you can manage the API server downtime during non-business hours.

Release notes

  • Bug Fixes

    • Fix for some abnormal slow put managedClusters/agentPool operations caused by hanging connections.
    • Fix for some throttling issue by increasing secrets store AKV provider cpu limit from 50m to 100m.
    • Fix for CVE by upgrading Azure file driver version to v1.24.11 on AKS 1.25.
    • Fix for Azure CNI Overlay when using Linux Kernel 6.2+ and K8s 1.28+. This fix prevents the CNI from setting up pod networking incorrectly.
  • Behavioral Change

    • Introduced acn-multitenancy-editor ClusterRole to give azure-cns permissions on "multitenantpodnetworkconfigs", "podnetworkinstances", and "podnetworks" resources.
  • Component Updates

Release 2023-10-15

Choose a tag to compare

@kevinkrp93 kevinkrp93 released this 20 Oct 21:24
0048d7c

Release 2023-10-15

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from 1st Dec. 2023. We recommend updating your cluster with AKS-managed Azure AD before 1 Dec 2023. This way you can manage the API server downtime during non-business hours.
  • CVE-2023-29332 Microsoft Azure Kubernetes Service Elevation of Privilege Vulnerability. An attacker who successfully exploited this vulnerability could gain Cluster Administrator privileges. Please update your AKS VHD to at least VHD version 230801 as mentioned in the issue
  • CVE-2023-44487 - The HTTP/2 protocol allows a denial of service (server resource consumption) because request cancellation can reset many streams quickly

Release notes

Release 2023-10-08

Choose a tag to compare

@charleswool charleswool released this 18 Oct 01:56
7e11c65

Release 2023-10-08

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from 1st Dec. 2023. We recommend updating your cluster with AKS-managed Azure AD before 1 Dec 2023. This way you can manage the API server downtime during non-business hours.

Release notes

  • Features

  • Bug Fixes

    • Microsoft Defender for Containers has been updated to image version 1.3.81 to support kernel versions 6.2 or higher.
  • Behavioral Changes

    • With the release of Container Insights 3.1.14, default 1-year tokens will be set to 1-hour expiry and refreshed at 10 minutes.
    • A warning has been added for clusters utilizing dual-stack networking and outbound type user-defined routing if the associated route table does not have a default IPv6 route in place. Visit Dual-stack kubenet networking for full details.
    • Customers can now disable Windows GMSA on an existing cluster.
    • Node OS Auto Upgrade now has a built-in Policy Definition that can be used to validate and enforce whether it is enabled on an AKS cluster.
  • Component Updates

    • Windows CNI has been updated to v1.4.39.1 for Azure CNI Overlay and Azure CNI with dynamic allocation.
    • Azure Monitor Metrics for AKS has been updated to image version 6.7.7. Please see their release notes for full details.
    • The AKS vscode extension v1.3.15 has been released

Release 2023-10-01

Choose a tag to compare

@stl327 stl327 released this 06 Oct 17:49
80f2064

Release 2023-10-01

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from 1st Dec. 2023. We recommend updating your cluster with AKS-managed Azure AD before 1 Dec 2023. This way you can manage the API server downtime during non-business hours.

Release notes

  • Features

  • Bug Fixes

    • Fix for some events during an upgrade such as "Deleting node" not appearing in kubectl get events.
    • Fix for metricDefinitions operation not exposed in Azure China.
    • Fix for Cluster Autoscaler condition where nodes that VPA pods are scheduled to could not be evicted.
  • Behavioral Changes

    • The pod CPU request from ama-metrics daemonsets will be reduced in Windows from 500m to 150m and in Linux from 75m to 50m.
    • AKS will now validate, and block if necessary, service CIDRs placed in public and multicast IP address ranges.
    • If the ama-logs add-on is enabled, host port 28330 will be mounted to the ama-logs daemonset in order to facilitate syslog collection.
    • To reduce vertical pod autoscaling (VPA) out of memory (OOM) errors, the vpa-recommender CPU limit will increase to 1000m, memory limit to 2000Mi, and memory request to 800Mi from 200m, 1000m, and 500Mi respectively.
    • The default max surge value during upgrades will be changed from 1 to 10% for AKS 1.28+ on new clusters to improve upgrade latency.
  • Component Updates

    • Linux Network Policy Manager (NPM) version has been rebuilt to v1.4.45.2, containing patches for Ubuntu CVEs.
    • ip-masq-agent-v2 onboarded to semantic versioning and has been updated to v0.1.8.
    • Upgraded Azure File CSI driver to v1.24.10 on AKS 1.25, v1.26.8 on AKS 1.26, and v1.28.5 on AKS 1.27.
    • Blob CSI driver upgraded to v1.22.2 on AKS 1.27+ to support AZNFS mount helper.

Release 2023-09-24

Choose a tag to compare

@qpetraroia qpetraroia released this 30 Sep 00:14
95ebd13

Azure Kubernetes Service Changelog

Release 2023-09-24

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from 1st Dec. 2023. We recommend updating your cluster with AKS-managed Azure AD before 1 Dec 2023. This way you can manage the API server downtime during non-business hours.

Release notes

  • Behavioral changes

  • Preview Features

    • Image Integrity allows you to sign container images via a process that ensures their authenticity and integrity.
  • Bug Fixes

    • Fix for the Private Link Service (PLS) creation failure that can occur if the customer selects a subnet name or PLS name that is too long.
  • Component Updates

    • Microsoft Defender Publisher container (part of defender for containers solution) image version has been updated to 1.0.67 from 1.0.64 which improves memory utilizaiton to reduce pod restarts due to OOMKills
    • Cilium version has been updated to 1.13.5 for AKS clusters with kubernetes versions 1.28 or greater
    • Azure File CSI driver updated to version v1.24.9 for clusters with kubernetes version 1.25, v1.26.7 for clusters with kubernetes version 1.26 and v.1.28.4 for clusters with kubernetes version 1.27
    • Hotfix: There were 3 CVE's in the upstream Kubernetes related to insufficient input sanitiztion which leads to privilege escalation. AKS Patched the AKS cluster nodes for clusters version 1.24.9, 1.24.10, 1.24.15, 1.25.5, 1.25.6, 1.25.11, 1.26.0, 1.26.3, 1.26.6, 1.27.3. CVE links - CVE-2023-3676, CVE-2023-3955, and CVE-2023-3893. Update your AKS cluster's node images if the cluster does not have node OS auto-upgrade feature enabled.