Kubernetes v1.26.0 is live!

2022年12月10号，v1.26.0 使用 Golang 版本 1.19.4 构建并推送。

变化

以下是自Kubernetes(k8s) v1.25.0 以来的变更日志。

紧急升级说明

(真的，你必须在升级前阅读这个)
在v1.26版本中删除的废弃测试版API不再提供。更多信息见 https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-26

种类变更

弃用

• CLI pod-eviction-timeout已被废弃，并将在v1.27版本中与enable-ta-manager一起被删除。
• Kube-apiserver：未使用的 --master-service-namespace 已被废弃，将在 v1.27 中移除。
• gcp和azure auth插件已经从client-go和kubectl中移除。关于特定云的替换，请参见GKE中的kubelogin和Kubectl Auth变化。

API变化

- “调度器的组件配置v1beta2/v1beta3/v1添加了一个新的preEnqueue扩展点。”
- “添加了一个ResourceClaim API（位于resource.k8s.io/v1alpha1 API组中，位于DynamicResourceAllocation功能网关后）。新的API比Kubernetes现有的Device Plugins功能更灵活，因为它允许Pod请求（声明）特殊类型的资源，这些资源可以在节点级别、集群级别或按照您实现的任何其他模型中可用。”
- “使用httpGet的容器preStop和postStart生命周期处理程序现在遵循指定的scheme和headers字段。这使得设置自定义headers和将scheme更改为HTTPS，与容器启动/可用性/活性探测功能一致。使用scheme：HTTPS配置的生命周期处理程序遇到指示端点实际上使用HTTP的错误时，会回退到通过HTTP进行请求，以兼容先前的版本。当发生这种情况时，在Pod的名称空间中记录一个LifecycleHTTPFallback事件，并在kubelet中增加一个kubelet_lifecycle_handler_http_fallbacks_total指标。集群管理员可以通过在kubelet中设置--feature-gates=ConsistentHTTPGetHandlers=false来选择退出扩展的生命周期处理程序功能。”
- “Graduated JobTrackingWithFinalizers为稳定版。在启用该功能之前创建的job仍将被跟踪，但不带有finalizers。使用finalizers跟踪的job注解batch.kubernetes.io/job-tracking。如果存在该注解且用户尝试删除它，控制平面会将其添加回来。注解batch.kubernetes.io/job-tracking现在已弃用。控制平面将忽略它，并在v1.27中停止为新job添加它。”
- “Kubelet添加了以下Pod失败条件：
 - DisruptionTarget（优雅的节点关闭，节点压力驱逐）。”
- “优先级和公平性引入了一项新功能，称为借贷，它允许API优先级级别从其他优先级级别中借用一定数量的座位。作为集群操作员，您可以通过指定优先级级别配置对象下的两个新字段lendablePercent和borrowingLimitPercent来启用某个优先级级别的借用。该更改添加了以下指标：”
 - apiserver_flowcontrol_nominal_limit_seats: Nominal number of execution seats configured for each priority level
 - apiserver_flowcontrol_lower_limit_seats: Configured lower bound on number of execution seats available to each priority level
 - apiserver_flowcontrol_upper_limit_seats: Configured upper bound on number of execution seats available to each priority level
 - apiserver_flowcontrol_demand_seats: Observations, at the end of every nanosecond, of (the number of seats each priority level could use) / (nominal number of seats for that level)
 - apiserver_flowcontrol_demand_seats_high_watermark: High watermark, over last adjustment period, of demand_seats
 - apiserver_flowcontrol_demand_seats_average: Time-weighted average, over last adjustment period, of demand_seats
 - apiserver_flowcontrol_demand_seats_stdev: Time-weighted standard deviation, over last adjustment period, of demand_seats
 - apiserver_flowcontrol_demand_seats_smoothed: Smoothed seat demands
 - apiserver_flowcontrol_target_seats: Seat allocation targets
 - apiserver_flowcontrol_seat_fair_frac: Fair fraction of server's concurrency to allocate to each priority level that can use it
 - apiserver_flowcontrol_current_limit_seats: current derived number of execution seats available to each priority level The possibility of borrowing means that the old metric apiserver_flowcontrol_request_concurrency_limit can no longer mean both the configured concurrency limit and the enforced concurrency limit. Henceforth it means the configured concurrency limit.'
- 'NodeInclusionPolicy in podTopologySpread plugin is now enabled by default.'
- 'PodDisruptionBudget now adds an alpha spec.unhealthyPodEvictionPolicy field. When the PDBUnhealthyPodEvictionPolicy feature-gate is enabled in kube-apiserver, setting this field to "AlwaysAllow" allows pods to be evicted if they do not have a ready condition, regardless of whether the PodDisruptionBudget is currently healthy.'
- 'metav1.LabelSelectors specified in API objects are now validated to ensure they do not contain invalid label values that will error at time of use. Existing invalid objects can be updated, but new objects are required to contain valid label selectors.'
- Add percentageOfNodesToScore as a scheduler profile level parameter to API version v1. When a profile percentageOfNodesToScore is set, it will override global percentageOfNodesToScore.
- Add auth API to get self subject attributes (new selfsubjectreviews API is added). The corresponding command for kubctl is provided - kubectl auth whoami.
- Added kubernetes_feature_enabled metric series to track whether each active feature gate is enabled.
- Added a --topology-manager-policy-options flag to the kubelet to support fine tuning the topology manager policies. The first policy option, prefer-closest-numa-nodes, allows these policies to favor sets of NUMA nodes with shorter distance between nodes when making admission decisions.
- Added a feature that allows a StatefulSet to start numbering replicas from an arbitrary non-negative ordinal, using the .spec.ordinals.start field.
- Added a kube-proxy flag (--iptables-localhost-nodeports, default true) to allow disabling NodePort services on loopback addresses. Note: this only applies to iptables mode and ipv4.
- Added a new namespace alpha field to DataSourceRef field in PersistentVolumeClaim API.
- Aggregated discovery will be alpha and can be toggled with the AggregatedDiscoveryEndpoint feature flag.
- Clarified the CFS quota as 100ms in the code comments and set the minimum cpuCFSQuotaPeriod to 1ms to match Linux kernel expectations.
- Component-base: make the validation logic about LeaderElectionConfiguration consistent between component-base and client-go
- Deprecated the apiserver_request_slo_duration_seconds metric for v1.27 in favor of apiserver_request_sli_duration_seconds for naming consistency purposes with other SLI-specific metrics and to avoid any confusion between SLOs and SLIs.
- Enable the "Retriable and non-retriable pod failures for jobs" feature into beta.
- Enabled kube-controller-manager to support '--concurrent-horizontal-pod-autoscaler-syncs' flag to set the number of horizontal pod autoscaler controller workers.
- Fixed spurious field is immutable errors validating updates to Event API objects via the events.k8s.io/v1 API.
- Graduated ServiceInternalTrafficPolicy feature to GA.
- In `kube-proxy`: The "userspace" proxy mode (deprecated for over a year) is no longer supported on either Linux or Windows. Users should use "iptables" or "ipvs" on Linux, or "kernelspace" on Windows.
- Introduce v1beta3 for Priority and Fairness with the following changes to the API spec:
- rename 'assuredConcurrencyShares' (located under `spec.limited`) to 'nominalConcurrencyShares'.
- apply strategic merge patch annotations to 'Conditions' of flowschemas and prioritylevelconfigurations.
- Introduced v1alpha1 API for validating admission policies, enabling extensible admission control via CEL expressions (KEP 3488: CEL for Admission Control). To use, enable the ValidatingAdmissionPolicy feature gate and the admissionregistration.k8s.io/v1alpha1 API via --runtime-config.
- KMS: added validation for duplicate kms config name when auto reload is enabled. If you enabled automatic reload of encryption configuration with API server flag --encryption-provider-config-automatic-reload, ensure all the KMS provider names (v1 and v2) in the encryption configuration are unique.
- Kubelet external Credential Provider feature is moved to GA. Credential Provider Plugin and Credential Provider Config APIs updated from v1beta1 to v1 with no API changes.
- Legacy klog flags are no longer available. Only -v and -vmodule are still supported.
- Moved MixedProtocolLBService from beta to GA.
- New Pod API field .spec.schedulingGates is introduced to enable users to control when to mark a Pod as scheduling ready.
- Protobuf serialization of metav1.MicroTime timestamps (used in Lease and Event API objects) has been corrected to truncate to microsecond precision, to match the documented behavior and JSON/YAML serialization. Any existing persisted data is truncated to microsecond when read from etcd.
- Removed feature gates ServiceLoadBalancerClass and ServiceLBNodePortControl. These feature gates were enabled (and locked) since v1.24.
- Reverted regression that prevented client-go latency metrics to be reported with a template URL to avoid label cardinality.
- The EndpointSliceTerminatingCondition feature gate was graduated to GA. The gate is now locked and will be removed in v1.28.
- DynamicKubeletConfig feature gate has been removed from the API server. Dynamic kubelet reconfiguration now can't be used even when older nodes are still attempting to rely on it. This is aligned with the Kubernetes version skew policy.
- kubectl wait command with jsonpath flag will wait for target path until timeout.

功能

'Added selector validation to HorizontalPodAutoscaler: when multiple HPAs select the same set of Pods, scaling now will be disabled for those HPAs with the reason AmbiguousSelector. This change also covers a case when multiple HPAs point to the same deployment.'
'Pod Security admission: the pod-security warn level will now default to the enforce level.'
'Promoted the APIServerIdentity feature to Beta. By default, each kube-apiserver will now create a Lease in the kube-system namespace. These lease objects can be used to identify the number of active API servers in the cluster, and may also be used for future features such as the Storage Version API.'
'The iptables kube-proxy backend now process service/endpoint changes more efficiently in very large clusters.'
'CSIMigrationvSphere was upgraded to GA and locked to true. Do not upgrade to K8s 1.26 if you need Windows, or XFS, or raw block support until vSphere CSI Driver adds support for them in a version post v2.7.x.'
'DelegateFSGroupToCSIDriver feature is GA.'
'NodeOutOfServiceVolumeDetach is now beta.'
'RetroactiveDefaultStorageClass feature is now beta.'
'registered_metric_total will now report the number of metrics broken down by stability level and deprecated version.'
A new DisableCompression field (default = false) has been added to kubeconfig under cluster info. When set to true, clients using the kubeconfig opt out of response compression for all requests to the apiserver. This can help improve list call latencies significantly when client-server network bandwidth is ample (>30MB/s) or if the server is CPU-constrained.
A new pod_status_sync_duration_seconds histogram is reported at alpha metrics stability that estimates how long the Kubelet takes to write a pod status change once it is detected.
API Server Tracing now includes a variety of new spans and span events.
API Server tracing now includes the latency of authorization, priorityandfairness, impersonation, audit, and authentication filters.
API Server tracing root span name for opentelemetry is changed from KubernetesAPI to HTTP GET.
Added --disable-compression flag to kubectl (default = false). When true, it opts out of response compression for all requests to the apiserver. This can help improve list call latencies significantly when client-server network bandwidth is ample (>30MB/s) or if the server is CPU-constrained.
Added a method StreamWithContext to remotecommand.Executor to support cancelable SPDY executor stream.
Added a new feature gate CelValidatingAdmissionExtensibility to enable expression validation for Admission Control.
Added alpha support for WindowsHostNetworking feature.
Added alpha support for returning container and pod metrics from CRI, instead of cAdvsior.
Added categories column to the kubectl api-resources command's wide output (-o wide). Added --categories flag to the kubectl api-resources command, which can be used to filter the output to show only resources belonging to one or more categories.
Added kubelet metrics to track the cpumanager cpu allocation and pinning
Added new Golang runtime-related metrics to Kubernetes components:
- go_gc_cycles_automatic_gc_cycles_total
- go_gc_cycles_forced_gc_cycles_total
- go_gc_cycles_total_gc_cycles_total
- go_gc_heap_allocs_by_size_bytes
- go_gc_heap_allocs_bytes_total
- go_gc_heap_allocs_objects_total
- go_gc_heap_frees_by_size_bytes
- go_gc_heap_frees_bytes_total
- go_gc_heap_frees_objects_total
- go_gc_heap_goal_bytes
- go_gc_heap_objects_objects
- go_gc_heap_tiny_allocs_objects_total
- go_gc_pauses_seconds
- go_memory_classes_heap_free_bytes
- go_memory_classes_heap_objects_bytes
- go_memory_classes_heap_released_bytes
- go_memory_classes_heap_stacks_bytes
- go_memory_classes_heap_unused_bytes
- go_memory_classes_metadata_mcache_free_bytes
- go_memory_classes_metadata_mcache_inuse_bytes
- go_memory_classes_metadata_mspan_free_bytes
- go_memory_classes_metadata_mspan_inuse_bytes
- go_memory_classes_metadata_other_bytes
- go_memory_classes_os_stacks_bytes
- go_memory_classes_other_bytes
- go_memory_classes_profiling_buckets_bytes
- go_memory_classes_total_bytes
- go_sched_goroutines_goroutines
- go_sched_latencies_seconds
Added new metric job_controller_terminated_pods_tracking_finalizer which can be used to monitor whether the job controller is removing Pod finalizers from terminated Pods after accounting them in Job status.
Added publishing events when enabling/disabling TopologyAwareHints.
Added reconstruction of SELinux mount context after kubelet restart. Feature SELinuxMountReadWriteOncePod is now fully implemented and kubelet does not lose its cache of SELinux contexts after kubelet process restart.
Added support for Evented PLEG feature gate.
Added the metric pod_start_sli_duration_seconds to kubelet.
Added validation for the --container-runtime-endpoint flag of kubelet to be non-empty.
Adds alpha --output plaintext protected by environment variable KUBECTL_EXPLAIN_OPENAPIV3
Adds metrics force_delete_pods_total and force_delete_pod_errors_total in the Pod GC Controller.
Azure File CSI migration is now GA.
Changed preemption_victims metric bucket from LinearBuckets to ExponentialBuckets.
Exposed health check SLI metrics on metrics/slis for apiserver.
Extend the job job_finished_total metric by new reason label and introduce a new job metric to count pod failures handled by pod failure policy with respect to the action applied.
Graduate ServiceIPStaticSubrange feature to GA.
Graduated Kubelet CPU Manager to GA.
Graduated Kubelet Device Manager to GA.
If ComponentSLIs feature gate is enabled, then /metrics/slis becomes available on kubelet, allowing you to scrape health check metrics.
If ComponentSLIs feature gate is enabled, then /metrics/slis now becomes available on cloud-controller-manager allowing you to scrape health check metrics.
If more than one StorageClass is designated as default (via the "storageclass.kubernetes.io/is-default-class" annotation), choose the newest one instead of throwing an error.'
In `client-go SharedInformerFactory` will now support waiting for goroutines during shutdown.
In kubeadm, command kubeadm join phase control-plane-prepare certs now supports to run with dry-run mode on it's own.
Kube-apiserver: gzip compression switched from level 4 to level 1 to improve large list call latencies in exchange for higher network bandwidth usage (10-50% higher). This increases the headroom before very large unpaged list calls exceed request timeout limits.
Kubeadm: added show-join-command as a new separate phase at the end of kubeadm init. You can skip printing the join information by using kubeadm init --skip-phases=show-join-command. Executing only this phase on demand will throw an error because the phase needs dependencies such as bootstrap tokens to be pre-populated.
Kubeadm: added the "--cleanup-tmp-dir" flag for kubeadm reset. It will cleanup the contents of /etc/kubernetes/tmp. The flag is off by default.
Kubeadm: now supports image repository format validation.
Kubeadm: sub-phases are now able to support the dry-run mode, e.g. kubeadm reset phase cleanup-node --dry-run
Kubeadm: tried to load CA cert from external CertificateAuthority file when CertificateAuthorityData is empty for existing kubeconfig.
Kubectl shell completions for the bash shell now include descriptions.
Kubernetes is now built with Go 1.19.1
Kubernetes is now built with Go 1.19.2
Kubernetes is now built with Go 1.19.3.
Logs of requests that were timed out by a timeout handler no longer contain a statusStack and logging error output fields.
Metrics for RetroactiveDefaultStorageClass feature are now available. To see an attempt count for updating PVC retroactively with a default StorageClass see retroactive_storageclass_total metric and for total numer of errors see retroactive_storageclass_errors_total.
Promoted kubectl alpha events to kubectl events.
Promoting WindowsHostProcessContainers to stable.
Scheduler now retries updating a pod's status on ServiceUnavailable and InternalError errors, in addition to net.ConnectionRefused error.
Shell completion now shows plugin names when appropriate. Furthermore, shell completion will work for plugins that provide such support.
Switched kubectl to use github.com/russross/blackfriday/v2
The ExpandedDNSConfig feature has graduated to beta and is enabled by default. Note that this feature requires container runtime support.
The LegacyServiceAccountTokenNoAutoGeneration feature gate was promoted to GA.
The ProxyTerminatingEndpoints feature is now Beta and enabled by default. When enabled, kube-proxy will attempt to route traffic to terminating pods when the traffic policy is Local and there are only terminating pods remaining on a node.
The goroutines metric is newly added in the scheduler. It replaces scheduler_goroutines metric and it counts the number of goroutine in more places than scheduler_goroutine does.
Updated cAdvisor to v0.46.0.
Updated the Lease identity naming format for the APIServerIdentity feature to use a persistent name.
When ComponentSLIs feature gate is enabled, /metrics/slis becomes available on kube-scheduler, allowing you to scrape health check metrics.
When ComponentSLIs feature gate is enabled, then /metrics/slis becomes available on kube-proxy allowing you to scrape health check metrics.
When ComponentSLIs feature gate is enabled, then /metrics/slis becomes available on kube-controller-manager, allowing you to scrape health check metrics.
When the alpha LegacyServiceAccountTokenTracking feature gate is enabled, secret-based service account tokens will have a kubernetes.io/legacy-token-last-used applied to them containing the date they were last used.
CSRDuration feature gate that graduated to GA in 1.24 and is unconditionally enabled now removed in v1.26.
kubectl config view now automatically redacts any secret fields marked with a datapolicy tag.

Documentation

• Clarified the default CFS quota period as being 100µs and not 100ms.

Bug 或 回归

Added back unused flags on kubectl run command, which did not go through the required deprecation period before being removed.
Added support for RSA and ECDSA format keys in preflight check on kubeadm.
Allowed Label section in vSphere e2e cloud provider configuration.
Apiserver /healthz/etcd endpoint rate limits the number of forwarded health check requests to the etcd backends, answering with the last known state if the rate limit is exceeded. The rate limit is based on 1/2 of the timeout configured, with no burst allowed.
Apiserver: used the correct error when logging errors updating managedFields.
Avoided propagating hosts search . into containers in /etc/resolv.conf.
Bump golang.org/x/net to v0.1.1-0.20221027164007-c63010009c80.
Bump runc to v1.1.4.
Callers using DelegatingAuthenticationOptions can now use DisableAnonymous to disable Anonymous authentication.
Changed error message when resource is not supported by given patch type in kubectl patch.
Correct the calculating error in podTopologySpread plugin to avoid unexpected scheduling results.
Etcd: Updated to v3.5.5.
Fixed Admission controllers that caused unnecessary significant load on `apiserver`.
Fixed DaemonSet to update the status even if it fails to create a pod.
Fixed a bug where a change in the appProtocol for a Service did not trigger a load balancer update.
Fixed a bug where the kubelet choose the wrong container by its name when running kubectl exec.
Fixed an ephemeral port exhaustion bug caused by improper connection management that occurred when a large number of objects were handled by kubectl while exec auth was in use.
Fixed an issue in winkernel proxier that causes proxy rules to leak anytime service backends are modified.
Fixed bug in kubectl rollout history where only the latest revision was displayed when a specific revision was requested and an output format was specified.
Fixed bug where dry run message was not printed when running kubectl label with --dry-run flag.
Fixed code to ensure that pods running on nodes tainted with NoExecute continue to run when the PodDisruptionConditions feature gate is enabled.
Fixed cost estimation of token creation request for service account in Priority and Fairness.
Fixed issue where the APIServer would panic on startup if an egress selector without a controlplane configuration is specified when using APIServerTracing.
Fixed list cost estimation in Priority and Fairness for list requests with metadata.name specified.
Fixed race condition in GCE between containerized mounter setup in the kubelet.
Fixed relative CPU priority for pods where containers explicitly request zero cpu by giving the lowest priority instead of falling back to the cpu limit to avoid possible cpu starvation of other pods.
Fixed that disruption controller used to change the status of a stale disruption condition after 2 min when the PodDisruptionConditions feature gate is enabled.
Fixed the PodAndContainerStatsFromCRI feature, instead of supplementing with stats from cAdvisor.
Fixed the occasional double-counting of the job_finished_total metric.
For kubectl, --server-side now migrates ownership of all fields used by client-side-apply to the specified --fieldmanager. This prevents fields previously specified using kubectl from being able to live outside of server-side-apply's management and become undeleteable.
For raw block CSI volumes on Kubernetes, kubelet was incorrectly calling CSI NodeStageVolume for every single "map" (i.e. raw block "mount") operation for a volume already attached to the node. This change modified that behavior to ensure it is only called once per volume per node.
Improved kubectl display of invalid request errors returned by the API server.
In `kube-apiserver,x-kubernetes-list-type` validation is now enforced when updating status of custom resources.'
In kube-apiserver, custom resources can now be specified in the --encryption-provider-config file and can be encrypted in etcd.
Increased the maximum backoff delay of the endpointslice controller to match the expected sequence of delays when syncing Services.
Known issue: Job field .spec.podFailurePolicy.rules[*].onExitCode might be ignored if the Pod is deleted before it terminates.
Kube-apiserver: DELETECOLLECTION API requests are now recorded in metrics with the correct verb.
Kube-apiserver: redirect responses are no longer returned from backends by default. Set --aggregator-reject-forwarding-redirect=false to continue forwarding redirect responses.
Kube-apiserver: redirects from backend API servers are no longer followed when checking availability with requests to /apis/$group/$version
Kube-apiserver: resolved a regression that treated 304 Not Modified responses from aggregated API servers as internal errors.
Kube-proxy no longer falls back from ipvs mode to iptables mode if you ask it to do ipvs but the system is not correctly configured. Instead, it will just exit with an error.
Kube-scheduler: added taints filtering logic consistent with TaintToleration plugin for PodTopologySpread plugin.
Kubeadm will cleanup the stale data on best effort basis. Stale data will be removed when each reset phase are executed, default etcd data directory will be cleanup when the remove-etcd-member phase are executed.
Kubeadm: fixed a bug when performing validation on ClusterConfiguration networking fields.
Kubeadm: when a sub command is needed but not provided for a kubeadm command, print a help screen instead of showing a short message.
Kubectl apply: warning that kubectl will ignore no-namespaced resource pv & namespace in a future release if the namespace is specified and allowlist is not specified.
Kubectl: fixed a bug where kubectl convert did not pick the right API version
Kubelet now cleans up the Node's cloud node IP annotation correctly if you stop using --node-ip. (In particular, this fixes the problem where people who were unnecessarily using --node-ip with an external cloud provider in 1.23, and then running into problems with 1.24, could not fix the problem by just removing the unnecessary --node-ip from the kubelet arguments, because that wouldn't remove the annotation that caused the problems.) 
Kubelet: Fixed a startup crash in devicemanager.
Kubelet: fixed log spam from kubelet_getters.go Path does not exist.
Kubelet: fixed nil pointer in reflector start for standalone mode.
Kubelet: when there are multi option lines in /etc/resolv.conf, merge all options into one line in a pod with the Default DNS policy.
Log messages and metrics for the watch cache are now keyed by `<resource>.<group>` instead of go struct type. This means e.g. that *v1.Pod becomes pods. Additionally, resources that come from CustomResourceDefinitions are displayed as the correct resource and group, instead of *unstructured.Unstructured.
Moved LocalStorageCapacityIsolationFSQuotaMonitoring back to Alpha.
NOTE
Nested MountPoints are now grouped correctly on all cases.
Pod failed in scheduling due to expected error will be updated with the reason of SchedulerError rather than Unschedulable.
Pod logs using --timestamps are not broken up with timestamps anymore.
Removed of raising an error when setting an annotation with the same value, just ignore it.
Resolved an issue that caused winkernel proxier to treat stale VIPs as valid.
The ResourceVersion returned in objects from delete responses is now consistent with the ResourceVersion contained in the delete watch event.
The kube-scheduler and kube-controller-manager now use server side apply to set conditions related to pod disruption.
The errors in k8s.io/apimachinery/pkg/api/meta now support for the stdlibs errors.Is matching, including when wrapped.
The metrics etcd_request_duration_seconds and etcd_bookmark_counts now differentiate by group resource instead of object type, allowing unique entries per CustomResourceDefinition, instead of grouping them all under *unstructured.Unstructured.
The pod admission error message was improved for usability.
The time duration of a failed or unschedulable scheduling attempt will be longer, it now includes the time duration of the unreserve operation.
Updated kube-proxy to restart in case it detects that the Node assigned pod.Spec.PodCIDRs have changed.
Updated creation of LoadBalancer services, for there to be fewer AWS security group rules in most cases.
Updated the system-validators library to v1.8.0
Updates golang.org/x/text`` to v0.3.8`` to fix CVE-2022-32149
Volume mount cleanup now considers only plugin directory and not the entire kubelet root
kubectl now escapes terminal special characters in output. This fixes CVE-2021-25743.

Other (Cleanup or Flake)

'Promoted cronjob_job_creation_skew metric to stable to follow the cronjob v2 controller, the following metrics had their name updated to match metrics API guidelines:
cronjob_job_creation_skew_duration_seconds -> job_creation_skew_duration_seconds.'
'Promoted job-related metrics to stable to follow IndexedJobs GA. The following metrics have their name updated to match metrics API guidelines:
job_sync_total -> job_syncs_total
job_finished_total -> jobs_finished_total'
'kubelet_kubelet_credential_provider_plugin_duration was renamed to kubelet_credential_provider_plugin_duration and kubelet_kubelet_credential_provider_plugin_errors was renamed to kubelet_credential_provider_plugin_errors.'
A new API server flag --encryption-provider-config-automatic-reload was added to control when the encryption config should be automatically reloaded without needing to restart the server. All KMS plugins are now merged into a single healthz check at /healthz/kms-providers when reload is enabled, or when only KMS v2 plugins are used.
Added a --prune-allowlist flag that can be used with kubectl apply --prune. This flag now replaces and functions the same as the --prune-whitelist flag, which has been deprecated.
Added a kubernetes_feature_enabled metric which will tell you if a feature is enabled.
Deprecated the following kubectl run flags, which are ignored if set: --cascade, --filename, --force, --grace-period, --kustomize, --recursive, --timeout, --wait.
Dropped support for the Container Runtime Interface (CRI) version v1alpha2, which means that container runtimes just have to implement v1.
E2e: tests can now register callbacks with ginkgo.BeforeEach, ginkgo.AfterEach or ginkgo.DeferCleanup directly after creating a framework instance and are guaranteed that their code is called after the framework is initialized and before it gets cleaned up. ginkgo.DeferCleanup replaces f.AddAfterEach and AddCleanupAction which got removed to simplify the framework.
Introduce ComponentSLIs alpha feature-gate for component SLIs metrics endpoint.
Kube scheduler Component Config release version v1beta3 is deprecated in v1.26 and will be removed in v1.29, also v1beta2 will be removed in v1.28.
Kube-scheduler: the DefaultPodTopologySpread, NonPreemptingPriority, PodAffinityNamespaceSelector and PreferNominatedNode feature gates that graduated to GA in v1.24 and were unconditionally enabled have been removed in v1.26.
Kubeadm: removed the UnversionedKubeletConfigMap feature gate. The feature has been GA and locked to enabled since v1.25.
Kubeadm: removed the toleration for the node-role.kubernetes.io/master taint from the CoreDNS deployment of kubeadm. With the 1.25 release of kubeadm the taint node-role.kubernetes.io/master is no longer applied to control plane nodes and the toleration for it can be removed with the release of 1.26. You can also perform the same toleration removal from your own addon manifests.
Kubeadm: removed the usage of the --container-runtime=remote flag for the kubelet during kubeadm init/join/upgrade. The flag value remote had been the only possible value since dockershim was removed from the kubelet.
Locked ServerSideApply feature gate to true with the feature already being GA.
Refactored test/e2e/framework so that the core framework is smaller. Optional functionality like resource monitoring, log size monitoring, metrics gathering and debug information dumping must be imported by specific e2e test suites. Init packages are provided which can be imported to re-enable the functionality that traditionally was in the core framework. If you have code that no longer compiles because of this PR, you can use the script from a commit message to update that code.
Release-note
Removed PodOverhead feature gate as the feature is in GA since v1.24.
Removing Windows Server, Version 20H2 flavors from various container images.
Renamed the feature gate for CEL in Admission Control to ValidatingAdmissionPolicy.
Reworded log message upon image garbage collection failure to be more clear.
Scheduler dumper now exposes a summary to indicate the number of pending pods in each internal queue.
Service session affinity timeout tests are no longer required for Kubernetes network plugin conformance due to variations in existing implementations. New conformance tests will be developed to better express conformance in future releases.
The IndexedJob and SuspendJob feature gates that graduated to GA in 1.24 and were unconditionally enabled have been removed in v1.26.
The e2e.test binary no longer emits JSON structs to document progress.
The in-tree cloud provider for OpenStack (and the cinder volume provider) has been removed. Please use the external cloud provider and csi driver from cloud-provider-openstack instead.
The metric etcd_db_total_size_in_bytes is renamed to apiserver_storage_db_total_size_in_bytes.
Updated cri-tools to [v1.25.0(https://github.com/kubernetes-sigs/cri-tools/releases/tag/v1.25.0)
GlusterFS in-tree storage driver which was deprecated in kubernetes 1.25 release is now removed entirely in 1.26.

依赖

已添加的

cloud.google.com/go/datastore: v1.1.0
cloud.google.com/go/firestore: v1.1.0
cloud.google.com/go/pubsub: v1.3.1
github.com/OneOfOne/xxhash: v1.2.2
github.com/alecthomas/template: fb15b89
github.com/alecthomas/units: f65c72e
github.com/armon/consul-api: eb2c6b5
github.com/armon/go-metrics: f0300d1
github.com/armon/go-radix: 7fddfc3
github.com/bgentry/speakeasy: v0.1.0
github.com/bketelsen/crypt: 5cbc8cc
github.com/cenkalti/backoff/v4: v4.1.3
github.com/cespare/xxhash: v1.1.0
github.com/client9/misspell: v0.3.4
github.com/coreos/bbolt: v1.3.2
github.com/coreos/etcd: v3.3.13+incompatible
github.com/coreos/go-systemd: 95778df
github.com/coreos/pkg: 399ea9e
github.com/dgrijalva/jwt-go: v3.2.0+incompatible
github.com/dgryski/go-sip13: e10d5fe
github.com/fatih/color: v1.7.0
github.com/go-gl/glfw: e6da0ac
github.com/go-logr/stdr: v1.2.2
github.com/google/martian: v2.1.0+incompatible
github.com/grpc-ecosystem/grpc-gateway/v2: v2.7.0
github.com/hashicorp/consul/api: v1.1.0
github.com/hashicorp/consul/sdk: v0.1.1
github.com/hashicorp/errwrap: v1.0.0
github.com/hashicorp/go-cleanhttp: v0.5.1
github.com/hashicorp/go-immutable-radix: v1.0.0
github.com/hashicorp/go-msgpack: v0.5.3
github.com/hashicorp/go-multierror: v1.0.0
github.com/hashicorp/go-rootcerts: v1.0.0
github.com/hashicorp/go-sockaddr: v1.0.0
github.com/hashicorp/go-syslog: v1.0.0
github.com/hashicorp/go-uuid: v1.0.1
github.com/hashicorp/go.net: v0.0.1
github.com/hashicorp/golang-lru: v0.5.1
github.com/hashicorp/hcl: v1.0.0
github.com/hashicorp/logutils: v1.0.0
github.com/hashicorp/mdns: v1.0.0
github.com/hashicorp/memberlist: v0.1.3
github.com/hashicorp/serf: v0.8.2
github.com/jpillora/backoff: v1.0.0
github.com/jstemmer/go-junit-report: v0.9.1
github.com/kr/logfmt: b84e30a
github.com/kr/pty: v1.1.1
github.com/magiconair/properties: v1.8.1
github.com/mattn/go-colorable: v0.0.9
github.com/mattn/go-isatty: v0.0.3
github.com/miekg/dns: v1.0.14
github.com/mitchellh/cli: v1.0.0
github.com/mitchellh/go-homedir: v1.1.0
github.com/mitchellh/go-testing-interface: v1.0.0
github.com/mitchellh/gox: v0.4.0
github.com/mitchellh/iochan: v1.0.0
github.com/oklog/ulid: v1.3.1
github.com/pascaldekloe/goe: 57f6aae
github.com/pelletier/go-toml: v1.2.0
github.com/posener/complete: v1.1.1
github.com/prometheus/tsdb: v0.7.1
github.com/ryanuber/columnize: 9b3edd6
github.com/sean-/seed: e2103e2
github.com/shurcooL/sanitized_anchor_name: v1.0.0
github.com/spaolacci/murmur3: f09979e
github.com/spf13/cast: v1.3.0
github.com/spf13/jwalterweatherman: v1.0.0
github.com/spf13/viper: v1.7.0
github.com/subosito/gotenv: v1.2.0
github.com/ugorji/go: v1.1.4
github.com/xordataexchange/crypt: b2862e3
go.opentelemetry.io/contrib/propagators/b3: v1.10.0
go.opentelemetry.io/otel/exporters/otlp/internal/retry: v1.10.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc: v1.10.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace: v1.10.0
gopkg.in/ini.v1: v1.51.0
gopkg.in/resty.v1: v1.12.0
rsc.io/binaryregexp: v0.2.0
rsc.io/quote/v3: v3.1.0
rsc.io/sampler: v1.3.0

变化的

dmitri.shuralyov.com/gpu/mtl: 28db891 → 666a987
github.com/antlr/antlr4/runtime/Go/antlr: f25a4f6 → v1.4.10
github.com/aws/aws-sdk-go: v1.38.49 → v1.44.116
github.com/container-storage-interface/spec: v1.6.0 → v1.7.0
github.com/containerd/ttrpc: v1.0.2 → v1.1.0
github.com/cpuguy83/go-md2man/v2: v2.0.1 → v2.0.2
github.com/dnaeon/go-vcr: v1.0.1 → v1.2.0
github.com/docker/docker: v20.10.17+incompatible → v20.10.18+incompatible
github.com/docker/go-units: v0.4.0 → v0.5.0
github.com/emicklei/go-restful/v3: v3.8.0 → v3.9.0
github.com/felixge/httpsnoop: v1.0.1 → v1.0.3
github.com/fsnotify/fsnotify: v1.4.9 → v1.6.0
github.com/go-kit/log: v0.1.0 → v0.2.0
github.com/go-logfmt/logfmt: v0.5.0 → v0.5.1
github.com/go-openapi/jsonreference: v0.19.5 → v0.20.0
github.com/google/cadvisor: v0.45.0 → v0.46.0
github.com/google/cel-go: v0.12.4 → v0.12.5
github.com/google/go-cmp: v0.5.6 → v0.5.9
github.com/google/pprof: 94a9f03 → 4bb14d4
github.com/gopherjs/gopherjs: fce0ec3 → 0766667
github.com/inconshreveable/mousetrap: v1.0.0 → v1.0.1
github.com/karrick/godirwalk: v1.16.1 → v1.17.0
github.com/konsorten/go-windows-terminal-sequences: v1.0.2 → v1.0.3
github.com/matttproud/golang_protobuf_extensions: v1.0.1 → v1.0.2
github.com/moby/sys/mountinfo: v0.6.0 → v0.6.2
github.com/moby/term: 3f7ff69 → 39b0c02
github.com/onsi/ginkgo/v2: v2.1.4 → v2.4.0
github.com/onsi/gomega: v1.19.0 → v1.23.0
github.com/opencontainers/runc: v1.1.3 → v1.1.4
github.com/prometheus/client_golang: v1.12.1 → v1.14.0
github.com/prometheus/client_model: v0.2.0 → v0.3.0
github.com/prometheus/common: v0.32.1 → v0.37.0
github.com/prometheus/procfs: v0.7.3 → v0.8.0
github.com/smartystreets/assertions: v1.1.0 → b2de0cb
github.com/spf13/afero: v1.6.0 → v1.2.2
github.com/spf13/cobra: v1.4.0 → v1.6.0
github.com/stretchr/objx: v0.2.0 → v0.4.0
github.com/stretchr/testify: v1.7.0 → v1.8.0
go.etcd.io/etcd/api/v3: v3.5.4 → v3.5.5
go.etcd.io/etcd/client/pkg/v3: v3.5.4 → v3.5.5
go.etcd.io/etcd/client/v2: v2.305.4 → v2.305.5
go.etcd.io/etcd/client/v3: v3.5.4 → v3.5.5
go.etcd.io/etcd/pkg/v3: v3.5.4 → v3.5.5
go.etcd.io/etcd/raft/v3: v3.5.4 → v3.5.5
go.etcd.io/etcd/server/v3: v3.5.4 → v3.5.5
go.opentelemetry.io/contrib/instrumentation/github.com/emicklei/go-restful/otelrestful: v0.20.0 → v0.35.0
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc: v0.20.0 → v0.35.0
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.20.0 → v0.35.0
go.opentelemetry.io/otel/metric: v0.20.0 → v0.31.0
go.opentelemetry.io/otel/sdk: v0.20.0 → v1.10.0
go.opentelemetry.io/otel/trace: v0.20.0 → v1.10.0
go.opentelemetry.io/otel: v0.20.0 → v1.10.0
go.opentelemetry.io/proto/otlp: v0.7.0 → v0.19.0
go.uber.org/goleak: v1.1.10 → v1.2.0
golang.org/x/crypto: 3147a52 → v0.1.0
golang.org/x/exp: 85be41e → 6cc2880
golang.org/x/mobile: e6ae53a → d2bd2a2
golang.org/x/mod: 86c51ed → v0.6.0
golang.org/x/net: a158d28 → 1e63c2f
golang.org/x/oauth2: d3ed0bb → ee48083
golang.org/x/sys: 8c9f86f → v0.3.0
golang.org/x/term: 03fcf44 → v0.3.0
golang.org/x/text: v0.3.7 → v0.5.0
golang.org/x/tools: v0.1.12 → v0.2.0
google.golang.org/grpc: v1.47.0 → v1.49.0
google.golang.org/protobuf: v1.28.0 → v1.28.1
k8s.io/gengo: c02415c → c0856e2
k8s.io/klog/v2: v2.70.1 → v2.80.1
k8s.io/kube-openapi: 67bda5d → 172d655
k8s.io/system-validators: v1.7.0 → v1.8.0
k8s.io/utils: ee6ede2 → 1a15be2
sigs.k8s.io/apiserver-network-proxy/konnectivity-client: v0.0.32 → v0.0.33
sigs.k8s.io/yaml: v1.2.0 → v1.3.0

已移除的

github.com/ajstarks/svgo: 644b8db
github.com/auth0/go-jwt-middleware: v1.0.1
github.com/boltdb/bolt: v1.3.1
github.com/fogleman/gg: 0403632
github.com/getkin/kin-openapi: v0.76.0
github.com/go-ozzo/ozzo-validation: v3.5.0+incompatible
github.com/golang/freetype: e2365df
github.com/gophercloud/gophercloud: v0.1.0
github.com/gorilla/mux: v1.8.0
github.com/heketi/heketi: v10.3.0+incompatible
github.com/heketi/tests: f3775cb
github.com/jung-kurt/gofpdf: 24315ac
github.com/kr/fs: v0.1.0
github.com/lpabon/godbc: v0.1.1
github.com/mvdan/xurls: v1.1.0
github.com/pkg/sftp: v1.10.1
github.com/remyoudompheng/bigfft: 52369c6
github.com/russross/blackfriday: v1.5.2
github.com/urfave/negroni: v1.0.0
go.opentelemetry.io/contrib/propagators: v0.20.0
go.opentelemetry.io/contrib: v0.20.0
go.opentelemetry.io/otel/exporters/otlp: v0.20.0
go.opentelemetry.io/otel/oteltest: v0.20.0
go.opentelemetry.io/otel/sdk/export/metric: v0.20.0
go.opentelemetry.io/otel/sdk/metric: v0.20.0
gonum.org/v1/gonum: v0.6.2
gonum.org/v1/netlib: 7672324
gonum.org/v1/plot: e2840ee
modernc.org/cc: v1.0.0
modernc.org/golex: v1.0.0
modernc.org/mathutil: v1.0.0
modernc.org/strutil: v1.0.0
modernc.org/xc: v1.0.0
rsc.io/pdf: v0.1.1

相关链接

英文原文：https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.26.md#v1260