29 KiB
Table of Contents generated with DocToc
- v1.13.1
- v1.13.0
- v1.13.0-rc.0
- v1.13.0-beta.0
- v1.13.0-alpha.2
- v1.13.0-alpha.1
v1.13.1
Downloads for v1.13.1
Download v1.13.1 in the v1.13.1 release page.
Changelog since v1.13.0
Changes by Kind
Bug Fixes
karmada-operator
: Thekarmada-app
label key previously used for control plane components and for the components of the operator itself has been changed to the more idiomaticapp.kubernetes.io/name
label key. (#6200, @jabellard)karmada-agent
: Fixed a panic issue where the agent does not need to report secret when registering cluster. (#6222, @jabellard)karmada-controller-manager
: Fixed the issue that the gracefulEvictionTask of ResourceBinding can not be cleared in case of schedule fails. (#6234, @XiShanYongYe-Chang)helm
: Fixed the issue where the required ServiceAccount was missing when the certificate mode was set to custom. (#6242, @seanlaii)
Others
None.
v1.13.0
Downloads for v1.13.0
Download v1.13.0 in the v1.13.0 release page.
Urgent Update Notes
- The Karmada Lua interpreter will no longer support the Lua functions
string.rep
andstring.gsub
. Typically, these functions are not frequently used in custom Karmada resource interpreters. They are disabled due to potential security risks. Before upgrading, please review your Lua scripts to verify whether these functions are being used. If they are, please replace them with alternative implementations.
What's New
Application Priority Scheduling
In some real-world scenarios, like AI training, some jobs are more critical than others and require preferential treatment in terms of scheduling and resource allocation. To enhance support for the above scenarios, this release karmada introduces the application priority scheduling feature, which allows users to set the priority of applications to control the order of application scheduling. Applications with higher priority will be scheduled ahead of other applications with lower priority.
With this feature, users can:
- Classify workloads based on priority to achieve that high-priority workloads are scheduled ahead of others, rather than waiting to be scheduled in the order they were queued. This will allow users to maintain the quality of service for important workloads even during resource contention.
- Set a higher priority for AI training tasks, ensuring that they can use GPU resources preferentially and guaranteeing the normal progress of AI training tasks.
For a detailed description of this feature, see the Proposal: Binding Priority and Preemption
(Feature contributors: @whitewindmills, @LeonZh0u, @seanlaii, @wengyao04, @zclyne)
Support for resource scheduling suspend and resume capabilities
Resource propagation in Karmada can be simply understood as two phases, resource scheduling and resource propagation. The propagation phase could already be flexibly suspended and resumed by users, a feature introduced in release v1.11. Now, the scheduling phase also has this feature.
By expanding the ResourceBinding.ResourceBindingSpec.Spec.Suspension
field, users gain control over the suspension and resumption of scheduling. This enhancement opens up opportunities for external controllers to implement more advanced functionalities. For example:
-
It helps in multi-tenancy scenarios, enabling priority control for different tenants. A cluster administrator can perform workload sorting and other operations on the upper layer, and resume scheduling if the sorting is completed.
-
Quota checks are also possible during scheduling. Workloads will be dispatched only when the tenant resources are sufficient. Otherwise, the workload will be suspended for scheduling to achieve multi-tenant quota control.
For a detailed description of this feature, see the Proposal: Support for resource scheduling suspend and resume capabilities
(Feature contributors: @Vacant2333, @Monokaix)
Karmada Operator Continuous Enhancement
This release continues to enhance the Karmada Operator, which is responsible for managing the lifecycle of Karmada components. The following features are added:
- Support for API Server sidecar in Karmada Operator: This feature allows users to customize the Karmada API server container with sidecar containers, enabling users to integrate auxiliary services such as KMS plugins for configuring encryption at rest. For more information, see the Proposal: Add Support for API Server Sidecar Containers in Karmada Operator
- Support for Component Priority Class Configuration in Karmada Operator: This feature allows users to configure the priority class for Karmada control plane components in the Karmada CR, ensuring reliable resource allocation and system stability across workloads. For more information, see the Proposal: Support Priority Class Configuration for Karmada Control Plane Components
These enhancements allow karmada-operator to be more flexible and customizable, enhancing the overall karmada system reliability and stability.
(Feature contributors: @jabellard)
Remarkable Performance Optimization of the Karmada Controllers
As Karmada is being adopted by an increasing number of vendors and the deployment scale is growing larger, the performance optimization of Karmada has become a top priority. Karmada adopters spontaneously organize themselves to optimize the performance of Karmada, and achieved remarkable results.
In this release, the performance optimization of Karmada mainly centers around the controllers. The primary focus is to address the issue where the detector and binding-controller consume a significant amount of time during component restarts or master switches in large-scale data scenarios.
To address this, in v1.13, we carried out systematic performance optimizations. Testing was done on a 12C CPU, 28G RAM physical machine with 1000 deployments and 1000 configmaps (2000 resourceBindings). After restarting karmada-controller-manager, workqueue metrics showed significant improvements: For detector: Element Average Processing Time(1-minute Sampling Period) Peak from 2.96s to 1.74s (41.22% reduction), total queue-clear time decreased by 60%, and peak-to-clear time decreased by 33.33%. For binding-controller: Element Average Processing Time(1-minute Sampling Period) Peak from 1.98s to 0.96s (51.72% reduction), total queue-clear time decreased by 25%, and peak-to-clear time also decreased by 25%.
These data prove that the performance of these two controllers has been greatly enhanced in v1.13. In the future, we will continue to carry out systematic performance optimizations on the controllers and the scheduler.
(Feature contributors: @zach593, @CharlesQQ, @chaosi-zju)
Release of the First Version of the Karmada Dashboard
With the continuous efforts of several enthusiastic developers, karmada dashboard finally welcomes its milestone first version (v0.1.0)! This version marks a significant step forward in the visual management of Karmada.
Karmada Dashboard is a general-purpose, web-based control panel for Karmada which is a multi-cluster management project. It aims to simplify the operation process of multi-cluster management and enhance the user experience. Through the Dashboard, users can intuitively view the cluster status, resource distribution, and task execution status. At the same time, users can also easily complete configuration adjustments and policy deployments.
The main features of this version include:
- Cluster Management: It provides cluster access and an overview of the cluster status, including health status and nodes number.
- Resource Management: It manages the configuration of business resources, covering namespaces, workloads, services, etc.
- Policy Management: It conducts the management of Karmada policies, including propagation policies, override policies, etc.
For a detailed description and usage guide, see the Karmada Dashboard
(Feature contributors: @samzong, @RainbowMango, @warjiang, @axif0, @chouchongYHMing, @chaosi-zju, @Heylosky, @adwait-godbole, @carlory, @chouchongYHMing, @devadapter, @guozheng-shen, @jhnine, @LeonZh0u, @LiZhenCheng9527, @shauvet, @vibgreon, @zhouqunjie-cs)
Other Notable Changes
API Changes
- Add
SchedulePriority
field toPropagationPolicy
andClusterPropagationPolicy
: (#5962, @seanlaii)- Supports multiple priority class sources (Kubernetes, PodTemplate, Federated[future])
- Configurable priority class resolution strategy
- Maintains compatibility with Kubernetes priority behavior
- Optional fields preserve backward compatibility
- Add
SchedulePriority
,Priority
, andPreemptionPolicy
fields toResourceBinding
API to support priority-based scheduling and preemption control. (#5963, @seanlaii)- Priority: Defines scheduling priority (higher values = higher priority)
- PreemptionPolicy: Controls whether binding can preempt lower priority bindings
- Make
priorityClassSource
in theSchedulePriority
ofPropagationPolicy
a required field. (#6163, @seanlaii) API Change
: IntroducedPriorityClassName
inKarmada
API which will be used to specify the priority class name of that component. (#6068, @jabellard)API Change
: IntroducedScheduling
suspension in bothResourceBinding
andClusterResourceBinding
which will be used for third-party systems to suspend application scheduling. (#5937, @Monokaix)karmada-operator
: The newsidecarContainers
field as part of the Karmada API server component config can optionally be used to specify sidecar containers for the Karmada API server. (#6133, @jabellard)
Deprecation
- Replace
grpc.DialContext
andgrpc.WithBlock
withgrpc.NewClient
since DialContext and WithBlock are deprecated while maintaining the original functionality. (#6026, @seanlaii) karmadactl
: The flag--ca-cert-path
of commandregister
has been marked deprecated because it has never been used, and will be removed in the future release. (#5862, @zhzhuang-zju)
Bug Fixes
karmada-controller-manager
: Fixed an issue that the scheduling suspension on ResourceBinding might be mistakenly overwritten. (#6062, @Monokaix)karmada-controller-manager
: Fixed the issue that newly created attached-ResourceBinding be mystically garbage collected. (#6034, @whitewindmills)karmada-controller-manager
: Fixed the problem of ResourceBinding remaining after the resource template is deleted in the dependencies distribution scenario. (#5943, @XiShanYongYe-Chang)karmada-controller-manager
: Fixed the bug of WorkloadRebalancer doesn't get deleted after TTL. (#5989, @chaosi-zju)karmada-controller-manager
: Fixed the issue of missing work queue metrics. (#5972, @XiShanYongYe-Chang)karmada-controller-manager
: Fixed the issue where thedetector
unnecessary updates for RB issue. (#6157, @CharlesQQ)karmada-operator
: fix the "no such host" error when accessing the /convert webhook if Karmada instance is deployed in a namespace other than karmada-system. (#6079, @zhzhuang-zju)karmada-operator
: Fixed the issue that external ETCD certificate be overwritten by generated in-cluster ETCD certificate. (#5976, @jabellard)karmada-operator
: Fixed the error of validation failure when some fields of the Karmada CR are not configured. (#6158, @zhzhuang-zju)karmada-webhook
: Fixed panic when validating ResourceInterpreterWebhookConfiguration with unspecified service port. (#5960, @XiShanYongYe-Chang)karmada-search
: Fixed the issue that namespaces in different ResourceRegistry might be overwritten. (#6065, @JimDevil)karmadactl
: fix the "no such host" error when accessing the /convert webhook if Karmada instance is deployed in a namespace other than karmada-system via theinit
command. (#6079, @zhzhuang-zju)
Security
karmada-controller-manager
: For security reasons, we made the following changes to restrict the string library functions used when users customize the Karmada Lua interpreter. (#6087, @zhzhuang-zju)- do not allow users to use string.gsub and string.rep when interpreting resources with lua scripts, which may be used to create overly long strings.
- limit the length of the string type parameters of the function to 1000,000.
- add timeout checks to the internal for loops within the functions.
Features & Enhancements
karmada-controller-manager
: FlinkDeployment health interpreter improvements, adding status.error to reflected status. (#6073, @mszacillo)karmada-controller-manager
: Populate schedule priority when building ResourceBinding. (#6165, @LeonZh0u)karmada-metrics-adapter
: Introduced--metrics-bind-address
flag which will be used to expose Prometheus metrics. (#6013, @chaosi-zju)karmada-operator
: standardize the naming of karmada config in karmada-operator installation method. (#6082, @seanlaii)karmadactl
: commandinit
now can specify the priority class name of the karmada components, default tosystem-node-critical
. (#6110, @zhzhuang-zju)karmadactl
: Theunjoin
command is restricted to only unjoin push mode member clusters. Theunregister
command is restricted to only unregister pull mode member clusters. (#6081, @zhzhuang-zju)karmadactl
: standardize the naming of karmada config in karmadactl installation method. (#5797, @chaosi-zju)karmadactl
: Add Fish shell autocompletion support for improved command-line efficiency. (#5876, @tiansuo114)karmadactl
: commandaddon
now can specify the priority class name of the karmada components, default tosystem-node-critical
. (#6111, @zhzhuang-zju)
Other
Dependencies
- Karmada now built with Golang v1.22.12. (#6131, @sachinparihar)
- The base image
alpine
now has been promoted from 3.21.2 to 3.21.3. (#6124) - update kubernetes version to v1.31.3 (#5910, @dongjiang1989)
Helm Charts
helm
: The newPriorityClassName
field added as part of the Karmada control plane component configurations can be used to specify the priority class name of that component, default tosystem-node-critical
. (#6108, @zhzhuang-zju)- upgrade helm chart index to v1.12.0. (#5918, @chaosi-zju)
Instrumentation
- The cluster status-related metrics, emitting from
karmada-controller-manager
, will be cleaned up after the cluster is removed. (#5866, @CharlesQQ)
Contributors
Thank you to everyone who contributed to this release!
Users whose commits are in this release (alphabetically by username)
- @adwait-godbole
- @anujagrawal699
- @axif0
- @carlory
- @chaosi-zju
- @CharlesQQ
- @chouchongYHMing
- @devadapter
- @dongjiang1989
- @gabrielsrs
- @guozheng-shen
- @Heylosky
- @iawia002
- @jabellard
- @jhnine
- @JimDevil
- @LavredisG
- @LeonZh0u
- @LiZhenCheng9527
- @ls-2018
- @mohamedawnallah
- @Monokaix
- @mszacillo
- @RainbowMango
- @sachinparihar
- @samzong
- @seanlaii
- @shauvet
- @SkySingh04
- @tiansuo114
- @Vacant2333
- @vibgreon
- @warjiang
- @whitewindmills
- @XiShanYongYe-Chang
- @y1hao
- @yashpandey06
- @zach593
- @zhouqunjie-cs
- @zhzhuang-zju
- @ZwangaMukwevho
v1.13.0-rc.0
Downloads for v1.13.0-rc.0
Download v1.13.0-rc.0 in the v1.13.0-rc.0 release page.
Changelog since v1.13.0-beta.0
Urgent Update Notes
None.
Changes by Kind
API Changes
None.
Features & Enhancements
karmada-controller-manager
: FlinkDeployment health interpreter improvements, adding status.error to reflected status. (#6073, @mszacillo)karmada-operator
: standardize the naming of karmada config in karmada-operator installation method. (#6082, @seanlaii)karmadactl
: commandinit
now can specify the priority class name of the karmada components, default tosystem-node-critical
. (#6110, @zhzhuang-zju)karmadactl
: Theunjoin
command is restricted to only unjoin push mode member clusters. Theunregister
command is restricted to only unregister pull mode member clusters. (#6081, @zhzhuang-zju)
Deprecation
None.
Bug Fixes
karmada-operator
: fix the "no such host" error when accessing the /convert webhook if Karmada instance is deployed in a namespace other than karmada-system. (#6079, @zhzhuang-zju)karmadactl
: fix the "no such host" error when accessing the /convert webhook if Karmada instance is deployed in a namespace other than karmada-system via theinit
command. (#6079, @zhzhuang-zju)
Security
None.
Other
Dependencies
None.
Helm Charts
helm
: The newPriorityClassName
field added as part of the Karmada control plane component configurations can be used to specify the priority class name of that component, default tosystem-node-critical
. (#6108, @zhzhuang-zju)
Instrumentation
- The cluster status-related metrics, emitting from
karmada-controller-manager
, will be cleaned up after the cluster is removed. (#5866, @CharlesQQ)
v1.13.0-beta.0
Downloads for v1.13.0-beta.0
Download v1.13.0-beta.0 in the v1.13.0-beta.0 release page.
Changelog since v1.13.0-alpha.2
Urgent Update Notes
- The Karmada Lua interpreter will no longer support the Lua functions
string.rep
andstring.gsub
. Typically, these functions are not frequently used in custom Karmada resource interpreters. They are disabled due to potential security risks. Before upgrading, please review your Lua scripts to verify whether these functions are being used. If they are, please replace them with alternative implementations.
Changes by Kind
API Changes
API Change
: IntroducedPriorityClassName
inKarmada
API which will be used to specify the priority class name of that component. (#6068, @jabellard)
Features & Enhancements
karmadactl
: standardize the naming of karmada config in karmadactl installation method. (#5797, @chaosi-zju)
Deprecation
None.
Bug Fixes
karmada-controller-manager
: Fixed an issue that the scheduling suspension on ResourceBinding might be mistakenly overwritten. (#6062, @Monokaix)karmada-search
: Fixed the issue that namespaces in different ResourceRegistry might be overwritten. (#6065, @JimDevil)
Security
karmada-controller-manager
: For security reasons, we made the following changes to restrict the string library functions used when users customize the Karmada Lua interpreter. (#6087, @zhzhuang-zju)- do not allow users to use string.gsub and string.rep when interpreting resources with lua scripts, which may be used to create overly long strings.
- limit the length of the string type parameters of the function to 1000,000.
- add timeout checks to the internal for loops within the functions.
Other
Dependencies
- Karmada now built with Golang v1.22.11. (#6066, @y1hao)
Helm Charts
None.
Instrumentation
None.
v1.13.0-alpha.2
Downloads for v1.13.0-alpha.2
Download v1.13.0-alpha.2 in the v1.13.0-alpha.2 release page.
Changelog since v1.13.0-alpha.1
Urgent Update Notes
None.
Changes by Kind
API Changes
None.
Features & Enhancements
karmada-metrics-adapter
: Introduced--metrics-bind-address
flag which will be used to expose Prometheus metrics. (#6013, @chaosi-zju)
Deprecation
- Replace
grpc.DialContext
andgrpc.WithBlock
withgrpc.NewClient
since DialContext and WithBlock are deprecated while maintaining the original functionality. (#6026, @seanlaii)
Bug Fixes
karmada-controller-manager
: Fixed the issue that newly created attached-ResourceBinding be mystically garbage collected. (#6034, @whitewindmills)
Security
None.
Other
Dependencies
- The base image
alpine
now has been promoted from 3.21.0 to 3.21.2. (#6040)
Helm Charts
None.
Instrumentation
None.
v1.13.0-alpha.1
Downloads for v1.13.0-alpha.1
Download v1.13.0-alpha.1 in the v1.13.0-alpha.1 release page.
Changelog since v1.12.0
Urgent Update Notes
None.
Changes by Kind
API Changes
API Change
: IntroducedScheduling
suspension in bothResourceBinding
andClusterResourceBinding
which will be used for third-party systems to suspend application scheduling. (#5937, @Monokaix)
Features & Enhancements
karmadactl
: Add Fish shell autocompletion support for improved command-line efficiency. (#5876, @tiansuo114)
Deprecation
karmadactl
: The flag--ca-cert-path
of commandregister
has been marked deprecated because it has never been used, and will be removed in the future release. (#5862, @zhzhuang-zju)
Bug Fixes
karmada-controller-manager
: Fixed the problem of ResourceBinding remaining after the resource template is deleted in the dependencies distribution scenario. (#5943, @XiShanYongYe-Chang)karmada-controller-manager
: Fixed the bug of WorkloadRebalancer doesn't get deleted after TTL. (#5989, @chaosi-zju)karmada-controller-manager
: Fixed the issue of missing work queue metrics. (#5972, @XiShanYongYe-Chang)karmada-webhook
: Fixed panic when validating ResourceInterpreterWebhookConfiguration with unspecified service port. (#5960, @XiShanYongYe-Chang)karmada-operator
: Fixed the issue that external ETCD certificate be overwritten by generated in-cluster ETCD certificate. (#5976, @jabellard)
Security
None.
Other
Dependencies
- update kubernetes version to v1.31.3 (#5910, @dongjiang1989)
- The base image
alpine
now has been promoted from3.20.3
to3.21.0
. (#5920)
Helm Charts
- upgrade helm chart index to v1.12.0. (#5918, @chaosi-zju)
Instrumentation
None.