karmada/docs/proposals/cleanup-propagated-resources/README.md

short-desc	title	authors	reviewers	approvers	creation-date	last-updated	status
Cleanup member cluster resources on unjoin	Cleanup Member Cluster Resources	@huiwq1990	@RainbowMango		2021-07-03	2020-07-03	provisional

Cleanup propagated resources

Table of Contents

• Cleanup propagated resources
  • Table of Contents
  • Motivation
    • Goals
  • Proposals
    • Implementation Details
      • Best Effort Strategy
      • Required Strategy
    • Addition ClusterConditionTypes (optional)
      • Unjoining ClusterConfigurationType
      • Failed ClusterConfigurationType

This document proposes a mechanism to specify that a member cluster resource should be removed from a managed cluster when leaving.

Motivation

When a cluster is unjoined, karmada should provide a mechanism to cleanup the resources propagated by karmada. Currently, when unjoin a cluster, Karmada first try to remove propagated resource, and will skip remove if the cluster not ready.

Goals

• Define how users can indicate that propagated resource should be removed when a cluster leaves
• Define cluster removal strategies

Proposals

The Cluster struct should be updated to contain a RemoveStrategy member. RemoveStrategy will initially support two values, Needless and Required.

• The Needless strategy will not cleanup any propagated resources.
• The Required strategy will halt the unjoin process and set the Cluster resource in a failed state when encountering errors. Unjoining is blocked until all propagated resources have been removed successfully.

By default, RemoveStrategy will be needless on all cluster. A user must explicitly set a removal strategy on the join cluster.

Implementation details

• Update cluster to add a new attribute:

    RemoveStrategy *RemoveStrategy `json:"removeStrategy,omitempty"`
  

    type RemoveStrategy string
  
    const (
      RemoveStrategyNeedless   RemoveStrategy = "Needless"
      RemoveStrategyRequired   RemoveStrategy = "Required"
    )
  

• Add a flag remove-strategy to karmadactl join , and set the value to RemoveStrategy attribute

• During unjoin cluster, karmada should consider remove strategy

In kubefed's propose, the author suggest add a BestEffort strategy, reviewers say we need a certain value, so we should not use it too. Karmada use the BestEffort strategy, currently.

Needless Strategy

Not need cleanup propagated resources when unjoining cluster. Karmada should use this strategy as default value, condsider the business risk.

Required Strategy

Clusters with the "required" removal strategy must remove execution namespaces successfully on unjoin. After removing the work resource, the client must verify that the propagated resource is no longer present. If a resource cannot be removed:

• UnjoinCluster should return a formatted error containing which work could not be removed
• A ClusterCondition should be added to the cluster status with a message indicating which resources could not be removed
• (optional) Set the ConditionType to Failed

Execution Controller

When cluster is unjoined, karmada should process by cluster remove strategy:

• Needless strategy ignore work delete event
• Required strategy return true if the work delete success, and set clustercondition if delete fail

Additional ClusterConditionTypes (optional)

Presently, there are only two ClusterConditionTypes, Unjoining and UnjoinFailed. Given the required remove strategy, a Cluster may end up in a state where the cluster is Ready but is in the process of "Unjoining". Adding additional conditions will make it easier to understand life cycle of cluster operator.

status:
  conditions:
  - lastTransitionTime: "2021-05-08T12:34:15Z"
    lastUpdateTime: "2021-05-08T12:34:30Z"
    message: Unjoining resources.
    reason: UnjoinCluster
    status: "True"
    type: Unjoining

Unjoining ClusterConditionType

The Unjoining condition should be set on a cluster when exec karmadactl unjoin command.

Failed ClusterConditionType

In the event of an error, such as failure to remove a resource with a required remove strategy, the cluster should enter a UnjoinFailed condition.