---
slug: openkruise-1.0
title: OpenKruise v1.0, Reaching New Peaks of application automation
authors: [FillZpp]
tags: [release]
---

We’re pleased to announce the release of Kubernetes 1.0, which is a CNCF Sandbox level project.

[OpenKruise](https://openkruise.io) is an extended component suite for Kubernetes, which mainly focuses on application automations, such as deployment, upgrade, ops and availability protection. Mostly features provided by OpenKruise are built primarily based on CRD extensions. They can work in pure Kubernetes clusters without any other dependences.

![openkruise-features|center|450x400](/img/blog/2021-12-13-release-1.0/features-en.png)

Overall, OpenKruise currently provides features in these areas:

- **Application workloads**: Enhanced strategies of deploy and upgrade for stateless/stateful/daemon applications, such as in-place update, canary/flowing upgrade.
- **Sidecar container management**: supports to define sidecar container alone, which means it can inject sidecar containers, upgrade them with no effect on application containers and even hot upgrade.
- **Enhanced operations**: such as restart containers in-place, pre-download images on specific nodes, keep containers launch priority in a Pod, distribute one resource to multiple namespaces.
- **Application availability protection**: protect availability for applications that deployed in Kubernetes.

## What's new?

### 1. InPlace Update for environments

*Author: [@FillZpp](https://github.com/FillZpp)*

OpenKruise has supported **InPlace Update** since very early version, mostly for workloads like CloneSet and Advanced StatefulSet. Comparing to recreate Pods during upgrade, in-place update only has to modify the fields in existing Pods.

![inplace-update-comparation|center|450x400](/img/docs/core-concepts/inplace-update-comparation.png)

As the picture shows above, we only modify the `image` field in Pod during in-place update. So that:

- Avoid additional cost of *scheduling*, *allocating IP*, *allocating and mounting volumes*.
- Faster image pulling, because of we can re-use most of image layers pulled by the old image and only to pull several new layers.
- When a container is in-place updating, the other containers in Pod will not be affected and remain running.

However, OpenKruise only supports to in-place update `image` field in Pod and has to recreate Pods if other fields need to update. All the way through, more and more users hope OpenKruise could support in-place update more fields such as `env` -- which is hard to implement, for it is limited by kube-apiserver.

After our unremitting efforts, OpenKruise finally support in-place update environments via Downward API since version v1.0. Take the CloneSet YAML below as an example, user has to set the configuration in annotation and write a env from it. After that, he just needs to modify the annotation value when changing the configuration. Kruise will restart all containers with env from the annotation in such Pod to enable the new configuration.

```yaml
apiVersion: apps.kruise.io/v1alpha1
kind: CloneSet
metadata:
  ...
spec:
  replicas: 1
  template:
    metadata:
      annotations:
        app-config: "... the real env value ..."
    spec:
      containers:
      - name: app
        env:
        - name: APP_CONFIG
          valueFrom:
            fieldRef:
              fieldPath: metadata.annotations['app-config']
  updateStrategy:
    type: InPlaceIfPossible
```

*At the same time, we have removed the limit of `imageID` for in-place update, which means you can update a new image with the same imageID to the old image.*

For more details please read [documentation](/docs/core-concepts/inplace-update).

### 2. Distribute resources over multiple namespaces

*Author: [@veophi](https://github.com/veophi)*

For the scenario, where the namespace-scoped resources such as Secret and ConfigMap need to be distributed or synchronized to different namespaces, the native k8s currently only supports manual distribution and synchronization by users one-by-one, which is very inconvenient. 

Typical examples: 
- When users want to use the imagePullSecrets capability of SidecarSet, they must repeatedly create corresponding Secrets in relevant namespaces, and ensure the correctness and consistency of these Secret configurations;
- When users want to configure some **common** environment variables, they probably need to distribute ConfigMaps to multiple namespaces, and the subsequent modifications of these ConfigMaps might require synchronization among these namespaces.

Therefore, in the face of these scenarios that require the resource distribution and **continuously synchronization across namespaces**, we provide a tool, namely **ResourceDistribution**, to do this automatically. 

Currently, ResourceDistribution supports the two kind resources --- **Secret & ConfigMap**. 

```yaml
apiVersion: apps.kruise.io/v1alpha1
kind: ResourceDistribution
metadata:
  name: sample
spec:
  resource:
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: game-demo
    data:
      ...
  targets:
  	namespaceLabelSelector:
      ...
    # or includedNamespaces, excludedNamespaces
```

So you can see ResourceDistribution is a kind of **cluster-scoped CRD**, which is mainly composed of two fields: **`resource` and `targets`**.
- `resource` is a **complete** and **correct** resource structure in YAML style.
- `targets` indicates the target namespaces that the resource should be distributed into.

For more details please read [documentation](/docs/user-manuals/resourcedistribution).

### 3. Container launch priority

*Author: [@Concurrensee](https://github.com/Concurrensee)*

Containers in a same Pod in it might have dependence, which means the application in one container runs depending on another container. For example:

1. Container A has to start first. Container B can start only if A is already running.
2. Container B has to exit first. Container A can stop only if B has already exited.

Currently, the sequences of containers start and stop are controlled by Kubelet.
Kubernetes used to have a KEP, which plans to add a type field for container to identify the priority of start and stop. However, it has been refused because of sig-node thought it may bring a huge change to code.

So OpenKruise provides a feature named **Container Launch Priority**, which helps user control the sequence of containers start in a Pod.

1. User only has to put the annotation `apps.kruise.io/container-launch-priority: Ordered` in a Pod, then Kruise will ensure all containers in this Pod should be started by the sequence of `pod.spec.containers` list.
2. If you want to customize the launch sequence, you can add `KRUISE_CONTAINER_PRIORITY` environment in container. The range of the value is `[-2147483647, 2147483647]`. The container with higher priority will be guaranteed to start before the others with lower priority.

For more details please read [documentation](/docs/user-manuals/containerlaunchpriority).

### 4. `kubectl-kruise` commandline tool

*Author: [@hantmac](https://github.com/hantmac)*

OpenKruise used to provide SDK like `kruise-api` and `client-java` for some programming languages, which can be imported into users' projects. On the other hand, some users also need to operate the workload resources with commandline in test environment.

However, the `rollout`, `set image` commands in original `kubectl` can only work for built-in workloads, such as Deployment and StatefulSet.

So, OpenKruise now provide a commandline tool named `kubectl-kruise`, which is a standard plugin of `kubectl` and can work for OpenKruise workload types.

```bash
# rollout undo cloneset
$ kubectl kruise rollout undo cloneset/nginx

#  rollout status advanced statefulset
$ kubectl kruise rollout status statefulsets.apps.kruise.io/sts-demo

# set image of a cloneset
$ kubectl kruise set image cloneset/nginx busybox=busybox nginx=nginx:1.9.1
```

For more details please read [documentation](/docs/cli-tool/kubectl-plugin).

### 5. Other changes

**CloneSet:**
- Add `maxUnavailable` field in `scaleStrategy` to support rate limiting of scaling up.
- Mark revision stable when all pods updated to it, won't wait all pods to be ready.

**WorkloadSpread:**
- Manage the pods that have created before WorkloadSpread.
- Optimize the update and retry logic for webhook injection.

**Advanced DaemonSet:**
- Support in-place update Daemon Pod.
- Support progressive annotation to control if pods creation should be limited by partition.

**SidecarSet:**
- Fix SidecarSet filter active pods.
- Add `SourceContainerNameFrom` and `EnvNames` fields in `transferenv` to make the container name flexible and the list shorter.

**PodUnavailableBudget:**
- Add no pub-protection annotation to skip validation for the specific Pod.
- PodUnavailableBudget controller watches workload replicas changed.

**NodeImage:**
- Add `--nodeimage-creation-delay` flag to delay NodeImage creation after Node ready.

**UnitedDeployment:**
- Fix pod NodeSelectorTerms length 0 when UnitedDeployment NodeSelectorTerms is nil.

**Other optimization:**
- kruise-daemon list and watch pods using protobuf.
- Export cache resync args and defaults to be 0 in chart value.
- Fix http checker reloading after webhook certs updated.
- Generate CRDs with original controller-tools and markers.

## Get Involved

Welcome to get involved with OpenKruise by joining us in Github/Slack/DingTalk/WeChat.
Have something you’d like to broadcast to our community?
Share your voice at our [Bi-weekly community meeting (Chinese)](https://shimo.im/docs/gXqmeQOYBehZ4vqo), or through the channels below:

- Join the community on [Slack](https://kubernetes.slack.com/channels/openkruise) (English).
- Join the community on DingTalk: Search GroupID `23330762` (Chinese).
- Join the community on WeChat: Search User `openkruise` and let the robot invite you (Chinese).
