The author | si-yu wang (the wish) source | alibaba cloud native public number

background

OpenKruise is an open source Cloud Native application automation management suite of Ali Cloud. It is also a Sandbox project currently hosted under Cloud Native Computing Foundation (CNCF). It comes from alibaba’s containerized, cloud-native technology precipitation over the years. It is a standard extension component based on Kubernetes for large-scale application in Alibaba’s internal production environment. It closely follows the upstream community standards and ADAPTS to the technical concept and best practice of the Large-scale Internet scene.

Overview of new releases

Kruise released the latest v0.8.0 version (ChangeLog) on March 4, 2021. The following article gives an overview of the new version.

1. Add kruise-Daemon

Students who have used OpenKruise in the past must know that the component running after Kruise installation is Kruise-Manager, which is a centrally deployed Operator component, including a series of controller controllers and Webhooks.

Since V0.8.0, we added kruise-Daemon, a node component, which was deployed on each node via DaemonSet. In this way, community partners have been raised such as mirror preheating, container restart and other needs, there is a way to achieve!

Tips:

  • At present, the official image provided by Kruise supports amD64 (x86), ARM64, ARM/V7 architecture of Linux. If there are nodes in your cluster that are not in the above architecture, kruise-Daemon cannot run normally temporarily. Students who have such needs can make an issue to explain your needs.

  • If this is the case, or if you do not want to install Kruise-Daemon on certain nodes, you can specify the affinity rule for kruise-Daemon deployment by using the daemon.affinity parameter at helm installation time.

2. Scale image preheating capacity

In the Kubernetes ecosystem, there was no mature open source solution for image preheating in the past. It was more likely that some companies implemented some internal preheating that was suitable for local scenarios, including Alibaba. However, since V0.8.0, we have exported the image preheating capability made by Alibaba to OpenKruise, and the image preheating capability made by Alibaba has been completely unified into this open source implementation.

The specific implementation principle of OpenKruise image preheating, we will do a detailed introduction in the follow-up special article, here is only a simple example to demonstrate how to do a image preheating:

apiVersion: apps.kruise.io/v1alpha1 kind: ImagePullJob metadata: name: job-nginx spec: image: Nginx :1.9.1 # [required] Complete mirror name:tag Parallelism: 10 # [optional] Maximum concurrent pull node sorting, default to 1 selector: # [optional] Specifies the list of node names or label selectors (only one of them can be set). If this parameter is not set, it indicates all nodes names: -node-1-node-2 matchLabels: node-type: XXX completionPolicy: Type: Always # [optional] Default to Always activeDeadlineSeconds: 1200 # [optional] No default value, Alway only ttlSecondsAfterFinished: 300 # [optional] No default value, Alway only pullPolicy: # [optional] backoffLimit=3, timeoutSeconds=600 backoffLimit: 3 timeoutSeconds: 300Copy the code

ImagePullJob has two completionPolicy types:

  • AlwaysIndicates that the job is preheated at one time and ends regardless of success or failure
    • activeDeadlineSeconds: Indicates the end time of the job deadline
    • ttlSecondsAfterFinished: The job is automatically cleared and deleted when the time expires
  • NeverIndicates that the job is running for a long time and will repreheat the mirror on the matched node every day

IO/zh-CN /docs/…

3. New reconstruction of SidecarSet

SidecarSet is a controller used to manage sidecar containers. After the user creates a SidecarSet, Kruise can automatically inject the user-defined Sidecar container into the Pod that meets the specified conditions, and upgrade the injected Sidecar container in situ without affecting the operation of the business container.

In previous versions, SidecarSet had many limitations, for example, users could not declare that only a namespace took effect, and sidecar had a weak grayscale capability when upgrading in place. In V0.8.0, we completely refactored the SidecarSet Controller and Webhook, and added some more powerful policy fields to the CRD definition. Some examples:

  1. Spec. namespace: Specifies that only sidecar injections and upgrades for a specific namespace are managed
  2. Multiple injection strategies:
    1. PodInjectPolicy: Specifies whether sidecar containers are injected before or after the original Pod containers list
    2. ShareVolumePolicy: Shares a volume policy with the Pod original container
    3. TransferEnv: Which environment variables are shared from which containers in the original Pod
  3. Multiple in-place upgrade strategies:
    1. MaxUnavailable: Indicates the maximum number of unavailable items during the upgrade
    2. Partition: Number of retained versions (grayscale/batch release)
    3. Update only the sidecar in the Pod that matches the selector condition.
    4. Scatter: Scatter according to the label

IO/zh-CN /docs/…

4. New feature-gate mechanism

In the past, the CRD and controller/ Webhook switches in OpenKruise were mainly configured in the CUSTOM_RESOURCE_ENABLE environment variable, while other configurable switches were concentrated in the command line parameters. The first problem was that they were scattered. Secondly, some function switches associated with multiple CRDS are actually difficult to control with CRD switches.

Therefore, the new feature-gate mechanism has replaced the CUSTOM_RESOURCE_ENABLE environment variable to focus on the functional level.

In V0.8.0, PodWebhook and KruiseDaemon switches are provided. Kruise will not do webhook interception for pod creation after the former is turned off, but will also turn off SidecarSet function. Kruise-daemon components will not be deployed, but the image preheating function will also be disabled. In the later versions, we will gradually unify the previous switch parameters into feature-gate.

5. Other points of variation

The rest of the optimization:

  • CloneSet, Advanced StatefulSet partial logic optimization.
  • In addition to the official DockerHub added Ali Cloud hosting image, domestic users can choose to use Ali Cloud image source to install/upgrade Kruise.
  • The user-agent that calls Apiserver is refined to the controller.
  • New GetScale/UpdateScale methods for CRDS that support scale subresources in ClientSet.

conclusion

The new version of OpenKruise v0.8.0 is the first product in the Kubernetes community to offer open source image warming at scale. In the later versions of this year, we also plan to provide the ability to use image preheating to speed up application release, application security protection, Controller grayscale/sharding control and so on. We expect to release v1.0 in the year.

OpenKruise is a mature CNCF sandbox project. In addition to its large-scale application in Alibaba, it also has extensive user cases in the industry:

  • Based on the demand of in-place upgrade and grayscale publishing, ctrip used CloneSet and AdvancedStatefulSet in production environment to manage stateless and stateful applications respectively, and the number of Kruise workload in a single cluster reached ten thousand levels.
  • OPPO not only uses OpenKruise on a large scale, but also further enhances in-place upgrade downstream with its customized Kubernetes, which is widely used in back-end operation services of multiple businesses, covering 87% of upgrade deployment requirements through in-place update.
  • In addition, domestic users include Douyu TV, Youzan, Suning, Bixin, Boss Zhipin, Shentong, Xiaohongshu, Spark Thinking, VIPKID, Master Education, Hangyin Consumption, Wanyi Technology, Duoduo Dmall, Zojiang Technology, Enjoy wisdom, Aijia Life, Yonghui Science and Technology Center, Follow who to learn, Deepexi, Foreign users include Lyft, Bringg, Arkane Systems, Spectro Cloud and others.

We welcome every cloud native lover to participate in the construction of OpenKruise and jointly create the industry’s top cloud native application automation engine!

Final open source news

Recently, Data Accelerator for Disaggregated Infrastructure (DADI) opened source! This is a container mirror accelerator project that has already been used in large-scale scenarios within Alibaba.

Its main principle is to eliminate the downloading and decompressing process of image, and to realize the on-demand pulling of fine-grained data blocks instead. This eliminates most of the data download effort, and hides the computation delay and data transfer delay from each other, resulting in a significant reduction in startup latency.

DADI combines the layered feature of container image with the block device interface of virtual machine image to form a new layered block device image, called Overlaybd. Thanks to the block device interface, DADI can support native file systems such as ext4, XFS, and even NTFS. The block device interface can also naturally support virtualized security containers and expose only minimal Attack surfaces. In addition, due to the simple and efficient nature of the block device Image, Overlaybd can provide users with better I/O performance.

If you are interested, please refer to the paper for a look at Github project:

  • The paper addresses

  • Making the address

If you are interested in the OpenKruise project and have any topics you would like to talk about, please visit the OpenKruise website, GitHub, and the Tidbit search group number: 23330762 to join the exchange group!