, the author | fanux. The implement sealos author, pouch maintainer, founder of the sealer

The address of the project: https://github.com/alibaba/se…

What is a cluster mirror

As the name suggests, similar to operating system.iso or Docker images, cluster images areWith a certain technical means to the entire cluster of all files in a certain format into a resource package.



Some interesting things can be found when comparing a single machine with a cluster:

  • Single machines have drivers such as compute storage networks, and clusters have implementations of CNI/CSI/CRI that act as clusters’ drivers
  • With stand-alone operating systems like Ubuntu CentOS, we can think of Kubernetes as a cloud operating system
  • Docker containers or virtual machines can be run on a single machine, which is equivalent to a running instance. Clusters also have instances running K8S
  • The virtual machine image on a single machine, Docker image, so with the development of cloud computing technology, similar mirroring technology will be abstracts in the latitude of cluster.

Take the Kubernetes-based cluster image, which contains all the files except the operating system:

  • Docker relies on binary with systemd configuration, dockerd configuration, and a private container mirror repository
  • Kubernetes core component binary, container image, Kubelet system configuration, etc
  • The YAML configuration or Helm Chart needed for the application, and the container image for the application
  • Other scripts, configuration and binaries, all the dependencies needed for the application to run

Similarly, a cluster image cannot be run on a single container or on a single machine, but the image can be installed directly on multiple servers or connected directly to the infrastructure of the public cloud.

Sealer is introduced

Sealer is alibaba open source cluster mirror mode of an implementation, project address: https://github.com/alibaba/se… To implement clustering. Docker solves the mirroring problem for a single container, while Sealer implements Build Share Run for distributed software by packaging the entire cluster!!

Imagine that we’re going to deliver a SaaS application that relies on databases like MySQL, ES, Redis, and middleware, and everything is orchestrated on Kubernetes. If we don’t have a cluster image, we need to do the following:

  1. Find a tool to install the K8S cluster
  2. helm install mysql es redis… You may also need to import container images if you are in an offline environment
  3. kubectl apply yoursaas

It may not seem that complicated, but in fact it is process-oriented from an overall project delivery perspective

Now if there were another way to do it with just one command would you use it?

sealer run your-saas-application-with-mysql-redis-es:latest

You can see that the entire cluster is delivered as a whole by running a single cluster image, the detailed and complex operations are shielded, and any application can run in the same way. So where does this cluster image come from:



All we need to do is define a dockerfile-like file called Kubefile and execute the build command:

sealer build -t your-saas-application-with-mysql-redis-es:latest .

A comparison between the latitude of the single machine and the cluster can be clear:



Docker can build a Docker image from a Dockerfile, and use the compose to run the container.

Sealer builds a CloudImage from KubeFile and starts the entire cluster using ClusterFile.

Rapid experience

Make and run a cluster image of Kubernetes Dashboard to experience a complete process. Write Kubefile

# base image, Has been made ready for it contains all the kubernetes launch related dependence FROM registry.cn-qingdao.aliyuncs.com/sealer-io/cloudrootfs:v1.16.9-alpha.7 # Download the official Dashboard YAML choreography file, Have downloaded you can use the COPY command RUN wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml # To specify the mode of operation, CMD Kubectl Employ-F Recommended. YAML can be used, such as Kubectl Helm Kustomiz

Sealer Build-t Kubernetes-with-Dashobard: Latest

Running the cluster image

# Install the K8S cluster on the server and apply Dashboard Passwd specified server SSH password, you can also use the key sealer run kubernetes - with - dashobard: latest \ - master 192.168.0.2, 192.168.0.3, 192.168.0.4 \ - node 192.168.0.5 192.168.0.6 \ - passwd XXX # check pod kubectl get pod - A | grep dashboard

​

Push the produced image to the image repository, compatible with Docker Registry

sealer tag kubernetes-with-dashobard:latest docker.io/fanux/dashobard:latest
sealer push docker.io/fanux/dashobard:latest

This allows the image to be delivered or reused by others.

Usage scenarios

What can Sealer do for us? Here are the main scenarios:

| install kubernetes and cluster life cycle management, upgrade/backup/recovery/telescopic) this is the simplest scenario, whether you need to install a development or test environment in the stand-alone installation of a high availability cluster in a production environment, both bare and docking public cloud, or a variety of operating system architecture, Both can be installed using Sealer. Just install Kubernetes and select a base image. Compared to other installation tools, Sealer has the following advantages:

  1. Sealer runs one command at the end
  2. Stunningly fast, install 6 nodes in 3min. You may have downloaded Sealer before using other tools. However, we have further technology optimizes to 2min or even 1min
  3. Compatibility and stability Compatibility with various operating systems, support x86 ARM and other architectures
  4. Consistent design allows the cluster to remain in the state defined in the Clusterfile. For example, upgrading can be achieved by changing the version number in the Clusterfile.

​

Faster because the first is golang implementation, means that we can do many very meticulous place concurrent processing, compared the ansible has more advantages, but also can do a more detailed error handling, and abandoned on the mirror to distribute the load before way, follow-up on file distribution will do optimize for installation on the performance of acme.

In terms of compatibility, Docker Kubelet adopts binary +systemd installation core components to be fully containerized, so that it does not have to rely on installation tools such as Yum APT that are aware of operating system. ARM and x86 use different mirroring support to decouple from Sealer itself. Adapting to the public cloud is also implemented from a separate module. The reason why we did not connect with TerraForm here is for performance. In our scenario, TerraForm started the infrastructure for nearly 3 minutes, but we optimized the infrastructure startup to within 30 seconds by backdating and retrying. Also, in the cluster mirroring scenario, such complex infrastructure management capabilities are not required, we do not want Sealer to be heavy and do not want to rely on a command line tool.

The design concept of consistency is worth mentioning in Sealer. Cluster image and Clusterfile determine what the cluster looks like. The same image and Clusterfile can run the same cluster. Change the Clusterfile either by adding more nodes, changing the node specifications, or by changing the image. When changing the image, because the cluster image is also a hierarchical structure, the layer with the same hash value will not be changed, and the change of the hash will help to reapply the layer.

| cloud native ecological software package/installation such as Prometheus mysql cluster sealer run Prometheus: latest can create a cluster with Prometheus, or install the Prometheus in an existing cluster. The question is: what is the difference between Helm and Helm?

  1. Sealer does not care about the choreography, but more about the packaging. For the example above, Prometheus can be choreographed by Helm. Sealer will package the chart and all container images needed in the chart, which is done with hack technology during build. Because the build process will set up a temporary Kubernetes cluster like Docker Build, then we will know which container images the cluster depends on, and then we will package those container images.
  2. Pack it with Kubernetes, take a chart, it may not be able to install successfully, for example, use the abandoned API version, but make a mirror to package Kubnernetes, as long as the build is OK, run is OK. This is similar to Docker packaging operating system rootfs together.
  3. Integration, cluster mirroring is more concerned with the whole package of distributed application clusters, such as the Prometheus Elk MySQL Cluster into a mirror service and business.

Therefore, Sealer and Helm have a cooperative relationship with a clear division of labor. These common cluster images can then be found in Sealer’s official mirror repository and used directly.

Proprietary cloud offline | SaaS software overall packaging/delivery delivery From the perspective of the distributed application, usually from the top down, less is more than a few hundreds of components, most of the existing overall delivery is process oriented, need a lot of things that intervention among sealer can wrap up all these things together on a key delivery.

You may ask, can’t I make a tar.gz and add an Ansible script to be keykey just the same? That’s for sure. Just like the delivery of tar.gz before Docker mirroring, you will find that standards and technologies are emerging to solve the problem of collaboration between people. They can also make good things for others to use.

A proprietary cloud scenario is a good fit for Sealer, many client rooms are offline and cluster mirroring will mirror all dependencies. As long as mirroring is done well, all game points can be delivered in the same way with one click, providing a very consistent experience.

| practice the above scenario sealer on public clouds bring docking public cloud properties, in many cases, the docking public clouds can have a better experience, such as installing a cluster only need to specify the server without having to care about the IP number and specification, telescopic directly modify Clusterfile defined in number.

Introduction to technical principles

| when writing copy cluster image storage is also, by means of written copy when doing so has two advantages, we can make a cluster of different distributed software in different layers, in order to realize the reuse, also can achieve the mirror cluster push to the docker image directly to the warehouse.

| container image cache build sealer is in the process of how to know what container to build clusters in the mirror image, and how to store the container mirror down, there are some difficulties in this question:

  1. How to know which container images there are in distributed software, because we need to cache these images, whether scanning user’s YAML file or using Helm template after scanning is not perfect, first of all, we can not determine the user’s orcharading method, second, some software even do not write the mirror address in the orcharading file. I’m going to pull it up through my own program. There is no guarantee that the build will run successfully.
  2. The container image needs to be stored in the private repository and packaged in the cluster image. The container image repository address is bound to be different from that written in the marshalling file, especially how to ensure that the user can still download the image in the private repository when alwayPull.

For the first problem, Sealer solves the problem by setting up a temporary Kubernetes cluster during Sealer build, just as with Docker build, and implementing the apply directive defined by the user in Kubefile.



This ensures that all images that the user depends on are packaged, regardless of the orchestration the user uses.

The second problem is that we package the container image into the private image repository. How to use the private image is also a problem, because if the private image repository is named localhost:5000, it will certainly be different from what is written in the orcharding file. There are two ways to solve this problem, the first is hack and docker. Do a as long as there is a private image warehouse directly from the private image to pull, not to the public net to pull the ability of the image.



Another solution is to call all Docker requests to the proxy without invading Docker’s proxy, and let the proxy decide to pull from the private warehouse if there is one. We have also enhanced Registry’s ability to cache multiple remote repositories.

Sealer’s solution perfectly solves the problem of mirror packaging for offline scenarios.

​

| load balancing

Sealer’s cluster high available uses the lightweight load-balancer LVSCare. First of all, LVSCare is a few hundred lines smaller than other load-balancer LVSCare, and LVSCare only acts as the guardian of IPVS rules. LVSCare does not do it itself and is very stable in load, directly monitoring APIServer on node. If you kneel down, you remove the corresponding rules and automatically add them back up. This is equivalent to a dedicated load balancer, which has been used for more than two years in SEALOS project, with extensive practice.



| runtime

The runtime is the environment that supports the application to run. For example, the runtime Sealer based on Kuberentes can be transparently supported. In the case of istio, the user only needs to:

The FROM kubernetes: v1.18.3 RUN curl - L | sh - https://istio.io/downloadIstio

You can build an Istio runtime for your application to use.

For runtimes that do not base on Kuberentes, such as k0S k3s, it is possible to extend the interface in Sealer.Runtime so that later:

The FROM k3s: v1.18.3 RUN curl - L | sh - https://istio.io/downloadIstio

Better extensions such as extended ACK Runtime

The FROM aliyum.com/ACK:v1.16.9 RUN curl - L | sh - https://istio.io/downloadIstio

This mirroring directly helps the user’s application run on the ACK. Some of these capabilities are under roadmap

| infrastructure Now many users want mirror in the cloud to run their own cluster, dock sealer bringing public clouds, sealer own realization infrastructure manager, thanks to our more sophisticated retreat retry mechanism, 30 s to complete infrastructure building (ali cloud 6 nodes) performance is among the same kind of tools, The number of API calls was greatly reduced, and the configuration was compatible with ClusterFile.

conclusion

Some future visions and value withdrawals for Sealer:

  • Sealer solves the problem of collaboration between producers and users of distributed software by enabling user-defined clusters in an extremely simple way.
  • The User Interface is very simple and friendly, which can shield and be compatible with various low-level technical details and run everywhere
  • Ecological construction, the official repository will contain commonly used distributed software

​

Finally, let’s summarize:

  • Use Sealer if you want to deliver your distributed SaaS as a whole
  • Use Sealer if you want to integrate multiple distributed services together, such as database message queues or microservice runtimes
  • Use Sealer if you want to install a distributed application such as the MySQL primary and secondary cluster
  • If you need to install/manage a Kubernetes high availability cluster, use Sealer
  • Use Sealer if you want to initialize multiple datacenter and keep the datacenter state strongly consistent
  • If you need to implement the above scenario on the public cloud, use Sealer Kubernetes to install with one click