The author | mysteries (d), good function of cloud computing development engineer ali

Takeaway: Serverless Kubernetes is a container and Kubernetes-based Serverless service. It provides an easy-to-use, extremely flexible, optimal cost and pay-as-needed Kubernetes container service. It requires no node management, operation and maintenance, and no capacity planning. Let users focus more on application management than infrastructure management. We can call Serverless Kubernetes ASK for short.

Serverless container

Let’s start with Serverless. I believe we are familiar with the core value of Serverless concept, including no need to manage the underlying infrastructure, no need to care about the upgrade and maintenance of the underlying OS, because Serverless allows us to pay more attention to the application development itself. So the app takes less time to go live. At the same time, the Serverless architecture is naturally extensible. When the number of service users or resource consumption increases, we only need to create more application resources, and the scalability behind it is incomparable to buying machines by users themselves. Serverless applications are generally created on demand and users do not have to pay for idle resources, reducing the overall computing cost.

These are the core values of the Serverless concept, and what makes the Serverless container similar to other Sererless forms. However, the Serverless container differs from other Serverless forms in that it is based on the container delivery form.

Based on the container means universality and standards, we can Build once and Run anywhere, container is not limited by language and library, no matter any application can be made into container image, and then launched in the container deployment mode. Based on the standardization of containers, the open source community has built a rich Cloud Native ecosystem centered on Kubernetes, which has greatly enriched the surrounding application frameworks and tools of Serverless container. For example, it is very convenient to deploy Helm Chart package. Based on containers and Kubernetes standardization, we can easily migrate applications between different environments (online and offline) and even between cloud vendors without worrying about vendor lock-in. These are the core values of the Serverless container.

▲ Serverless container product Landscape

At present, all major cloud manufacturers have launched their own Serverless container services. The picture above shows the Serverless container product Landscape organized by Gartner evaluation Agency, among which Ali Cloud has Serverless Kubernetes ASK and ECI. AWS has Fargate, based on Fargate, there are two forms: EKS on Fargate and ECS on Fargate. Azure with ACI. Gartner also predicts that 70% of AI applications will run as containers and Serverless by 2023.

ASK/ACK on ECI container services

The following is the introduction of aliyun Serverless container product family: ECI, ACK on ECI and Serverless Kubernetes.

1. ECI

ECI, an Elastic Container Instance, is the underlying infrastructure of Serverless containers, enabling the startup of Container images. ECI makes containers first-class cloud citizens like ECS. The underlying operating environment of ECI is strongly isolated based on security container technology. Each ECI has an independent operating environment to ensure runtime security. ECI supports 0.25C to 64C CPU specifications, and also supports Gpus. On-demand creation is charged per second. Like ECS, ECI supports preemptive instances of Spot, which can save up to 90% in some scenarios. ECI instances currently start up around 10s and then start pulling container images. We also provide the mirror snapshot function, which reads the image from the snapshot every time the container is started, saving the time for the remote pull. It is worth emphasizing that ECI and ECS share the same elastic computing resource pool, which means that the elastic supply capacity of ECI can be fully guaranteed to the maximum extent, so that ECI users can enjoy the scale dividend of elastic computing resource pool.

ECI can only create a single container instance, but it does not have the ability to orchestrate. For example, it allows multiple copies of applications to expand, SLB and Ingress to access Pod traffic, so we need to use ECI in Kubernetes. We provide two ways to use ECI in Kubernetes. ACK on ECI (ACK on ECI)

In the integration with the Kubernetes choreography system, we manage each ECI container instance as a Pod, with each Pod corresponding to one ECI instance, isolated from each other, and each ECI Pod startup time of about 10 seconds. Because ECI Pod is managed in Kubernetes cluster, it is fully connected to Kubernetes ecosystem, which has the following aspects:

• It is easy to use Kubectl to manage ECI Pod. You can use standard Kubernetes API to operate resources.

• Connect SLB and ECI Pod via Service and Ingress;

• Deployment/Statefulset is used for container orchestration and HPA is used for dynamic expansion.

• You can use Proms to monitor ECI pods;

• Run Istio for flow management, Spark/Presto for data calculation, and Kubeflow for machine learning.

• Deploy various Helm Charts.

These are the values of using Kubernetes to manage container instances.

It should be noted that ECI Pod in Kubernetes is Serverless container, so compared with ordinary Pod, it does not support some functions (such as Daemonset), Prividge permissions, HostPort and so on. In addition, ECI Pod has the same capabilities as ordinary PODS, such as mounting cloud disks, NAS, and OSS data volumes.

2. ACK on ECI

Let’s take a look at using ECI in an ACK Kubernetes cluster. This method is suitable for users who have an ACK cluster and many ECS nodes in the cluster. In this case, users can Run some short-run applications based on the elastic capability of ECI to solve the problem of insufficient resources in the meta-cluster, or use ECI to support rapid application expansion. ECI capacity expansion is more efficient than ECS capacity expansion.

In ACK on ECI, ECS and ECI Pod can communicate with each other, and ECI Pod can access Coredns in the cluster as well as ClusterIP Service.

3. Serverless Kubernetes

Different from ACK on ECI, there is no ECS node in ASK Serverless Kubernetes cluster, which is the main difference with traditional Kubernetes cluster, so there is no need to manage any nodes in ASK cluster, realizing a complete noden-free operation and maintenance environment. Is a pure Serverless environment, it makes the use of Kubernetes threshold greatly reduced, but also discarded the tedious bottom node operation and maintenance work, not to encounter node Notready and other problems. In an ASK cluster, users focus only on the application itself, not the underlying infrastructure management.

The flexibility of ASK is better than that of ordinary Kubernetes cluster. Currently, it is 30 seconds to create 500 pods to Running state. ECI Pods in clusters are charged by volume by default, but Spot and reserved instance coupons are also supported to reduce costs. In terms of compatibility, no real nodes exist in ASK, so node-related functions such as Daemonset are not supported, while Deployment/Statefulset/Job/Service/Ingress/CRD are all seamlessly supported.

The default Ingress in ASK is based on SLB Layer 7 forwarding implementation, users do not need to deploy Nginx Ingress, easier maintenance.

Meanwhile, Knative Serving capability is realized based on SLB layer 7, in which Knative Controller is hosted by ASK and the user does not need to bear the cost of Controller.

As with ACK, cloud products such as ASK and Arms/SLS are well integrated, making it easy to monitor pods and collect Pod logs into SLS.

This is the overall architecture of ASK. The core part is Ask-Schduler, which is responsible for the change of Watch Pod, and then creates the corresponding ECI instance, and synchronizates the ECI instance state to Pod. No real ECS nodes in the cluster are registered with Apiserver. This Nodeless architecture decouples the Kubernetes choreography layer and ECI resource layer, so that Kubernetes completely gets rid of the elasticity and capacity limitation caused by the scale of the underlying node, and becomes the cloud-oriented Nodeless Kubernetes elastic architecture.

Typical functions of ASK

Here are some typical features of ASK:

1. The GPU instance

The first is a GPU instance. Using a GPU container instance in a Serverless cluster is very simple. There is no need to install a GPU driver, just specify the GPU Pod specification and the number of GPU cards required by the container, and then you can deploy it with one click. This can greatly improve the efficiency of development and testing for machine learning scenarios.

2. Spot preemption example

The second is an example of Spot preemption. Preemptive instances are on-demand instances that can reduce computation costs in scenarios such as data computation. Preemptive instances have a one-hour protection period after successful creation. The market price of a preemptive instance fluctuates with supply and demand. We support two Spot strategies: one is based entirely on the market price, and the other is to specify a price cap. We just need to annotate the Pod.

3. Elastic Workload

The third important feature is the Elastic Workload, which enables Deployment of multiple copies on different cells, such as ECS, ECI, and ECI-SPOT. This mode of mixed scheduling can reduce the computational cost of loads. In this example, Deployment is six copies, two of which are normal ECI pods and the other copies are ECI-SPOT instances.

ASK Usage Scenarios

We have covered the basic product and functionality of Serverless Kubernetes above, so in what scenarios is ASK suitable for use?

1. O&m free application hosting

The biggest characteristic of Serverless cluster is to solve the operation and maintenance problem of the underlying node resources, so it is very suitable for the application free operation and maintenance hosting, so that users focus on the application development itself. Applications in a traditional K8s cluster can be seamlessly deployed in a Serverless cluster, including various Helm Charts. At the same time, the long computing cost of Pod can be reduced by using reserved instance coupons.

2. ECI elastic resource pool

The second scenario is the advantage of ACK on ECI. We can choose ECI as an elastic resource pool and add it to the existing Kubernetes cluster. When application business peak comes, ECI can dynamically and flexibly expand capacity, which is more efficient than ECS node expansion. This is suitable for business scenarios with obvious peaks and valleys, such as e-commerce or online education. Users do not need to manage a large node resource pool, and the overall computing cost can be reduced through ECI flexibility.

3. Big data computing

The third scenario is big data computing. Many users use Serverless clusters or ACK on ECI for Spark, Presto, AI and other data computing or machine learning. ECI can easily solve the problem of resource planning and insufficiency.

4. CI/CD continuous integration

The fourth scenario is CI/CD continuous integration. Jenkins and Gitlab-Runner are connected to ASK cluster, and CI/CD construction tasks are created as required. After completion of construction, they are directly deployed to ASK test environment for verification, so we do not need to maintain a fixed resource pool for Job tasks. Creating on demand greatly reduces costs, and can be further reduced when combined with Spot instances.

This is a typical Serverless Kubernetes cluster scenario, along with quick usage links, product documentation, and usage examples for you to learn and exchange.