The author does not aversion |
Introduction: The two lines of user demand and cloud development have promoted the rise, development and large-scale application of cloud native technology. This paper will mainly discuss what is cloud native application, what are the elements of cloud native application, what is Serverless computing, and how Serverless simplifies the technical complexity, helps users cope with rapidly changing needs, and realizes elastic and highly available services, and explains through specific cases and scenarios. Nowadays, all industries are talking about digital transformation, especially new retail, media, transportation and so on. Digital business form has become the mainstream, gradually replacing the traditional business form. In some other industries (such as industrial manufacturing), although the business form of enterprises is not in the form of digital performance, but under the concept of digital twin, the full use of data technology for production operation optimization is becoming a research hotspot and industry consensus.
Digital transformation of enterprises can be seen from the following four aspects: means of production, relations of production, strategic planning and growth curve:
-
Means of production: data become the most important means of production, demand/risk changes at any time, enterprises face huge uncertainty;
-
Production relations: data-centered, fixed production relations not based on processes and rules. Network effects enable production relations to transcend temporal and spatial constraints, and multiple connected ways spawn new businesses and species.
-
Strategic planning: rapid response to uncertain business environment based on data decisions;
-
Growth curve: The ability to reach a mass audience through digital technology can lead to breakthrough growth.
From the perspective of Cloud service providers, the evolution trend of Cloud. In the era of Cloud 1.0, the Cloud of infrastructure is its theme. Cloud hosting mode is adopted to maintain compatibility of applications on and off the Cloud, and traditional applications can be directly migrated to the Cloud. After massive computing power is provided by infrastructure, how to help users make better use of computing power and accelerate the speed of enterprise innovation has become the core capability of cloud.
If you still build the base application on the server, it becomes expensive to develop and difficult to manage, hence Cloud 2.0, the Cloud native era. In the era of cloud native, cloud service providers provide rich hosting services to facilitate digital transformation and innovation of enterprises. Users can build applications based on various cloud services just like building blocks, greatly reducing r&d costs.
Cloud native application elements
Cloud native applications have three key elements: microservices architecture, application containerization and Serverless, and agile software delivery processes.
1. Microservices architecture
Individual architecture and microservice architecture each have their own characteristics, and the main characteristics are shown in the following figure. In general, the single architecture is fast, but difficult to maintain, while the microservice architecture is difficult to deploy, but more independent and agile, and more suitable for cloud native applications.
▲ Individual architecture vs. microservice architecture
2. Application containerization and Serverless
Container is the most popular code packaging method at present. With the help of K8s and its ecological capabilities, it greatly reduces the management difficulty of the entire infrastructure. Moreover, container provides excellent flexibility and portability in the support of programs. Serverless computing is another kind of form, do a lot of end-to-end integration and cloud service integration, greatly improve the efficiency of research and development, but not as flexible container compatibility on the application of the traditional, but also brought a lot of neat, users only need to focus on the business logic code, focusing on the innovation of the business logic.
3. Agile application delivery process
An agile application delivery process is a very important element, including process automation, focus on feature development, quick problem finding, and quick launch.
Serverless calculation
1. Ali Cloud function calculation
Serverless is a new concept, but its connotation is already there. The first cloud service of Ali Cloud or AWS is object storage. Object storage is actually a Serverless service in the storage field. Also, Serverless refers to a system of products, not a single product. At present, the vast majority of new functions or products launched by cloud service providers in the industry are Serverless. Ali Cloud Serverless product system includes computing, storage, API, analysis and middleware, etc. Currently, the cloud product system is being Serverless.
Ali Cloud Serverless computing platform function computing, has four characteristics:
-
Seamless integration with the cloud: seamless integration of various cloud services and function computing through event-driven way, users only need to pay attention to the development of function, event trigger is completed by the service provider;
-
Real-time elastic scaling: elastic scaling calculated by the automatic completion function of the system is very fast. Users can use this capability in online applications.
-
Sub-second metering: Sub-second metering provides a complete on-demand metering method, with 100% resource utilization;
-
Highly available: Functional computing platforms do a lot to help users build highly available applications.
So, ali cloud function calculation is how to do the above 4 points? The product capacity of Aliyun function computing is shown in the figure below. Firstly, the function computing product is built on Alibaba’s infrastructure services, and the computing layer on it has been greatly optimized. Then in the application layer developed a large number of capabilities and tools, based on the above product capabilities, to provide users with a variety of scenarios under the complete solution, the entire excellent functional computing products. Function computing is a very basic cloud product of Ali Cloud, and many products and functions of Ali Cloud are built on the basis of function computing. At present, Aliyun function computing has provided services in 19 regions around the world.
▲ Ali Cloud function computing product capacity diagram
2. Serverless helps users simplify the ha design and implementation of cloud native applications
The high availability of cloud native applications is a systematic project, including many aspects, and the construction of a complete high availability system needs a lot of time and energy. How does Serverless Computing help users simplify the design and implementation of cloud native high availability applications? As shown in the figure below, the high availability architecture includes the infrastructure layer, the runtime layer, the data layer and the application layer, and each layer has a lot of work to do to achieve high availability. Function calculation mainly does a lot of work to achieve high availability from four aspects of fault tolerance, elasticity, flow control and monitoring. The functions corresponding to the blue dotted box in the figure below are implemented by the platform, so users do not need to consider. Blue solid wire boxes although the platform has done some work to simplify the work of users, they still need users’ attention, while orange solid wire boxes represent some functions that users are responsible for. Combined with the functions provided by the platform and part of the user’s energy input, users can greatly reduce the difficulty of building a high availability system.
▲ Function calculation high availability
Function computing is optimized in many ways to help users build high availability systems. The following figure shows the power of function calculation in availability zone disaster recovery. As can be seen from the figure, load balancing is performed in the function calculation, which greatly improves the DISASTER recovery capability.
The following figure shows the asynchronous processing of events by function calculation. The processing pipeline mainly includes three links: event queue, event distribution and event consumption, and each link can be scaled horizontally. One of the key points is that event distribution needs to match the consumption capacity of downstream. In addition, by specifying different amounts of computing resources for different functions, users can easily dynamically adjust the consumption rate of different types of events. In addition, error retry logic can be customized, and there is back pressure feedback and flow control that does not overwhelm the next service with a large number of requests in a short period of time.
In addition to the observability of function calculation, the elog provides log collection and query functions. In addition to the default simple log query function, the elog also provides advanced log query functions to facilitate log analysis. In terms of indicator collection and visualization, functional computing provides rich indicator collection capabilities, standard indicators, and overview information, facilitating user o&M. The following figure is a schematic diagram of application delivery. In the whole application delivery process, only when every link is well done, can an agile application delivery process be built. Automation is the core, and only when automation is achieved, can the efficiency and agility of the whole assembly line be improved.
The following figure shows the specific tasks of the automated application delivery pipeline at each step. It is important to pay attention to the infrastructure is code, template definition and automatic setting of the application running environment, so as to achieve automatic continuous integration.
▲ Automated application delivery pipeline
Once the application is delivered automatically, the overall development efficiency is greatly improved. In the Serverless application, AliYun provides a variety of tools to help users implement infrastructure as code. Serverless’s model has the nice ability to pass in different parameters to the same template to generate definitions of different environments that can then be managed automatically.
For the delivery and grayscale release of different service versions of the application itself, function computing provides the service version and service alias to provide corresponding services. The grayscale release process of the entire application can be simplified into some API operations, greatly improving the efficiency of business. With these capabilities provided by the Serverless computing platform, the automation of the entire software application delivery pipeline has been greatly improved.
Another useful feature of function calculation is compatibility with stock applications. The Custom Runtime is compatible with many popular frameworks and traditional applications, so that it can be easily adapted to the Serverless platform. The console provides a series of services such as application creation, deployment, associated resource management, and monitoring.
In addition to function calculation, Serverless workflow can also be used to orchestrate different application links and functions. Workflow can be defined in a descriptive language to reliably execute each step, which greatly reduces the difficulty of orchestrating complex tasks for users.
Application Scenarios
There are several typical application scenarios of function computing, one of which is Web/API back-end service. Ali Cloud has had several successful application cases including Graphite Document, Weibo and Century Hualian.
Another application scenario of function calculation is large-scale data parallel processing, such as uploading a large number of images, audio, text and other data to OSS, which can trigger functions to perform custom processing, such as transcoding, frame cutting, etc. Successful cases in this regard include Hupu, Focus media, Baijia Internet and so on.
Another application scenario of functional computing is real-time streaming of data. For example, messages generated by different devices, logs sent to message queues and other services with similar pipes can trigger functions for streaming.
The last application scenario is the automation of operation and maintenance. Functions are called to complete operation and maintenance tasks by means of timing trigger, cloud monitoring event trigger and process choreography, which greatly reduces the cost and difficulty of operation and maintenance. Typical successful cases include Tucson Future.
Tucson Future is an ai enterprise focusing on r&d and application of L4 level driverless truck technology, providing driverless truck technology for large-scale commercial operation around the world, empowering the global logistics and transportation industry. In the process of road testing, a large amount of data will be generated, and the processing process of these data is complex and changeable. Even for the same batch of data, different business groups will have different use and processing methods. How to effectively manage different data processing processes and reduce the frequency of human intervention can greatly improve productivity.
The characteristic of unscheduled operation of road test makes the running time and running time of process choreography task have great uncertainty. It is difficult to optimize the machine utilization rate and waste resources to establish the process management system in the local computer room alone. There are already many unitary business processing scripts and applications in Tucson in the future, but due to various limitations, it is not possible to migrate to the cloud in full, which also poses a challenge to how to rationalize the use of cloud services.
In view of the above situation, Tucson began to explore the automation of data processing platform in the future. Ali Cloud Serverless workflow is charged according to The Times of scheduling. It has many advantages such as ease of use, easy integration, simple operation and maintenance, and can solve the problems encountered in the above scenarios. It is very suitable for such off-line task scenarios with irregular operation.
Serverless workflow also supports scheduling local or self-built machine room tasks. Tucson will use the Serverless Workflow natively supported message service MNS to solve the problem of data access between cloud and cloud, so that local original tasks can be well organized and managed.
In addition to scheduling, The Serverless workflow also supports maintenance of the state of tasks and the data generated during their execution. Tucson future efficiently manages the life cycle of tasks in the process and the transfer of data to each other by using input-output mapping and status reporting mechanisms for tasks.
In the future, with the expansion of business scale, Tucson will continue to optimize the operational efficiency and automation level of offline big data processing processes. Through various explorations, Tucson will further improve the efficiency of its engineering team in the future, and devote more energy and capital to business innovation.
conclusion
Serverless workflow is a key part of Ali Cloud Serverless product system. Through Serverless workflow, users can arrange multiple Ali-cloud services, such as functional computing and visual intelligence platform, or self-built services into workflows in a simple and intuitive way, and quickly build flexible and highly available cloud native applications.
Since the launch of the function calculation in 2017, the service can flexibly expand and shrink in real time according to the change of application load. It can scale tens of thousands of instances in one minute and guarantee stable delay. At present, it has supported the key applications of users such as Weibo, Mango TV, BGI, Tucson Future, graphite Technology and so on, easily coping with business peaks.