preface
Microservices are an essential technology in large distributed IT architectures. In essence, microservices are an architectural style that breaks up a large system into multiple applications with independent life cycles that communicate with each other using lightweight communication mechanisms. These applications are built around a specific business and can be deployed independently, iterated independently, or scaled horizontally independently depending on the business load.
The idea of microservices and related technologies has brought a series of profound changes to the development of IT architecture:
** Easy to develop and maintain: ** An application focuses on a specific set of business functions. Service separation reduces coupling between applications, making development and maintenance easier.
** Unlimited technology stack: ** In the microservice architecture, the technology stack can be reasonably selected based on the characteristics of the project business and the team.
** Speed up the evolution of the system: ** Each application can be updated independently, through grayscale release and other technical means to ensure the stable operation of the whole system in the release process.
** Breaks performance bottlenecks: ** Each application can scale independently and horizontally, so that system performance can scale linearly as computing resources increase.
The challenge of microservices
There is no such thing as a free lunch. While microservices technology makes IT systems more agile, robust and high-performance, IT also brings about an increase in architectural complexity. For developers, a series of challenges such as continuous integration, service discovery, application communication, configuration management, and traffic protection need to be solved in order to better harness microservices architecture. Fortunately, a number of excellent open source technical components and tools have emerged to address these common challenges, making it easier for developers to build microservices applications. Technology frameworks like Spring Cloud and Dubbo have evolved over the years to become the standard for microservices, dramatically lowering the bar for microservices, but they still fail to address two of the biggest challenges that developers face.
Challenge 1: The need for a comprehensive lifecycle management and service governance solution
In a system with frequent iteration, each application is often faced with new version release requirements. It is necessary to manage the online, offline, update, rollback and other processes of the application intensively, and cooperate with fine-grained grayscale release means to reduce the impact of version iteration on the business.
In A simple microservice architecture, if an application is at the entry point of the entire link, A load balancing component (application A in the figure above) will be attached to its front end to accept business requests from end users. The complexity of life cycle management for such applications is higher. To ensure the balance and stability of applications during the release of new versions, the following steps are performed:
In this process, there is no advanced grayscale scheme for fine-grained control of flow, but it is sufficiently complex and difficult to operate. If you only rely on a simple release script for management, it is not only inefficient, but also easy to lead to the loss of one and the loss of the other, resulting in a huge risk to system stability.
Challenge 2: Need to improve horizontal capacity expansion and reduction solutions
When an application has a performance bottleneck and needs to increase the number of instances to improve performance, new computing resources need to be introduced.
Where will the new computing resources come from?
For offline IDCs, computing resources need to be planned in advance. Capacity expansion is not easy and may fail due to various constraints. Of course, this problem no longer exists in the era of cloud computing. It is easy to add computing resources to an application, but it is not enough to have computing resources. You have to deploy applications on it and accommodate them in microservices.
According to this process, it is conservatively estimated that it takes more than 20 minutes to expand an application instance, which takes a lot of time to purchase, initialize the system, and deploy applications. If the system has a sudden increase in traffic and needs to be expanded within 2 minutes, this solution is useless.
One good medicine: containerization
To address these two challenges, developers have tried a variety of solutions, and new ideas and technical frameworks have emerged over the past five years. Under the condition of survival of the fittest, container technology represented by Docker, supported by Kubernetes ecology, has become the mainstream in the industry and is an essential element for the construction of Cloud Native applications. Container-related technologies can help developers solve these two problems to some extent by enabling larger applications to exploit the value of cloud computing.
In terms of application life cycle management and service governance, Kubernetes provides a relatively perfect implementation mechanism. By constructing Deployment resources and cooperating with proStop and postStart scripts, rolling release and graceful online and offline of applications can be realized more conveniently. Although there is still no way to directly control the flow with fine granularity in the process of grayscale publishing (the introduction of Service Mesh technology can enhance the flow control, which is not discussed in this paper), it has improved by leaps and bounds compared with simple publishing scripts.
In terms of horizontal capacity expansion and reduction of applications, containerization technology can greatly reduce the time of OS installation and system-level initialization. However, vm purchase is unavoidable. Therefore, rapid horizontal capacity expansion cannot be achieved when the system encounters a traffic surge. You can reserve some computing resources and store them in the resource pool. When the application needs to expand capacity, you can apply for resources from the resource pool. When the service load decreases, you can return the redundant computing resources to the resource pool.
This is not a good idea. Every computing resource has a cost, and resource pools can solve the problem of rapid utilization of computing resources, but cause huge waste. In addition, it is difficult to plan a resource pool. The larger the pool, the more waste it will cause, but the smaller the pool may not be able to meet the demand of capacity expansion.
A deeper analysis of resource costs
Some developers may think that the current business operation is very stable, there is no obvious sudden increase in user traffic, so expansion and reduction of capacity is a pseudo demand, and there will be no such demand in the future. This may be a misunderstanding of the Internet business, because there is no need for expansion at all.
First, as long as a system serves people, there must be peaks and troughs. It is impossible for a 7*24 hour system to maintain the same user traffic forever, and the 80/20 principle still applies to many business systems (80% of user traffic is concentrated in 20% of the time). Even for a system with relatively balanced user traffic, there is a traffic trough in the early morning. If idle computing resources can be further released and resource utilization can be improved, resource usage costs can be significantly reduced.
In addition, development and test environments require more capacity expansion and reduction than production environments. A suite of microservices applications is developed by different teams, and ideally, multiple teams share a suite of test environments:
However, each team iterates at their own pace, and at the same time, they want to have separate end-to-end testing environments that are isolated from each other to avoid interaction between teams. In this case, it is possible to form multiple test environments:
As the number of applications, teams, and business function points increases, the number of development test environments required increases exponentially, resulting in a huge waste of resources. For computing resources in the test environment, resource utilization is much lower than that in the production environment. Sometimes it’s just a simple function point validation that opens up a whole new set of microservices applications in order to run end-to-end through business functions without interplay between teams. Such a waste of resources, for many enterprises, is a problem that has not been solved for many years.
Therefore, microservice architecture in essence has a strong appeal for elastic scaling, in the process of elastic scaling, whether the horizontal elastic scaling of a single application, or the start and stop of the whole environment, resource utilization plays a decisive role in the final resource cost. If we can find ways to improve resource utilization, we can save a lot of resource costs for enterprises. It is important to note that the vast majority of microservices applications have very low resource utilization.
We can do a simple statistics: export the CPU usage of all servers every five minutes and calculate the average value by day. Then we can know the resource usage data of the system as a whole. If the server resources used to develop the test environment were included in the statistics, resource utilization would most likely be even lower.
Serverless, explore
The root cause of low resource utilization is that in a server based application architecture, developers need to deploy built packages to the server to respond to multiple user events. To ensure the timeliness of event responses, you need to have programs live on the server and plan resources as conservatively as possible to avoid overloading the service and crashing it. During this process, the actual load is not evenly distributed over time, resulting in low resource utilization.
The emergence of Serverless technology provides a new idea for improving resource utilization. Serverless is a complete process for building and managing microservices-based architectures that allow developers to deploy applications directly away from server resources. It differs from traditional architectures in that it is completely managed by third parties, triggered by events, and exists in Stateless computing containers. Building a serverless application means developers can focus on production code instead of managing and manipulating server resources, making it truly infrastructure free to deploy applications.
Serverless technology has many forms, the most typical one is FaaS (Function as a Service), such as Ali Cloud Function Compute (FC) product. In functional computing, the application and scheduling of all computing resources are triggered by specific service events. When the tasks corresponding to service events are completed, computing resources are released immediately. This way really achieves the distribution of computing resources on demand, can significantly improve the utilization of resources, is the ultimate form of Serverless technology.
The other is Serverless container technology. Serverless container instances run in case isolation, and each compute node is completely isolated by lightweight virtual security sandbox technology. Users can directly deploy container applications without purchasing server resources, and do not need to perform node maintenance and capacity planning for the cluster. They can pay on demand based on the amount of CPU and memory resources configured for applications. When micro-service applications need to be expanded, they can quickly obtain computing resources without the need to purchase servers. This helps developers reduce computing costs, reduce idle resources waste, and smooth response to sudden traffic peaks. Serverless Kubernetes (ASK) of AliYun is the representative product of Serverless container technology.
Further explore the developer’s appeal
Serverless seamless technology is the development direction of cloud computing and cloud native application architecture, but for the developers of microservice applications, both FaaS form and Serverless Kubernetes have certain limitations.
Not every service is suitable for FaaS construction, especially for applications with long links and obvious upstream and downstream dependence, FaaS transformation is impossible. Even if the FaaS transformation of some business systems proves feasible, the transformation of the existing microservices architecture into FaaS architecture requires a certain amount of work and cannot be seamless migration.
The Serverless Kubernetes architecture is suitable for all business scenarios, but for developers, building a whole set of Kubernetes architecture requires mastering a series of complex concepts related to Kubernetes, which has a very steep learning curve. Moreover, the construction of various components in the Kubernetes ecosystem, as well as the adaptation of the network layer and storage layer, all involve very complex work.
The reason for this limitation is simple. In the microservice technology camp represented by Spring Cloud, the system is built around the application (also understood as a single service), whether it is version update or horizontal expansion, it is for the application itself. The core of the Serverless Kubernetes architecture is Pod, which is more oriented to the system layer than the application layer, so users need to invest more energy in the management of the application layer resources. However, the core of FaaS architecture is function, which is more inclined to the upper layer of the system than application. Therefore, the flexibility will be reduced and cannot be adapted to all business scenarios.
For developers building microservice applications using the mainstream Spring Cloud or Dubbo architecture, if they need to introduce a solution to reduce resource costs, their ultimate appeal must include two aspects:
(1) Whether the transformation cost is zero or close to zero; (2) Whether it ADAPTS to all business scenarios.
Application layer Serverless technology
Is there a technology in between FaaS and Serverless containers that can fulfill these important requirements? Of course, this is the application layer Serverless technology represented by Ali Cloud Serverless Application Engine (SAE).
SAE has realized the perfect integration of Serverless architecture and microservice architecture. For mainstream microservice architectures such as Spring Cloud and Dubbo, seamless compatibility can be achieved, basically no transformation cost, and truly on-demand, billing by volume, saving idle computing resources. At the same time, IaaS layer operation and maintenance work is eliminated, effectively improving the efficiency of development operation and maintenance.
Take the Spring Cloud application as an example. If you need to deploy a new application, there are only 2 steps:
(1) Tell SAE how many instances the application needs and specify the CPU/memory size required for each instance. (2) Upload the APPLICATION JAR package or WAR package and start the application.
We found that these two steps did not involve capacity assessment, server purchase, operating system installation, resource initialization, etc., and enabled a microservice application with multiple peer instances to run. This is because in the Serverless world, there is no longer a concept of server resources. Applications are sandbox containers that are dispatched by SAE, and each instance is billed by time of use only after it is actually used.
For developers, they don’t need to know whether the application is deployed on a physical machine, a virtual machine, or a container. They don’t need to know the underlying operating system version, just how many computing resources each application instance takes up. If the application needs to be expanded from 4 instances to 6 instances, or reduced to 2 instances, it can be completed with a single command, and even the binding relationship with SLB can be automatically established or removed. This is the great value that Serverless technology brings to developers.
Using SAE to deploy microservices applications is 100% compatible with existing technology architectures and business functions because it only changes the carrier on which the application is run, with negligible migration costs.
SAE’s ultimate resilience
In addition to manual scaling orders, SAE also supports two automatic resilience mechanisms that allow flexible horizontal scaling of microservices applications, further leveraging the resilience of cloud computing.
** Timing elasticity mechanism: ** You can set timing elasticity policies for expected periodic behavior. For example, if the service is at its peak at 9:00 am every day, you can increase the number of instances at 8:30 am and reduce the number of instances at 9:30 am.
** Elastic mechanism based on indicator thresholds: ** For unexpected service traffic surges, you can set an elastic policy based on indicator thresholds to automatically shrink applications based on resource indicators such as CPU and memory.
Multiple elastic mechanisms enable fine-grained system capacity management. In this way, the resource usage can be adjusted as service traffic changes, greatly increasing resource utilization and reducing resource costs.
SAE has made a number of optimizations in computing resource scheduling and startup. New instances can be pulled up in just a few seconds after capacity expansion. This capability is of great significance for some emergent scenarios that require rapid and urgent capacity expansion.
For the development and testing environment, SAE’s mechanical resilience is even more evident, thanks to SAE’s excellent resource scheduling capabilities, which can start and stop a full set of microservice applications with one click. Even if you smoke test a simple new feature, you can do it in a completely new, isolated test environment. New environments can be built in seconds, put into use quickly, and released immediately after testing. In terms of cost, a new environment can be put into use for a very short period of time, so it costs very little. This is a huge change for multi-team collaboration in microservices application development.
Cost analysis
SAE is charged by the actual usage of the resource. The fee consists of two parts. Each part is settled according to the statistical result and calculation method, and the fee is deducted by the bill on an hourly basis. The resource metering method used by each application is as follows:
Application CPU usage =∑ INSTANCE CPU specifications x monthly running duration (in minutes), that is, THE CPU specifications of all instances in the application multiplied by the total running duration in the current month.
Application memory usage =∑ Instance memory specifications x monthly running duration (in minutes), that is, the memory specifications of all instances in the application multiplied by the total running duration in the current month.
The price of CPU part is 0.0021605 yuan/minute /Core, and the price of memory part is 0.0005401 yuan/minute /GiB. SAE also provides a pre-paid resource pack, which is equivalent to a wholesale purchase of computing resources. As long as the resource pack is used up within the validity period, the system will automatically switch to a pay-as-you-go model.
Let’s take a closer look at how SAE can help reduce resource costs for microservices applications with a practical example. Assume that a microservice system contains 87 application instances, each running time is 8 hours per day on average, and the instance configuration is 2 Core, 4 GiB, and 20 GB disk.
- Application deployment using ECS with annual and monthly payment: 87 sets of computing C5 need to be purchased, the monthly cost of a single set is 186 yuan, and the total monthly cost is 16,146 yuan.
- Pay-as-you-go ECS deployment application: the price of a single unit is 0.63 yuan/hour, with a total usage of 20,880 hours per month and a total cost of 13,154 yuan.
- Using SAE to deploy applications: buy a 75,000 yuan annual resource pack, 87 instances run for 8 hours every day, just use up the resource pack quota, equivalent to a total cost of 6,250 yuan per month.
From this comparison, we can conclude that properly running SAE’s resilient capabilities can significantly reduce resource costs for microservice applications.
Additional capacity
SAE not only simplifies the operation and maintenance workload and reduces resource costs, but also improves a series of additional functions for microservice applications. This is the added value that application layer Serverless technology brings to developers. We can make use of these functions out of the box as much as possible, making it easier to build microservice applications.
** Complete application life cycle management: ** After hosting an application to SAE, you can perform application life cycle management operations such as updating, expanding and shrinking, starting and stopping, deleting, and monitoring the application.
** Out-of-the-box Registry: ** The commercial VERSION of SAE’s Nacos registry is free to use, no need to build your own. If you have special needs, such as enabling saE-deployed applications to discover each other with other applications, you can also use the registry provided by the Microservices Engine (MSE) product, or your own registry.
** Out-of-the-box Configuration Management center: **SAE integrates the Configuration Management function of the Application Configuration Management (ACM), enabling the ACM to centrally manage Application configurations in SAE.
Application-level traffic protection: SAE integrates with AHAS to achieve application-level flow control and degradation capabilities to fully ensure high availability of applications.
** Monitoring capabilities: ** Applications hosted by SAE receive free monitoring capabilities for basic resources (including CPU, memory, load, and network) and application layer (including JVM analysis, interface call analysis, etc.). If you need more advanced SQL analysis, exception analysis, upstream and downstream links, and interface snapshot, you can integrate Ali Cloud Application Time Monitoring product (ARMS).
**CI/CD integration: **SAE has deep integration with Cloud Effect, Cloud Effect 2020, Jenkins and other products, making it easy for developers to quickly deploy their built applications.
Multilingual support
Can SAE support applications that are not written in Java, or that do not use microservices frameworks such as Spring Cloud, and help companies reduce resource costs?
Of course you can. SAE provides container image deployment, which means that no matter what programming language is used, the resulting application can be deployed on SAE as long as it can be published as container image.
SAE’s extreme resiliency capabilities are no different for Java microservice applications, generic Java applications, and non-Java applications, all providing system utilization through Serverless technology. Some of the added value SAE provides, such as a free microservice registry, is only available for Spring Cloud or Dubbo applications.
conclusion
Let’s use this diagram to recall the great value of Serverless technology:
FAQ:
1. Q: How do I use SSH to log in to a server when an application instance does not have fixed machine resources or even a fixed IP address?
A: It is not necessary to have fixed machine resources and fixed IP addresses to troubleshoot problems. In the era of cloud computing, it is not a good practice to log in to a machine over SSH to troubleshoot problems. On the contrary, SAE provides perfect monitoring capability and can be easily integrated with cloud monitoring, ARMS and other new-generation monitoring and diagnosis products, providing greater convenience for troubleshooting. Of course, if you have to log in to a particular machine, SSH is still supported, and SAE provides Webshell tools to simplify the process.
2. Q: Isn’t disk an accounting dimension? What if the application needs a large disk capacity? Does the data on the hit disk remain after the application restarts?
A: In the microservices world, applications are generally Stateless and do not need to keep large amounts of local data. If this requirement is required in a particular scenario, NAS can be integrated and implemented using centralized storage.
3. Q: How do I view application logs?
A: The SAE console interface provides real-time access to file logs, providing a distributed log collection platform for free. Of course, it is strongly recommended to access Ali Cloud Log Service (SLS) products to further play the value of application log.
Course recommended
In order for more developers to enjoy the dividends brought by Serverless, this time, we gathered 10+ Technical experts in the field of Serverless from Alibaba to create the most suitable Serverless open course for developers to learn and use immediately. Easily embrace the new paradigm of cloud computing – Serverless. Click on the links to free courses: developer.aliyun.com/learning/ro…
Serverless public number, release the latest information of Serverless technology, collect the most complete content of Serverless technology, pay attention to Serverless trend, pay more attention to the confusion and problems encountered in your landing practice.