This article is part of GitChat’s “Continuous Delivery of Serverless Style Microservices: Part 1: The Architecture Case.” Article chat transcript please see: “Gu Yu: Building Serverless Style micro-service actual analysis (PART 1)”
An introduction to Serverless Architectures
The Serverless architecture can be traced back to Ken Fromm’s article Why The Future Of Software And Apps Is Serverless. In this article, Ken Fromm describes a future where the cloud computing infrastructure matures and applications do not need a server side. When building applications without weapons. Developers and operations personnel do not have to worry about how the server is installed and configured, how the network and load balancing are set up, there is no need to monitor the state, and there is no server-related work. In this way, the time cost and currency cost of building computer rooms can be reduced from annual to second.
In his blog Serverless Architectures, Martin Fowler divides them into two types:
The first serverless architecture is called BaaS (Backend as a Service). The application architecture is organized by a bunch of third-party apis. All state and logic is managed by these service providers. With the popularity of Rich Client applications such as mobile applications and single-page Web applications, the communication between the front and back ends is gradually dominated by API calls, and the required services are no longer maintained by the application development engineers and operation and maintenance engineers of the server side. The corresponding functions can be completed only by calling the third-party APIS that provide services. Examples are database services and user authentication services on the cloud.
Another serverless architecture is called FaaS (Function as a Service). The rise of this architecture stems from the evolution of AWS Lambda. AWS Lambda is a stateless code run time service that provides minimal code run resources. You can write programs in Java, node. js, Python, and C# to handle events for various AWS services. You do not need to initialize a server, install the operating system, and configure the program running environment. With few running resources and limited computations, such applications cannot save state, so they exist as functions.
The Serverless architecture introduced in this article is mainly an application of AWS Lambda and Amazon API Gateway architecture, which also has the characteristics of BaaS.
The programming model for AWS Lambda
AWS Lambda runs in an imaginary virtual container, but you can’t configure the container through the API. In addition, this virtual container has some resource limitations, the main ones are as follows:
5 minutes (300 seconds) of program run time.
512 MB file system space. (in/TMP)
Maximum 1536 MB of memory. (Minimum 128 MB, in 64 MB increments)
A maximum of 1024 file descriptors.
Maximum 1024 internal threads.
The programming model for AWS Lambda is as follows:
The execution process of Lambda:
When an event triggers a Lambda execution, the Lambda passes the information carried by the event to a Handler via a Context Object. In addition, Lambda can read pre-set environment variables.
The handler function is executed and the logs are logged through CloudWatch.
After the execution is complete, an event is used to return the execution result or an exception is thrown.
The execution results and corresponding exceptions can be bound to other resources for further processing.
When the event request occurs in bulk. Lambda is executed separately for each event. This means that the content cannot be shared between each request during execution. (By my own test, the memory storage is shareable, but the retention time and status of the content can not be guaranteed.)
Amazon API Gateway + AWS Lambda microservices architecture
According to Martin Fowler’s descriptive definition of microservices, we can think that microservices technically contain the following characteristics:
Each service runs in its own process.
Communication between services uses a lightweight communication mechanism (usually using HTTP resource apis).
These services are built around business capabilities and can be deployed independently through a fully automated deployment mechanism.
These services share a minimal centralized management.
Services can be developed in different languages using different data storage technologies.
In the case of existing AWS services, AWS Lambda satisfies points 1, 3, and 5 above, which is just a microservice processing unit, not a snap. However, 2 and 4 require additional services as management units to constitute microservices, which are generally implemented by the API gateway.
Amazon API Gateway is a fully hosted API Gateway service that helps developers easily create, publish, maintain, monitor, and secure apis of any size. It integrates many API gateway functions, such as caching, user authentication, and so on. It also supports configuration via HAML and Swagger so that the API can be configured with the code management system.
The Amazon API Gateway can pass the requested data to different resources for processing based on different Restful API access points. The general AWS API architecture is as follows:
When a request is accessed through a domain name, the application forwards the HTTP request to a CDN (CloudFornt).
CloudFront forwards the corresponding API requests to the API Gateway according to the forwarding rules.
Depending on the access point and content of the request, the API Gateway hands it to the corresponding AWS Lambda or EC2 service, or it can send it to other accessible services.
The request result is returned to the client when the processing is complete. When returned, the API Gateway can also process the returned content via Lambda.
This integration of API Gateway and Lambda allows for much lighter microservices than traditional microservices architectures. By planning API access and completing function development, teams can quickly build the simplest microservices, reducing the time to set up the microservices infrastructure from weeks to hours. In addition, the development efficiency and stability of microservices architecture are greatly improved.
A microservices architecture adventure
In early December 2016, I was working as a DevOps consultant on a DevOps transformation project for a client in Australia. The project is to improve the division’s DevOps capabilities on the AWS (Amazon Web Services) cloud computing platform.
The self-service application is based on the Ruby on Rails framework. The front-end is AngularJS 1.0, but there is no front end separation. The page code is still composed by ERB. Yi mobile is developed using Cordova. To reduce development difficulty and effort, the mobile app actually embeds angularJs-generated Web pages in a responsive fashion. However, the APP experience is not good because of frequent timeout.
The entire Rails application suite is deployed on AWS and is isolated through gateways and internal Business Operating Support System (BOSS) systems. The BOSS system uses SOAP to expose services, and a different department is responsible for it. Therefore, the business of cloud applications is to present users with a user-friendly interface and interact with the internal BOSS system through data conversion. The system architecture is shown in the figure below:
The application interaction flow is as follows
The browser or mobile is routed to the CDN (using AWS Cloudfront) via a domain name (hosted by AWS Route 53).
CDN differentiates according to the type of content requested. Static files (images, JS, CSS styles, etc.) are transferred to AWS S3 storage. Dynamic requests are sent directly to the AWS Elastic Load Balancer.
The load balancer forwards requests to Ruby On Rails applications On different EC2 compute instances based On their load status. Each application is a typical MVC Web application.
Applications on EC2 will store some data in AWS RDS (Relational Database ServiceS) and some in local files.
After processing by the application, the SOAP request is converted into a SOAP request and sent to the BOSS system for processing through the gateway. The BOSS system returns the corresponding message after processing.
The Redis service of AWS ElasiCache will be used as a cache to optimize business response speed according to business needs.
Team (
The app has been in development for years and has gone through many technicians. But no one has a complete understanding of the application code base. So we did a pain-point summary of the entire team and product:
Organizational structure
The operations team became a bottleneck, with only 4 Ops in a development team of about 60. The operations team provides support to the development team in addition to the day-to-day tasks. Many resources are restricted to the team, which leads to further delays in the resolution of various problems.
As businesses grow, infrastructure code bases are required to provide a wide variety of capabilities. However, any changes made by the Ops team would cause all the development teams to stop working to fix the problems caused by the update.
Application architecture
The application architecture does not achieve the effect of front-end and back-end separation, and still requires the same engineer to write the front-end and back-end code. Such a technology stack is demanding for developers, but the lack of suitable RoR engineers in the market leads to higher maintenance costs. After three months, it is still difficult to recruit the right engineers.
Multiple teams work on a code base, with various dependency points between old and new functionality. Coupled with Ruby’s language features, there are many implicit dependency points and class/method overlays in the code, resulting in slow development. We had four teams working on a code base, and three teams working on new features. One team needs to fix bugs and clear technical debt, all at the same time.
Technical debt
There are a lot of repetitive cucumber automated tests in the code base, but due to the lack of correct parallel test strategy, automated tests will fail randomly. It is difficult to create slave nodes of continuous integration server (Jenkins) locally, which makes it more difficult to find the cause of failure. If we’re lucky, it will take at least 45 minutes from submitting the code to a new release. If you’re unlucky, you won’t be able to complete a successful build for two or three days.
Infrastructure As Code is built on top of a hybrid legacy Ruby Code base. This code base encapsulates command line tools such as Packer and AWS CLI, and includes some CloudFormation transformation capabilities. The lack of long-term planning and coding specifications, coupled with frequent staff changes, makes the code base difficult to maintain.
In addition, the infrastructure code base is coupled to the application code base as a GEM, and the operations team has sole maintenance authority. So there are a lot of infrastructure problems that the development team can’t solve, and won’t solve.
I have been involved in the maintenance of many legacy systems on the Ruby technology stack. After working through these Ruby projects, I have found Ruby to be a technology stack that is fun to develop but painful to maintain. Most of the maintenance changes are due to Ruby and Gem version updates. In addition, because Ruby is flexible and people have their own ideas and habits, the code base is difficult to maintain.
While the team had a good continuous delivery process in place, the lack of Ops capability and application architecture limitations held the entire product back. Therefore, it is imperative to improve the Ops capability of the team through DevOps, alleviate the lack of Ops resources, and weaken the Contradiction of DevOps.
There are generally two approaches to DevOps organizational transformation: one is to improve the Ops capabilities of devs, and the other is to lower the barriers to Ops work. In the case of limited time resources, lowering the threshold of Ops through technological improvement is the method with the greatest short-term benefits.
Microservice triggers: Merger of business functions resulting from mergers and acquisitions
At the time I joined the project, the client acquired a local broadband/landline operator. So the old system needs new services that need to carry landlines and broadband. There was an order query business that required the current team to complete a requirement for both mobile and fixed broadband orders through an existing order query function.
This requires the addition of some new options and content to the original order query function, which can look up both mobile and fixed broadband orders. From the above pain points, it can be seen that the cost of completing such a task was very expensive at that time.
Making a DevOps transition on a development project is like changing the wheels of a moving car and stopping the entire team if you’re not careful. Therefore, I suggest setting up a new team in parallel to pilot the DevOps transformation while developing new features.
This is a function split and new function split requirements, just order query is a relatively independent and mature function in the original system. In order to avoid affecting the original development progress of each function. We decided to use a microservices architecture to do this.
Strategies for building an architecture for microservices
We don’t want to repeat the mistakes of previous application architectures, we want to separate the front and back ends. This allows smaller development teams to develop in parallel, and as long as the contracts between interfaces are negotiated, future development will be well integrated after completion.
This reminds me of Chris Richardson’s three microservices architecture strategies: Stop digging, separate the front and back ends, and extract microservices.
Stop digging means: If you find yourself in a hole, stop immediately.
The original single application was a tar pit for us, so we stopped working on the original codebase. And create a separate code base for the new application. So, we split the strategy pattern as follows:
In our architecture, new requirements require changing old applications. Here’s the idea:
Build new business pages and generate microservice contracts.
Build new microservices based on API contracts.
Deploy the Web front end to S3 and publish it using S3 Static Web Hosting.
Deploy back-end microservices online, and test with temporary domain names and CDN loading points.
Redirect traffic from the original application to the new microservice by updating the CDN.
Remove the old service code.
We were going to add an API to the original application to access the logic of the previous application. But consider that this is actually digging a hole. After assessing the complexity of the business. We found that this feature would only take two people two weeks (one person month) to develop, which is less than 20% of our estimated work. So we abandoned the idea of working on legacy code. Finally, access the background system directly through microservices, without the need for the original application.
The part where we unpack microservices is quite simple. For the back end, the integration of microservices can be completed only by modifying the CDN to cover the original Origin and the original function access point saved in route.rb.
Build new business pages and generate microservice contracts
Combined with the above application pain points and ideas, we determined the following directions when constructing the technical selection of microservices:
The front-end framework should have a good Responsive expansion.
Adopt Swagger to describe the behavior an API needs to have.
Contract tests drive microservice backend development through consumer drivers.
The front-end and back-end codebase are separate.
Front-end code frameworks are friendly to continuous delivery.
So we chose React as the front-end stack and managed dependencies and tasks with YARN. Another reason is that we can use React-Native to prepare us for building new applications in the future. In addition, we introduced the NODEJS version of the AWS SDK. Write common AWS related operations such as build, deploy, and configure. And describes the behavior of the back-end API through Swagger. In this way, the backend only needs to meet the API specification, and it is easy to do the front and back integration.
Deploy the front end to S3
The AWS S3 service comes with Static Web Hosting, which greatly reduces the time we spend building our base environment. If you’re still thinking about using Nginx and Apache as Web servers for static content, you’re not CloudNative enough.
The AWS S3 service has had its share of failures, but the SLAs handle static content much better than the EC2 instances we built ourselves. In addition, there are the following advantages:
Having separate urls makes it easy to do a lot of 301 and 302 redirects and overwrites.
It integrates well with CDN (CloudFront).
Easy to integrate with continuous integration tools.
Biggest advantage: Cheaper than EC2.
Build new microservices based on API contracts
When we first started building microservices, we had two choices:
Build a microservice using Sinatra (a Ruby gem used to build apis) that can reuse many components from the original Rails code base. In other words, just copy some code, put it into a separate code base, and you can do the function. But it also faces the same problems as the previous Ruby stack.
To build a microservice with Spring Boot, Java is currently the best choice as a mature engineering language, with a mature community and practices. You can reuse many jars in the background for SOAP processing. On the other hand, it solves the problems caused by the Ruby technology stack.
However, both solutions share a common problem: the need to build an infrastructure to run microservices using infrastructure tools written in Ruby. It is estimated that it will take at least one month to build this infrastructure, which is an optimistic estimate under the condition that the operation and maintenance team has some help.
So, find a way to reduce the congestion of the environment build and operations team and bypass the traditional EC2 build application.
Only Lambda can do that!
For all these considerations, we chose the Amazon API Gateway + Lambda combination. Amazon API Gateway + Lambda has additional benefits:
API Gateway configuration with Swagger specification is supported. That is, you can just import the Swagger specification on the front end and generate the API Gateway.
You can build Mock apis with data, which enables much of the consumer-driven contract development.
With the Stage functionality of the Amazon API Gateway, we don’t have to build a QA environment, a UAT environment, or a Staging environment. You only need to specify different stages to complete the corresponding switch.
Lambda’s launch took a short time and feedback was quick. Whereas the ORIGINAL API infrastructure built with CloudFormation took at least 15 minutes, Lambda took a few seconds to take effect.
Lambda is easy to write online. The online IDE isn’t very good, but it doesn’t really write many lines of code.
Lambda automatically self-scales on request, without regard to load balancing.
Despite these advantages, there is a key point: AWS Lambda may not be the right fit for your application scenario!
As described above, AWS Lambda has limited resources and time to run. Therefore, many business requirements for synchronization and strong consistency are unmet. AWS Lambda is therefore better suited for business scenarios that can be processed asynchronously. In addition, AWS Lambda does not support very well scenarios that consume a lot of storage space and CPU, such as AI and big data. (PS: AWS already has specialized AI and big data services, so there’s no need to fight against itself)
For our application scenario, the main function of the Ruby On Rails application above (at least 60% or more) is really just a data conversion adapter: processing the data input from the front end into the corresponding SOAP call.
So for such a simple scenario, Amazon API Gateway + Lambda fits the bill perfectly!
Deploy back-end microservices
After choosing Amazon API Gateway + Lambda, microservice deployment on the back end looks simple:
Update Lambda functions.
Update the API specification and require that API bindings correspond to Lambda functions to handle requests.
However, this is not an easy task. We’ll cover the pitfalls in this area in more detail in Continuous Delivery of Serverless Style Microservices (Middle) : The Challenges of Continuous Delivery.
Redirect requests from the original application to the new microservice
In this case, configure the new microservice API Gateway as a new Origin on the CDN, overwriting the original API access rules written in route.rb and nginx.conf. The CDN intercepts access requests so that they are forwarded to the API Gateway before nGINx can process them.
Of course, if you want to do grayscale publishing, you can’t do it in this way. CloudFront and ELB load balancers do not provide the weighted forwarding function. Therefore, you need to configure the Nginx to use the API Gateway as a Server in an upstream with access weights.
Remove the old service code
Don’t keep useless legacy code!
Don’t keep useless legacy code!
Don’t keep useless legacy code!
Say the most important and overlooked things three times. Cut the grass and remove the roots, although we can keep the code still. But cleaning up legacy code and automating tests that are no longer in use can save other teams a lot of unnecessary work.
Final architecture
After six people and two months of development (originally planned for eight people and three months), our Serverless microservice was finally launched. Of course, 60% of that time is spent exploring a whole new technology stack. If skilled, it is estimated that 4 people can finish the work in a month.
The final architecture is shown below:
In the figure above, the request still goes to the CDN (CloudFront) first, and then:
CDN forwards page requests to S3 and API requests to the API Gateway, depending on the request point.
The front-end content is put into different S3 buckets through blue-green deployment, and the corresponding content can be deployed only by changing the CDN Settings. While blue-green Bucket may seem a bit redundant for deployment at first glance, it is intended to be ready for integration online testing in a production environment. This will keep the environment inconsistent as little as possible.
The API Gateway has its own virtual private cloud (VPC), which provides good isolation at the network level.
The API requests forwarded through the API Gateway fall into three categories, each of which can be self-extensible depending on the status of the request:
Authentication classes: ElastCache (Redis) is requested on the first visit, and if the Token is invalid or nonexistent, the user authentication process is revisited.
Data request classes: The data request classes use Lambda to access Java microservices developed by other teams that are the only point of access for the backend system.
Operation audit class: Requests are logged in DynamoDB, a time series database, to track various logs of asynchronous requests.
The API Gateway has some caches of its own to speed up access to the API.
After the message is returned, the results of the three different requests are returned to the client through the API Gateway.
Benefits of the Serverless style microservices architecture
Since there is no time for EC2 facility initialization, we have reduced the workload by at least a month, respectively:
The time to initialize the network configuration.
Time to build the EC2 configuration.
Time to build the reverse proxy and front-end static content server.
Time to build the back-end API application infrastructure.
Time to build load balancing.
Infrastructure time to code the above in Ruby.
If you want to count API Gateway as infrastructure initialization time. The first API Gateway initialization took a day, and subsequent changes in the API Gateway combined with the continuous delivery process only took a few minutes.
In any case, Serverless significantly lowers the barriers to infrastructure configuration and operations.
In addition, Amazon API Gateway + Lambda’s microservices have other benefits for the team:
The development efficiency is high. The development feedback cycle of at least 45 minutes is reduced to less than 5 minutes.
Less irrelevant code, less code to maintain. Apart from focusing on the business itself. There is very little code to integrate upstream and API Gateways and downstream and back-end services.
Application maintenance costs are low. The code is only a few dozen lines long and all functional, making it easy to test. The increase in complexity within the code base is avoided.
In addition, we did a Java and NodeJs comparison. NodeJS is more efficient in developing the same functionality because Java converts the requested JSON to objects as well as the returned json, rather than processing JSON directly as NodeJS does. In addition, Java needs to introduce some other JAR packages as dependencies. Nodejs is 4 times more efficient than Java when developing the same functional microservice in an AWS scenario.
The last
Serverless style microservices greatly reduce development effort and infrastructure development and maintenance effort. But it also brings new challenges:
Management of a number of functions.
SIT, UAT environment management.
Continuous delivery pipeline configuration.
Testing for infrastructure integration.
This led us to rethink how microservices with the Serverless architecture can better deliver continuous delivery.