A Preliminary study on microservices

What are microservices

First of all, there is no official definition of microservices, so it is difficult to describe microservices directly. We can understand what microservices are by comparing traditional WEB applications.

The core of a traditional WEB application is divided into business logic, adapters, and an API or WEB interface accessed through a UI. Business logic defines business processes, business rules, and domain entities. Adapters include database access components, message components, access interfaces, and so on. The architecture diagram of a taxi-hailing software is as follows:

Although developed in a modular manner, they are eventually packaged and deployed as monolithic applications. For example, Java applications are packaged as WAR and deployed on Either Tomcat or Jetty.

This monomer application is more suitable for small projects with the advantages of:

1. Simple and direct development, centralized management

2. Basically no repetitive development

3. All functions are local, without distributed management overhead and invocation overhead

Of course, its disadvantages are also obvious, especially for Internet companies:

1. Low development efficiency: all the developers change the code in one project, submit the code to wait for each other, and the code conflicts constantly

2. Code maintenance is difficult: the code functions are coupled together, and newcomers don’t know where to start

3. Inflexible deployment: The build time is long, and any small change must rebuild the entire project, which is often a long process

4. Poor stability: A trivial problem can cause the entire application to fail

5. Poor scalability: Unable to meet service requirements under high concurrency

As a result, microservices architecture is now the norm in mainstream design. The idea is not to develop a single, monolithic application, but to break it up into small, interconnected microservices. A microservice performs a specific function, such as passenger management and order management. Each microservice has its own business logic and adapter. Some microservices also provide apis for other microservices and application clients to use.

For example, the system described above can be decomposed into:

Each business logic is decomposed into a microservice that communicates with each other through REST apis. Some microservices also develop apis to end users or clients. Typically, however, these clients do not have direct access to backend microservices, but instead pass requests through the API Gateway. The API Gateway is generally responsible for service routing, load balancing, caching, access control, and authentication.

Advantages of microservices architecture

Microservices architecture has a number of important advantages. First, it solves the problem of complexity. It breaks down individual applications into a set of services. While the total amount of functionality remains the same, the application has been decomposed into manageable modules or services. These services define explicit RPC or message-driven API boundaries. Microservices architecture reinforces a level of application modularity that is difficult to achieve with a single code base. As a result, microservices are much faster to develop and easier to understand and maintain.

Second, this architecture allows each service to be developed independently by a dedicated team. Developers are free to choose their development technologies as long as they comply with the service API contract. This means that developers can write or refactor services using new technologies, and since the services are relatively small, it doesn’t have much impact on the overall application.

Third, the microservice architecture enables each microservice to be deployed independently. Developers do not have to coordinate the deployment of service upgrades or changes. These changes can be deployed as soon as the tests pass. So the microservices architecture also makes CI/CD possible.

Finally, the microservice architecture allows each service to be independently extensible. We only need to define the configuration, capacity, number of instances, and other constraints that meet the service deployment requirements. For example, we can deploy CPU intensive services on EC2 compute optimization instances and in-memory database services on EC2 memory optimization instances.

Weaknesses and challenges of microservices architecture

There are no Silver Bullets, and the microservices architecture brings new problems and challenges. As the name suggests, microservices emphasize service size, but there is no universal standard. The rules by which business logic should be divided into microservices are an empirical project in itself. Some developers argue that 10-100 lines of code should be used to build a microservice. While building small services is what microservices architecture is all about, remember that microservices are a means to an end, not an end. The goal of microservices is to decompose applications sufficiently to facilitate agile development and continuous integration deployment.

Another major disadvantage of microservices is the complexity that comes with the distributed nature of microservices. Developers need to implement invocation and communication between microservices based on RPC or messaging, which makes the discovery between services, tracing of service invocation chains, and quality issues quite tricky.

Another challenge for microservices is partitioned database architecture and distributed transactions. Business transactions that update multiple business entities are quite common. These types of transactions are very simple to implement in a singleton application, because singleton applications tend to have only one database. But in a microservice architecture, different services may have different databases. The constraints of the CAP principle force us to abandon traditional strong consistency in favor of final consistency, which is a challenge for developers.

The microservices architecture also presents significant testing challenges. Whereas traditional monolithic WEB applications only need to test a single REST API, testing a microservice requires launching all the other services it depends on. The complexity should not be underestimated.

Another challenge with microservices is change across multiple services. For example, in A traditional single application, if services A, B, and C need to be changed, A depends on B, and B depends on C. We just need to change the corresponding module and deploy once and for all. But in the microservices architecture, we need to carefully plan and coordinate the deployment of changes for each service. We need to update C first, then B, and finally A.

Deploying microservices-based applications is also much more complex. Individual applications can simply be deployed on the same set of servers, and load balancing is used on the front end. Each application has the same address for underlying services, such as databases and message queues. Microservices are made up of a number of different services. Each service may have its own configuration, number of application instances, and underlying service addresses. This is where different configuration, deployment, extension, and monitoring components are required. In addition, we need a service discovery mechanism so that a service can discover the addresses of other services it communicates with. Therefore, successful deployment of microservice applications requires better deployment strategies and a high level of automation by developers.

The above problems and challenges can be summarized as follows:

• API Gateway
• Interservice invocation
• Service discovery
• Service fault tolerance
• Service deployment
• The data call

Fortunately, there are a number of microservices frameworks that can address these issues.

First generation microservices framework

Spring Cloud

Spring Cloud provides developers with tools to quickly build a generic model of distributed systems (including configuration management, service discovery, fuses, intelligent routing, micro-proxy, control bus, one-time tokens, global locks, leadership elections, distributed sessions, cluster state, etc.). Major projects include:

• Spring Cloud Config: Centralized external configuration management supported by git repositories. Configuration resources are mapped directly to the Spring Environment, but can be used by non-Spring applications if needed.
• Spring Cloud Netflix: Integrates with various Netflix OSS components (Eureka, Hystrix, Zuul, Archaius, etc.).
• Spring Cloud Bus: An event bus for associating services and service instances with distributed messaging. Used to propagate state changes (such as configuration change events) across the cluster
• Spring Cloud for Cloud Foundry: Integrate your application with Pivotal Cloud Foundry. Provides service discovery implementations, easily protects resources with SSO and OAuth2, and creates Cloudfoundry service proxies.
• Spring Cloud Cloud Foundry Service Broker: Provides a starting point for building a service broker that manages a service in cloud Foundry.
• Spring Cloud Cluster: Leadership election and Generic state Model (Based on Abstractions and implementations of ZooKeeper, Redis, Hazelcast, Consul)
• Spring Cloud Consul: Service discovery and configuration management with Hashicorp Consul
• Spring Cloud Security: support for load-balanced OAuth2 sleep clients and authentication header relaying in Zuul proxy.
• Spring Cloud SleUTH: Distributed tracing for Spring Cloud applications, compatible with Zipkin, HTrace, and log-based (for example, ELK) tracing.
• Spring Cloud Data Flow: Cloud-native choreography services for composable microservice applications for modern runtimes. The easy-to-use DSL, drag-and-drop GUI, and REST-API together simplify the overall choreography of microservices-based data pipelines.
• Spring Cloud Stream: Lightweight event-driven microservices framework for quickly building applications that can connect to external systems. A simple declarative model for sending and receiving messages between Spring Boot applications using Apache Kafka or RabbitMQ.
• Spring Cloud Stream APP Starters: The Spring Cloud task application launcher is a Spring Boot application, which can be any process, including Spring Batch jobs that don’t run forever, and which terminate/stop after limited time of data processing.
• Spring Cloud ZooKeeper: Discovers and manages zooKeeper services
• Spring Cloud for Amazon Web Services: Easily integrate hosted Amazon Web Services. It easily integrates AWS services, such as caching or messaging APIs, using Spring’s IDIOMS and APIs. Developers can build applications around managed services, regardless of the infrastructure.
• Spring Cloud Connectors: Make it easy for PaaS applications to connect to back-end services such as databases and messaging brokers on a variety of platforms (formerly known as the “Spring Cloud” project)
• Spring Cloud Starters: As a Spring Boot-based startup project, reducing dependency management (after Angel.sr2, no longer as a standalone project)
• Spring Cloud CLI: Plug-ins support the rapid creation of spring Cloud component applications based on Groovy predictions

Dubbo

Dubbo is an open source distributed service framework developed by Alibaba, which is committed to providing high-performance and transparent RPC remote service invocation solutions and SOA service governance solutions. Its core parts include:

• Remote communication: Provides an abstract encapsulation of a variety of long-connection-based NIO frameworks, including multiple threading models, serialization, and request-response mode of information exchange.
• Cluster fault tolerance: Provides transparent remote procedure calls based on interface methods, including multi-protocol support, and cluster support for soft load balancing, failure tolerance, address routing, and dynamic configuration.
• Automatic discovery: Based on registry directory services, service consumers can dynamically find service providers, address transparency, and service providers can smoothly add or subtract machines.

But obviously, both Dubbo and Spring Cloud are only suitable for specific application scenarios and development environments, and they are not designed to support generality and multilingualism. And they are just frameworks for the Dev layer, missing the overall solution for DevOps (which is what microservices architecture is all about). This was followed by the rise of Service Mesh.

Next-generation microservices: Service Mesh?

Service Mesh

Service Mesh, also known as “Service grid”, serves as the infrastructure layer for communication between services. In one sentence, a Service Mesh can be likened to TCP/IP between applications or microservices, responsible for network calls, traffic limiting, fuses, and monitoring between services. The TCP/IP layer is generally not a concern for writing applications (e.g., RESTful applications over HTTP), and the use of Service Mesh eliminates the need for relationships between services that are previously implemented by applications or other frameworks. Spring Cloud, OSS, for example, now just hand it over to Service Mesh.

Service Mesh has the following characteristics:

• An intermediate layer of communication between applications
• Lightweight Web proxy
• The application is not aware
• Decouple application retry/timeout, monitoring, tracing, and service discovery

The architecture of the Service Mesh is as follows:

The Service Mesh runs as Sidebar and is transparent to applications. All traffic between applications passes through it, so the control of application traffic can be realized in the Service Mesh.

Popular open source services for Service Mesh include Linkerd, Envoy, and Istio, while Buoyant recently released a Kubernetes-based Service Mesh project Conduit.

Linkerd

Linkerd is an open source network agent designed to be deployed as a service grid: a dedicated layer for managing, controlling, and monitoring service-to-service communication within an application.

Linkerd aims to solve problems that companies such as Twitter, Yahoo, Google and Microsoft have found when running large production systems. As a rule of experience, the source of the most complex, surprising, and emergent behavior is usually not the services themselves, but the communication between the services. Linkerd addresses these issues not only by controlling the communication mechanism, but by providing a layer of abstraction on top of it.

Its main features are:

• Load balancing: Linkerd provides a variety of load balancing algorithms that use real-time performance metrics to distribute load and reduce tail latency across the application.
• Fuses: Linkerd includes automatic fuses that will stop sending traffic to instances deemed unhealthy, giving them a chance to recover and avoid a chain reaction failure.
• Service discovery: Linkerd integrates with various service discovery backends to help you reduce the complexity of your code by removing ad-hoc service discovery implementations.
• Dynamic request routing: Linkerd enables dynamic request routing and rerouting, allowing you to set up staging services, Canaries, blue-green deployments with minimal configuration. Cross DC failover and Dark traffic.
• Number of retries and deadline: Linkerd can automatically retry requests during certain failures and can timeout requests after a specified period of time.
• TLS: Linkerd can be configured to send and receive requests using TLS, which you can use to encrypt communication across host boundaries without modifying existing application code.
• HTTP Proxy integration: Linkerd can be used as an HTTP proxy and is widely supported by almost all modern HTTP clients, making it easy to integrate into existing applications.
• Transparent proxy: You can use the Iptables rules on the host to set up transparent proxies through Linkerd
• GRPC: Linkerd supports HTTP/2 and TLS, allowing it to route gRPC requests, and supports advanced RPC mechanisms such as bidirectional flow, process control, and structured data loading.
• Distributed Tracing: Linkerd supports distributed tracing and measurement instruments that provide uniform visibility across all services.
• Instrumentation: Linkerd supports distributed tracking and measurement instruments that provide uniform visibility across all services.

Envoy

Envoy is a service-oriented architecture for L7 proxy and communication buses. This project was born with the following goals in mind:

The network should be transparent to the application so that when network and application failures occur, the root cause of the problem can be easily located.

Envoy offers the following features:

• External process architecture: works with applications developed in any language; Fast upgrade
• Based on the new C++11 encoding: provides efficient performance
• L3/L4 filter: The core is a L3/L4 network agent that can be plugged into the main service as a programmable filter to implement different TCP proxy tasks. Write filters to support tasks such as raw TCP proxy, HTTP proxy, TLS client certificate authentication, and so on.
• HTTP L7 filters: An additional HTTP L7 filter layer is supported. The HTTP filter is plugged into the HTTP link management subsystem as a plug-in to perform various tasks such as buffering, rate limiting, routing/forwarding, sniffing Amazon DynamoDB, and so on.
• HTTP/2: In HTTP mode, HTTP/1.1 and HTTP/2 and HTTP/1.1 and HTTP/2 bidirectional proxies are supported. This means that any combination of HTTP/1.1 and HTTP/2 can be bridged between client and target servers
• HTTP L7 routing: In HTTP mode, routes and redirection based on the Content Type and Runtime values are supported. Can be used as a front-end/edge proxy for a service
• GRPC support: gRPC is an RPC framework from Google that uses HTTP/2 as the underlying multiplexer. GRPC requests and responses hosted by HTTP/2 can use Envoy routing and LB capabilities
• Support for MongoDB L7: Obtain statistics and connection records
• DynamoDB L7: Obtains statistics and connection information
• Service discovery: Supports multiple service discovery methods, including asynchronous DNS resolution and service discovery through REST request services
• Health check: Contains a health check subsystem to perform proactive health checks on upstream service clusters. Passive health checks are also supported.
• Advanced LB: includes automatic retry, circuit breaker, global speed limiting, request blocking, exception detection. Support for request rate control is also planned for the future
• Front-end proxy: Can serve as a front-end proxy, including TLS, HTTP/1.1, HTTP/2, and HTTP L7 routes
• Excellent observability: Provides reliable statistical capability for all subsystems. Currently supports STATSD and compatible statistics libraries. Statistics can also be viewed through the management port, and a third-party distributed tracking mechanism is supported
• Dynamic configuration: Provides layered dynamic configuration apis that users can use to build complex centralized management deployments

Istio

Istio is an open platform for connecting, managing, and securing microservices. Istio provides a simple way to set up a network of deployed services with load balancing, authentication between services, monitoring, and other functions without changing any service code. To add Istio support to your services, you simply deploy a special sidecar in your environment that uses the Istio control panel function to configure and manage agents that intercept all network traffic between microservices.

Istio currently only supports service deployment on Kubernetes, but future releases will support other environments.

Istio provides a complete solution to meet the diverse needs of microservice applications by providing behavioral insights and operational control for the entire service grid. It provides many key functions uniformly across the service network:

• Traffic management: Controlling the flow of traffic between services and API calls to make the calls more reliable and make the network more robust in bad situations
• Observability: Understanding the dependencies between services and the nature and direction of traffic between them provides the ability to quickly identify problems
• Policy enforcement: Apply organizational policies to interactions between services to ensure that access policies are enforced and resources are well distributed among consumers. Policy changes are made by configuring the grid rather than modifying the application code
• Service identity and security: Provide verifiable identities for services in the grid and the ability to protect service traffic so that it can be moved across networks with different levels of confidence

The Istio service grid is logically divided into data panels and control panels:

• The data panel consists of a set of intelligent agents deployed as side cars to mediate and control all network traffic between microservices
• The control panel manages and configures agents to route traffic and enforce policies at run time

The following figure shows the different components that make up each panel:

Conduit

Conduit is an ultralight service grid service designed for Kubernetes to transparently manage runtime communications for services running on Kubernetes, making them more secure and reliable. Conduit provides visibility, reliability, and security without requiring code changes.

The Conduit Service Mesh also consists of a data panel and a control panel. The data panel carries the actual network traffic of applications. The control panel drives the data panel and provides northbound interfaces externally.

contrast

Linkerd is similar to Envoy in that both are web proxies for service-oriented communication that can perform functions such as service discovery, request routing, and load balancing. They are designed to solve the problem of communication between services so that applications are unaware of Service communication, which is the core concept of Service Mesh. Linkerd and Envoy are like distributed Sidebar. Multiple proxies like Linkerd and Envoy are connected to form a Service mesh.

Istio takes a higher perspective by dividing the Service Mesh into a Data Plane and a Control Plane. The Data Plane is responsible for all network communication between microservices, while the Control Plane is responsible for managing the Data Plane Proxy:

And Istio’s innate support for Kubernetes Bridges the gap between the application scheduling framework and the Service Mesh.

There is little information about Conduit. According to the official introduction, its positioning and functions are similar to Istio.

Kubernetes + Service Mesh

Kubernets have become the de facto standard for container scheduling, and containers can serve as the smallest unit of work for microservices to maximize the benefits of microservices architecture. So I think the future of microservices architecture will revolve around Kubernetes. Service meshes such as Istio and Conduit are designed for Kubernetes. They complement Kubernetes’ weakness in Service communication between microservices. While Dubbo, Spring Cloud, and others are mature microservice frameworks, they are more or less tied to specific languages or application scenarios and only address the microservice Dev level. To solve the Ops problem, they also need to be combined with resource scheduling frameworks such as Cloud Foundry, Mesos, Docker Swarm, or Kubernetes:

However, due to the initial design and ecology, there are many applicability problems to be solved.

Kubernetes, on the other hand, is a general-purpose container management platform independent of the development language, supporting both cloud-native and traditional containerized applications. It covers the Dev and Ops phases of microservices and, combined with the Service Mesh, provides a complete end-to-end microservice experience for users.

So I think the future microservices architecture and technology stack may look like this:

Multi-cloud platform provides resource capability (computing, storage and network, etc.) for microservices. Container is scheduled and arranged by Kubernetes as the minimum unit of work. Service Mesh manages the Service communication of microservices, and finally exposes the business interface of microservices externally through API Gateway.

I believe that the popularity of microservice frameworks based on Kubernetes and Service Mesh will greatly reduce the cost of microservice implementation and ultimately provide a solid foundation and guarantee for the implementation and large-scale use of microservices.

This article is from “Java Architecture Meditation”, a partner of the cloud community. For more information, you can pay attention to “Java Architecture Meditation”.