How does the client access the service?


Traditional development mode, all services are local, the client can call directly, now according to the function of independent services, how to access the client?

There are N services in the background, and the front desk needs to manage N services. If a service is offline/updated/upgraded, the front desk needs to be redeployed, which is obviously not in line with our concept of splitting. In addition, the invocation of N services is also a large network overhead. There are general micro services in the system, usually stateless, user login information and permission management is best to have a unified local maintenance management (OAuth2).

Therefore, there is usually a proxy (API Gateway) between N services and clients in the background. Its functions are as follows:

  • Unified service entrance is provided, aggregation interface makes service transparent to callers, and the coupling degree between client and back end is reduced
  • Aggregate background services to save traffic, improve performance, and improve user experience
  • Provides API management functions such as security, flow control, filtering, caching, charging, and monitoring


How do services communicate with each other?


Since the services are independently deployed, communication is a problem, but fortunately there are many mature solutions in the industry, such as:

  • Synchronous communication:
    • REST (JAX-RS, Spring Boot)
    • RPC (Dubbo, Thrift)
  • Asynchronous communication:
    • The RabbitMQ, Kafka


Three, so many services how to search?


In microservices architecture, for high availability, cluster is commonly used to build services. A service may go offline at any time, or new service nodes may be added in response to temporary access pressures.

How do services feel about each other? How are services managed? That’s the problem with service discovery. Basically, the distributed management of service registration information is done through similar technologies such as ZooKeeper. When the service goes live, the service provider registers its service information with ZooKeeper (or a similar framework) and maintains long links via heartbeat, updating the link information in real time. The service caller addresses a service using ZooKeeper and can also cache service information locally to improve performance. When the service goes offline, ZooKeeper sends a notification to the service client.


Four, service hang how to do?


In microservice architectures, it is quite common for a single request to invoke multiple services. For example, when A client accesses service A, service A needs to invoke service B, and service B needs to invoke service C. Due to network reasons or its own reasons, if service B or SERVICE C cannot respond in time, service A will be blocked until service B and service C respond. If a large number of requests flood in, the thread resources of the container will be consumed and the service will break down. The “avalanche” effect of service failures is the spread of service-to-service dependencies, causing a chain reaction that can have catastrophic consequences for the entire microservice system.

Avalanches are caused by butterfly effect in the system, and there are various reasons for their occurrence. We cannot completely eliminate the occurrence of avalanches from the source, but the root cause of avalanches comes from the strong dependence between services, so we can evaluate service fault tolerance in advance. Solutions can be classified as follows:

  • Request caching: Support to cache a request and return results;
  • Request merge: Merge the same requests and invoke the batch interface;
  • Traffic limiting: When too many requests may drag down the entire website, traffic limiting measures are usually taken to reduce the load on the machine.
  • Service isolation: Limits the resources that call distributed services so that the failure of one called service does not affect other service invocations;
  • Service fuse: sacrifice local service to preserve the overall system stability measures;
  • Service degradation: After a service meltdown, the client calls its own local method to return the default value.

The next post will bring you an ecosystem of micro services architecture, please pay attention. If you want to learn about this video, click here to get the Java Microservices Architecture Video Tutorial.