What is load balancing?

What is load balancing?

I remember my first encounter with Nginx was in the lab, when Nginx was needed to deploy websites on servers. Nginx is a service component for reverse proxies, load balancing, HTTP caching, and more. So what is the load balancing here?

Load Balance (LB) is a technical solution. It is used to distribute load among multiple resources (usually servers) to optimize resource usage and avoid overload.

Resource is equivalent to the execution operation unit of each service instance. Load balancing is to allocate a large number of data processing operations to multiple operation units for execution, which is used to solve the problems of large traffic, high concurrency and high availability in the Distributed Internet system. So what is high availability?

What is high availability?

What is high availability first?

This is the basis of the CAP theorem for distributed systems, and also the three indicators of distributed systems:

  1. Consistency
  2. Availability
  3. Partition tolerance

What is High Availability? High availability (HA) is a feature or indicator of the system. It usually means that the service running time of a certain performance is higher than the average normal time. Conversely, eliminate the time when system services are unavailable.

High availability (HA) is a measure of whether the system is still available when one or more servers are down.

For example, some well-known sites guarantee four nines or more, which is more than 99.99% availability. That 0.01 percent is the percentage of time out of order. For example, if the e-commerce website is popular, the service is not available, which will cause the merchants to lose money and users. There will be compensation for system downtime and service unavailability on the basis of improved availability.

For example, a single single instance of an ordered service can be replaced by multiple instances of an ordered service with load balancing, that is, redundancy can be used to improve reliability.

In a word, Load Balance is one of the factors that must be considered in distributed system architecture design. Generally, load balancing and redundancy of the same service instance are used to solve the problems of heavy traffic, high concurrency, and high availability in distributed systems. The core of load balancing is whether it is evenly distributed.

Common load balancing cases

Scenario 1: Microservices architecture, gateway routes to specific service instances hello:

  • Two identical service instances hello Service, one on port 8000 and the other on port 8082
  • Through Kong’s load balancing LB function, requests are evenly distributed between the two Hello service instances
  • Kong has many load balancing algorithms: default weighted-round-robin algorithm, and consumer: Consumer ID as hash algorithm input value

Scenario 2: Cluster of service A calling service B in microservices architecture. Through the Ribbon client load balancing component:

  • Load balancing strategy algorithm is not advanced, the simplest is random selection and round robin

Iv. Internet distributed system solutions

The common Internet distributed system architecture is divided into several layers, which are generally as follows:

  • Client layer: for example, user browser and APP
  • Reverse proxy layer: technology selection Nignx or F5, etc
  • Web layer: In the scenario where the front and back ends are separated, NodeJS, RN, and Vue can be used on the Web end
  • Business service layer: Java, Go, general Internet companies, technical solution selection is SC or Spring Boot + Dubbo servitization
  • Data storage layer: DB MySQL, Cache Redis, search ES, etc

A request from layer 1 through layer 4, each layer of access requires load balancing. That is, when each upstream calls multiple downstream business parties, the call needs to be made evenly. In this way, the overall system is more load balanced

Layer 1: Client layer -> reverse proxy layer load balancing

How is load balancing at the client layer -> reverse proxy layer implemented?

The answer is: DNS polling. DNS can set multiple IP addresses through A (Address: returns the IP Address pointing to the domain name). For example, the DNS for accessing bysocket.com is configured with IP1 and ip2. For high availability of the reverse proxy layer, there are at least two A records. This redundant nginx service instance of two CORRESPONDING IP addresses prevents a single point of failure.

Each time a bysocket.com domain name is requested, DNS polling returns the corresponding IP address. Each IP address corresponds to the service instance of the reverse proxy layer, that is, the external IP address of nginx. This ensures that each reverse proxy layer instance gets a balanced allocation of requests.

Layer 2: Reverse proxy layer -> Load balancing of the Web layer

How is the load balancing of the reverse proxy layer -> Web layer implemented?

It is handled by the load balancing module of the reverse proxy layer. For example, nginx has several balancing methods:

  1. Request polling. Requests are assigned to web tier services one by one, in chronological order, and then the cycle repeats. If the Web layer service becomes Down, it is automatically deleted
upstream web-server {
    server ip3;
    server ip4;
}Copy the code

  1. IP hash. Determine the route to the corresponding Web layer according to the IP hash value. As long as the USER’s IP is uniform, requests to the Web layer are uniform. Another benefit is that requests from the same IP address are distributed to the same Web layer service. In this way, each user has fixed access to a Web layer service, which can solve the session problem.
upstream web-server {
    ip_hash;
    server ip3;
    server ip4;
}Copy the code

  1. Weight Weight, fair, urL_hash, etc

Layer 3: Web layer -> load balancing of the business services layer

How to achieve load balancing in the Web layer -> business services layer?

For example, Dubbo is a service governance solution, including service registration, service degradation, access control, dynamic configuration of routing rules, weight adjustment, and load balancing. One of these features is intelligent load balancing: multiple load balancing policies are built in to intelligently sense the health status of downstream nodes, significantly reducing call latency and improving system throughput.

To avoid single points of failure and support service horizontal expansion, multiple instances of a service are deployed, that is, Dubbo cluster deployment. Multiple service instances are added as one service Provider, and one Provider is randomly selected from 20 providers based on the configured random load balancing policy. Assume that the seventh Provider is randomly selected. The LoadBalance component selects a provider from the provider address list to call using a balancing policy. If the call fails, it selects another call.

Dubbo has four built-in load balancing policies:

  • RandomLoadBalance: random load balancing. Pick one at random. Is the default load balancing policy for Dubbo.
  • RoundRobinLoadBalance: Polls load balancing. Poll to select one.
  • Leastactive VeloadBalance: minimum number of active calls, random for the same number of active calls. Active count refers to the count difference before and after the call. Slow providers receive fewer requests because the difference between the count before and after a slower Provider is greater.
  • ConsistentHashLoadBalance: consistency hash load balance. Requests with the same parameters always land on the same machine.

Also, you can implement your own load balancing strategy for business needs

Layer 4: Business service layer -> Load balancing of the data storage layer

DBProxy is used to implement load balancing at the data storage layer. For example, MySQL sub-library sub-table.

When the access of single library or single table is too large and the amount of data is too large, two dimensions of vertical split and horizontal split are needed. For example, the horizontal segmentation rule:

  • Range, time,
  • Hash model, order according to store ID, etc

But with this load comes the following problems that need to be solved:

  • Distributed transaction
  • Cross-database joins, etc

The current situation of the sub-database sub-table product solutions are many: Dangdang Sharding – JDBC, Ali Cobar and so on

Five, the summary

Externally, load balancing is a whole system or software. Internally, layers are called up and down. As long as there are calls, you need to consider load balancing. Therefore, Load Balance is one of the factors that must be considered in distributed system architecture design. Consider primarily how requests received downstream are evenly distributed:

  • Layer 1: Client layer -> reverse proxy layer load balancing. Through DNS polling
  • Layer 2: Reverse proxy layer -> Load balancing of the Web layer. Load balancing module via Nginx
  • Layer 3: Web layer -> load balancing of the business services layer. Load balancing modules through the service governance framework
  • Layer 4: Business service layer -> Load balancing of the data storage layer. By distributing the data horizontally, the data is evenly distributed, and theoretically the requests are evenly distributed. Like sharding by buyer ID

Original is not easy, try to draw more pictures, illustration is better than a thousand words ([email protected])

References:

  • “All about load balancing” at https://mp.weixin.qq.com/s/xvozZjmn-CvmQMAEAyDc3w
  • “Dubbo load balancing” at http://dubbo.apache.org/zh-cn/blog/dubbo-loadbalance.html
  • https://zh.wikipedia.org/wiki/%E8%B4%9F%E8%BD%BD%E5%9D%87%E8%A1%A1