SpringCloud06 — Ribbon load balancing service invocation

Introduce a,

Liverpoolfc.tv: github.com/Netflix/rib…

1, the introduction of

Spring Cloud Ribbon is a client load balancing tool based on Netflix Ribbon.

To put it simply, the Ribbon is an open source project released by Netflix. Its main function is to provide client-side software load balancing algorithm and service invocation. The Ribbon client component provides a comprehensive set of configuration options such as connection timeout and retry.

Simply put, the Ribbon lists all the machines behind the Load Balancer in the configuration file. The Ribbon automatically helps you connect these machines based on certain rules (simple polling, random linking, etc.). It’s easy to implement custom load balancing algorithms using the Ribbon.

2,

In simple terms, users’ requests are evenly distributed among multiple services to achieve high availability (HA) of the system.

Common load balancing software Nginx, LVS, hardware F5 and so on.

Ribbon Local load balancing client vs. Nginx server load balancing

Nginx is a server load balancer. All requests from clients are sent to Nginx, and Nginx forwards the requests. That is, load balancing is implemented by the server. When the Ribbon invokes the microservice interface, it retrieves the registry service list from the registry and caches it to the JVM. In this way, RPC remote service invocation technology can be implemented locally.

Centralized LB

That is, a separate LB facility (either hardware, such as F5, or software, such as Nginx) is used between the consumer and provider of the service, which is responsible for forwarding access requests to the provider of the service through some policy;

The in-process LB

The LB logic is integrated into the consumer, who learns from the service registry what addresses are available and then selects a suitable server from those addresses.

Load balancing and Rest calls in the Ribbon

The Ribbon is a soft load balancing client component that can be used in conjunction with other clients that require it. This is just one example.

Steps:

• The first step is to select EurekaServer, which preferentially selects servers with less load in the same region.
• The second step is to select an address from the service registration list obtained from the server according to the policy specified by the user.

The Ribbon provides several strategies: polling, randomization, and weighting by response time.

Netflix already has Ribbon, so it doesn’t have to.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
Copy the code

There is a dependency in the Ribbon:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
    <version>2.2.1. RELEASE</version>
    <scope>compile</scope>
</dependency>
Copy the code

RestTemplate (Spring Framework 5.2.2.release API)RestTemplate (Spring Framework 5.2.2.release API)

The difference between getForObject() and getForEntity()

• GetForObject () returns the object to which the data in the response body is converted, in Json format
• GetForEntity () returns the ResponseEntity object, which also contains more important information, such as the response header, response status code, and response body

The previous module used this time:

• EurekaMain7001 module
• EurekaMain7002 module
• OrderMain80 module
• PaymentMain8001 module
• PaymentMain8001 module

Here’s a demonstration of whether we can use getForEntity().

@GetMapping("/consumer/payment/get/{id}")
public CommonResult<Payment> getPayment(@PathVariable("id")Long id){
    return restTemplate.getForObject(PAYMENT_URL + "/payment/get/" + id,CommonResult.class);
}

@GetMapping("/consumer/payment/getForEntity/{id}")
public CommonResult<Payment> getPayment1(@PathVariable("id")Long id){
    ResponseEntity<CommonResult> entity =
        restTemplate.getForEntity(PAYMENT_URL + "/payment/get/" + id, CommonResult.class);
    if (entity.getStatusCode().is2xxSuccessful()){
        return entity.getBody();
    }else{
        return new CommonResult<>(444."Operation failed"); }}Copy the code

Ii. Advanced Ribbon

1. What load rules does the Ribbon have

You can enter the Irule interface to view:

public interface IRule{
    /* * choose one alive server from lb.allServers or * lb.upServers according to key * * @return choosen Server object. NULL is returned if none * server is available */

    public Server choose(Object key);
    
    public void setLoadBalancer(ILoadBalancer lb);
    
    public ILoadBalancer getLoadBalancer(a);    
}
Copy the code

The inheritance relationship is shown in the figure:

Click on the above IRule interface, right-click Diagrams–> Show Diagram, and then right-click show Implementations to choose which one to show.

Or select IRule and CTRL + H to display all of its subclasses.

The most commonly used are:

The name of the	The rules
RoundRobinRule	polling
RandomRule	random
RetryRule	Obtain the service based on RoundRobinRule. If the service fails to be obtained, the system reobtains the service within a specified period
WeightedResponseTimeRule	With an extension to RoundRobinRule, instances that are faster in response have greater selection weight and are easier to select
BestAvailableRule	Services that are in the breaker trip state due to multiple access failures are filtered out and the service with the least concurrency is selected
AvailabilityFilteringRule	Filter out the failed instances first, and then select the instances with low concurrency
ZoneAvoidanceRule	Default rules that compound the performance of the region where the server resides and the availability of the server to select the server

2. Set load rules

Official documents give clear warnings:

This custom configuration class cannot be placed under the current package or sub-package that @ComponentScan scans.

Otherwise, our custom configuration class will be shared by all Ribbon clients and will not be specialized.

So when you modify comSumer80, you need to create a new package at the same level as the SpringCloud package:

Create a rule under it:

@Configuration
public class MySelfRule {
    // Set it to random
    @Bean
    public IRule myRule(a){
        return newRandomRule(); }}Copy the code

Add comments to the main configuration class:

@SpringBootApplication
@EnableEurekaClient
@RibbonClient(name = "CLOUD-PAYMENT-SERVICE",configuration = MySelfRule.class)
public class OrderMain80 {
    public static void main(String[] args) { SpringApplication.run(OrderMain80.class,args); }}Copy the code

Test again: localhost/consumer/payment/get / 1, port random selection.

3. Load balancing algorithm

Default load rotation algorithm: Number of REST interface requests % Total number of server clusters = Actual server location. The rest interface count starts from 1 after each service restart.

List instances = discoveryClient.getInstances(“CLOUD-PAYMENT-SERVICE”);

List [0] Instances = 127.0.0.1:8002 List [1] Instances = 127.0.0.1:8001 8001+ 8002 combine to form a cluster. There are two machines in total and the total number of clusters is 2. According to the principle of polling algorithm:

• When the total number of requests is 1: 1%2=1 The corresponding subscript position is 1, and the service address is 127.0.0.1:8001
• When the total request digit is 2 :2%2= corresponding to a “wise man” and the subscript position is 0, the service address is 127.0.0.1:8002
• When the total request digit is 3 :3%2=1 the corresponding subscript position is 1, then the service address is 127.0.0.1:8001
• When the total request digit is 4 :4%2= corresponding to a “wise man” and the subscript position is 0, the service address is 127.0.0.1:8002
• And so on…

4, the source code

First go to the IRule interface and click on the RoundRobinRule class, which is the code for polling.

public class RoundRobinRule extends AbstractLoadBalancerRule {

    private AtomicInteger nextServerCyclicCounter;
    private static final boolean AVAILABLE_ONLY_SERVERS = true;
    private static final boolean ALL_SERVERS = false;

    private static Logger log = LoggerFactory.getLogger(RoundRobinRule.class);

    public RoundRobinRule(a) {
        nextServerCyclicCounter = new AtomicInteger(0);
    }
    / /... Other code
}
Copy the code

Where nextServerCyclicCounter is an atomic integer, the class construction initializes this parameter, which defaults to 0.

This class implements the methods of the interface, which is written like this:

public interface IRule{
    /* * choose one alive server from lb.allServers or * lb.upServers according to key * * @return choosen Server object. NULL is returned if none * server is available */

    public Server choose(Object key);
    
    public void setLoadBalancer(ILoadBalancer lb);
    
    public ILoadBalancer getLoadBalancer(a);    
}
Copy the code

Here’s a look at the Choose method implemented by the RoundRobinRule class, which selects a load balancing algorithm.

public Server choose(ILoadBalancer lb, Object key) {
    if (lb == null) {
        log.warn("no load balancer");
        return null;
    }

    Server server = null;
    int count = 0;
    while (server == null && count++ < 10) {
        List<Server> reachableServers = lb.getReachableServers();
        List<Server> allServers = lb.getAllServers();
        int upCount = reachableServers.size();
        int serverCount = allServers.size();

        if ((upCount == 0) || (serverCount == 0)) {
            log.warn("No up servers available from load balancer: " + lb);
            return null;
        }

        int nextServerIndex = incrementAndGetModulo(serverCount);
        server = allServers.get(nextServerIndex);

        if (server == null) {
            /* Transient. */
            Thread.yield();
            continue;
        }

        if (server.isAlive() && (server.isReadyToServe())) {
            return (server);
        }

        // Next.
        server = null;
    }

    if (count >= 10) {
        log.warn("No available alive servers after 10 tries from load balancer: "
                 + lb);
    }
    return server;
}
Copy the code

If lb is null, an error is reported. Set server=null (which server is used for processing), and set count to 0. Then enter the loop:

while (server == null && count++ < 10) {}
Copy the code

If the count count is too large, the server will be searched 10 times, and no server will exist.

How to find the server:

while (server == null && count++ < 10) {
    List<Server> reachableServers = lb.getReachableServers();
    List<Server> allServers = lb.getAllServers();
    int upCount = reachableServers.size();
    int serverCount = allServers.size();

    if ((upCount == 0) || (serverCount == 0)) {
        log.warn("No up servers available from load balancer: " + lb);
        return null;
    }

    int nextServerIndex = incrementAndGetModulo(serverCount);
    server = allServers.get(nextServerIndex);

    if (server == null) {
        /* Transient. */
        Thread.yield();
        continue;
    }

    if (server.isAlive() && (server.isReadyToServe())) {
        return (server);
    }

    // Next.
    server = null;
}
Copy the code

GetReachableServers is to see which servers are alive and return them, and getAllServers is to find all servers and get the number of them. If there is no service left alive or there is no service left alive, an error is returned. UpCount is the number of services left alive.

There are two up here.

int nextServerIndex = incrementAndGetModulo(serverCount);
server = allServers.get(nextServerIndex);
Copy the code

The method is to get the index of the next server, and then get the server based on the index.

Because it’s polling, if you go into the method, you can see that it gets the subscript in the same way that you get the remainder.

private int incrementAndGetModulo(int modulo) {
    for (;;) {
        int current = nextServerCyclicCounter.get();
        int next = (current + 1) % modulo;
        if (nextServerCyclicCounter.compareAndSet(current, next))
            returnnext; }}Copy the code

CompareAndSet ==>CAS, spin lock. As you can see, it’s + 1 and then modulo, initial 1, next time 2, and next time 1…… This gets the server’s index, which in turn gets the Server service.

5, the algorithm of handwriting load

Add the following code to controller 8001 and 8002:

@GetMapping(value = "/payment/lb")
public String getPaymentLB(a) {
    return serverPort;
}
Copy the code

The order80 module cancels load balancing with its own:

@Configuration
public class ApplicationContextConfig {
    @Bean
    // @loadBalanced
    public RestTemplate getRestTemplate(a){
        return newRestTemplate(); }}Copy the code

Create a package LB in springCloud with a LoadBalanced interface.

public interface LoadBalancer {
    ServiceInstance instances(List<ServiceInstance> serviceInstances);
}
Copy the code

Create its implementation class:

@Component
public class MyLB implements LoadBalancer{
    // Set from the class default
    private AtomicInteger atomicInteger = new AtomicInteger(0);

    // get the subindex and increment it
    public final int getAndIncrement(a){
        int current;
        int next;
        do{
            // Get the current number
            current = this.atomicInteger.get();
            // Next number
            next = current >= Integer.MAX_VALUE ? 0 : current + 1;
        }while (!this.atomicInteger.compareAndSet(current,next));
        System.out.println("next = " + next);
        return next;
    }

    @Override
    public ServiceInstance instances(List<ServiceInstance> serviceInstances) {
        int index = getAndIncrement() % serviceInstances.size();
        returnserviceInstances.get(index); }}Copy the code

Modify the controller of port 80:

@RestController
@Slf4j
public class OrderController {
    public static final String PAYMENT_URL = "http://CLOUD-PAYMENT-SERVICE";

    @Resource
    private LoadBalancer loadBalancer;

    @Autowired
    private RestTemplate restTemplate;

    @Resource
    private DiscoveryClient discoveryClient;

    @GetMapping("/consumer/payment/create")
    public CommonResult<Payment> create(Payment payment){
        return restTemplate.postForObject(PAYMENT_URL + "/payment/create",payment,CommonResult.class);
    }

    @GetMapping("/consumer/payment/get/{id}")
    public CommonResult<Payment> getPayment(@PathVariable("id")Long id){
        return restTemplate.getForObject(PAYMENT_URL + "/payment/get/" + id,CommonResult.class);
    }

    @GetMapping("/consumer/payment/getForEntity/{id}")
    public CommonResult<Payment> getPayment1(@PathVariable("id")Long id){
        ResponseEntity<CommonResult> entity =
                restTemplate.getForEntity(PAYMENT_URL + "/payment/get/" + id, CommonResult.class);
        if (entity.getStatusCode().is2xxSuccessful()){
            return entity.getBody();
        }else{
            return new CommonResult<>(444."Operation failed"); }}@GetMapping(value = "/consumer/payment/lb")
    public String getPaymentLB(a){
        List<ServiceInstance> instances = discoveryClient.getInstances("CLOUD-PAYMENT-SERVICE");
        if(instances == null || instances.size() <= 0) {return null;
        }

        ServiceInstance serviceInstance = loadBalancer.instances(instances);
        URI uri = serviceInstance.getUri();
        return restTemplate.getForObject(uri+"/payment/lb",String.class); }}Copy the code

Test: localhost/consumer/payment/lb, through, 8001 and 8002 polling.