This article has participated in the “Digitalstar Project” and won a creative gift package to challenge the creative incentive money.

17 WeightedResponseTimeRule load balancing policy

This strategy is an extension of RoundRobinRule, which increases the weight calculation according to the operation of instances, and selects instances according to the weight, so as to achieve better distribution effect. There are three main parts

Timing task

WeightedResponseTimeRule initialize method:

void initialize(ILoadBalancer lb) {        
        if(serverWeightTimer ! =null) {
            serverWeightTimer.cancel();
        }
        serverWeightTimer = new Timer("NFLoadBalancer-serverWeightTimer-"
                + name, true);
        serverWeightTimer.schedule(new DynamicServerWeightTask(), 0,
                serverWeightTaskTimerInterval);
        // do a initial run
        ServerWeight sw = new ServerWeight();
        sw.maintainWeights();

        Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
            public void run(a) {
                logger
                        .info("Stopping NFLoadBalancer-serverWeightTimer-"+ name); serverWeightTimer.cancel(); }})); }class DynamicServerWeightTask extends TimerTask {
    public void run(a) {
        ServerWeight serverWeight = new ServerWeight();
        try {
            serverWeight.maintainWeights();
        } catch (Throwable t) {
            logger.error(
                    "Throwable caught while running DynamicServerWeightTask for "+ name, t); }}}Copy the code

DynamicServerWeightTask is an internal subclass, Initialization process through serverWeightTimer. The schedule (new DynamicServerWeightTask (), 0, serverWeightTaskTimerInterval); Start a scheduled task to calculate the weight of each service instance. The task is executed once in 30 seconds by default.

weighting

The weights are calculated using the maintainWeights method:

public void maintainWeights(a) {
    ILoadBalancer lb = getLoadBalancer();
    if (lb == null) {
        return;
    }
    if (serverWeightAssignmentInProgress.get()) {
        return; // Ping in progress - nothing to do
    } else {
        serverWeightAssignmentInProgress.set(true);
    }
    try {
        logger.info("Weight adjusting job started");
        AbstractLoadBalancer nlb = (AbstractLoadBalancer) lb;
        LoadBalancerStats stats = nlb.getLoadBalancerStats();
        if (stats == null) {
            // no statistics, nothing to do
            return;
        }
        double totalResponseTime = 0;
        // find maximal 95% response time
        for (Server server : nlb.getAllServers()) {
            // this will automatically load the stats if not in cache
            ServerStats ss = stats.getSingleServerStat(server);
            totalResponseTime += ss.getResponseTimeAvg();
        }
        // weight for each server is (sum of responseTime of all servers - responseTime)
        // so that the longer the response time, the less the weight and the less likely to be chosen
        Double weightSoFar = 0.0;
        
        // create new list and hot swap the reference
        List<Double> finalWeights = new ArrayList<Double>();
        for (Server server : nlb.getAllServers()) {
            ServerStats ss = stats.getSingleServerStat(server);
            double weight = totalResponseTime - ss.getResponseTimeAvg();
            weightSoFar += weight;
            finalWeights.add(weightSoFar);   
        }
        setWeights(finalWeights);
    } catch (Throwable t) {
        logger.error("Exception while dynamically calculating server weights", t);
    } finally {
        serverWeightAssignmentInProgress.set(false); }}Copy the code
  1. The statistics of each instance are recorded according to LoadBalancerStats, and the average response time of all instances is accumulated to obtain the total average response time (totalResponseTime)
  2. WeightSoFar +totalResponseTime- the actual average response time. The initial value of weightSoFar is 0. The weights are calculated and added up to be used in the next calculation interval. The whole interval is left closed and right open, and the interval is left open and right closed

The shorter the average response time of the instance, the larger the width of the weight interval, and the larger the width, the higher the probability of being selected

Instance selection

@edu.umd.cs.findbugs.annotations.SuppressWarnings(value = "RCN_REDUNDANT_NULLCHECK_OF_NULL_VALUE")
@Override
public Server choose(ILoadBalancer lb, Object key) {
    if (lb == null) {
        return null;
    }
    Server server = null;

    while (server == null) {
        // get hold of the current reference in case it is changed from the other thread
        List<Double> currentWeights = accumulatedWeights;
        if (Thread.interrupted()) {
            return null;
        }
        List<Server> allList = lb.getAllServers();

        int serverCount = allList.size();

        if (serverCount == 0) {
            return null;
        }

        int serverIndex = 0;

        // last one in the list is the sum of all weights
        double maxTotalWeight = currentWeights.size() == 0 ? 0 : currentWeights.get(currentWeights.size() - 1); 
        // No server has been hit yet and total weight is not initialized
        // fallback to use round robin
        if (maxTotalWeight < 0.001 d) {
            server =  super.choose(getLoadBalancer(), key);
            if(server == null) {
                returnserver; }}else {
            // generate a random weight between 0 (inclusive) to maxTotalWeight (exclusive)
            double randomWeight = random.nextDouble() * maxTotalWeight;
            // pick the server index based on the randomIndex
            int n = 0;
            for (Double d : currentWeights) {
                if (d >= randomWeight) {
                    serverIndex = n;
                    break;
                } else {
                    n++;
                }
            }

            server = allList.get(serverIndex);
        }

        if (server == null) {
            /* Transient. */
            Thread.yield();
            continue;
        }

        if (server.isAlive()) {
            return (server);
        }

        // Next.
        server = null;
    }
    return server;
}
Copy the code
  1. Generates a random number within an interval
  2. Traverse the weight list and compare the weight value with the size of random number. If the weight value is greater than or equal to the random number, take the index value of the current weight list to obtain the specific instance in the service instance list. Why is the interval left closed and right open? Because the random number may be 0, but the maximum value of the random number cannot be the maximum weight value, so all are left closed and right open