The company uses Spring Cloud microservices, and the gateway uses Spring Gateway, in conjunction with the NACOS registry

There is a requirement for daily development and testing, that is, local microservices can be started. Whether it is through the front-end page click debugging, or tools such as Postman send API requests, they hope that the requests initiated by their local IP will be forwarded to their local microservices. Gateway is shared by both the development environment and the test environment. In addition, there is a complete set of microservices for the development environment or test environment, so that there is no need for additional gateway and corresponding microservices unrelated to their own development.

In fact, I am not familiar with the source code of Spring Gateway. I debugged the Gateway memory leak once before, which was an official bug. The count of the out-of-heap memory was forgotten to be released, which caused the Gateway to stop service every period of time. Now I want to rewrite the LB policy and use my own custom policy to fulfill the above requirements.

What’s the fastest way to get familiar with the code? Start the gateway with local debug, and then set a breakpoint to get familiar with the whole request forwarding and lb policy.

The gateway must be LoadBalancerClientFilter. The gateway must be LoadBalancerClientFilter. The gateway must LoadBalancerClientFilter

public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain)
Copy the code
protected ServiceInstance choose(ServerWebExchange exchange)

Copy the code

All break points and take a look

Send a request to the local gateway through Paw

The code for this requirement is:

if(url ! =null && ("lb".equals(url.getScheme()) || "lb".equals(schemePrefix)))
Copy the code

This is a dead lb string because of decomcompilation. according to the Spring specification, the source code should not be written like this.

The choose method is used only when lb protocol is configured. The return value of choose is used to select a service provider.

protected ServiceInstance choose(ServerWebExchange exchange) {
        return this.loadBalancer.choose(((URI)exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR)).getHost());
    }
Copy the code

The choose method implementation calls the Choose method of loadBalancer.

What are the implementation classes that implement the Choose interface method?

I don’t know which implementation class I’m going to go to, so I’m going to hit a breakpoint and see which implementation class I’m going to get into

As you can see, the implementation class is RibbonLoadBalancerClient. The method implementation passes to its own Choose method

public ServiceInstance choose(String serviceId, Object hint) {
        Server server = this.getServer(this.getLoadBalancer(serviceId), hint);
        return server == null ? null : new RibbonLoadBalancerClient.RibbonServer(serviceId, server, this.isSecure(server, serviceId), this.serverIntrospector(serviceId).getMetadata(server));
    }
Copy the code
protected ILoadBalancer getLoadBalancer(String serviceId) {
        return this.clientFactory.getLoadBalancer(serviceId);
    }
protected Server getServer(ILoadBalancer loadBalancer, Object hint) {
        return loadBalancer == null ? null: loadBalancer.chooseServer(hint ! =null ? hint : "default");
    }
Copy the code

Get the corresponding loadBalancer through the abstract factory and call its chooseServer method

Who is the real loadBalancer? Inheriting DynamicServerListLoadBalancer ZoneAwareLoadBalancer, why the loadBalancer is ZoneAwareLoadBalancer, can be configured? I don’t know yet. Ignore the factory strategy and move on.

ZoneAwareLoadBalancer is playing a lonely game because I only have one Zone so I call the parent BaseLoadBalancer chooseServer and pass a default key.

public Server chooseServer(Object key) {
        if (this.counter == null) {
            this.counter = this.createCounter();
        }

        this.counter.increment();
        if (this.rule == null) {
            return null;
        } else {
            try {
                return this.rule.choose(key);
            } catch (Exception var3) {
                logger.warn("LoadBalancer [{}]: Error choosing server for key {}".new Object[]{this.name, key, var3});
                return null; }}}Copy the code

This in turn passes to rule’s Choose method. Rule is the IRule interface. The implementation classes are as follows:

The actual implementation class is ZoneAvoidanceRule inherits PredicateBasedRule

public Server choose(Object key) {
        ILoadBalancer lb = this.getLoadBalancer();
        Optional<Server> server = this.getPredicate().chooseRoundRobinAfterFiltering(lb.getAllServers(), key);
        return server.isPresent() ? (Server)server.get() : null;
    }
Copy the code
    public abstract AbstractServerPredicate getPredicate();
Copy the code

To realize the chooseRoundRobinAfterFiltering AbstractServerPredicate abstract methods, and specific implementation method is:

private int incrementAndGetModulo(int modulo) {
        int current;
        int next;
        do {
            current = this.nextIndex.get();
            next = (current + 1) % modulo;
        } while(!this.nextIndex.compareAndSet(current, next) || current >= modulo);

        return current;
    }
Copy the code

At this point, a polling Server is found, which is the default implementation.

Do you think I have a chance? LoadBalancer rule loadBalancer rule loadBalancer rule loadBalancer rule

Let’s go back a little bit, but first we’re going to answer the first question

The gateway must be LoadBalancerClientFilter. The gateway must be LoadBalancerClientFilter. The gateway must LoadBalancerClientFilter

So irresponsible answer, I now see a little blush, don’t hide from you, directly on the code

 

Automatic loading mechanism, the initialization LoadBalancerClientFilter, and rely on RibbonAutoConfiguration. LoadBalancerClientFilter initialization also requires two parameters: LoadBalancerClient and LoadBalancerProperties. Check the RibbonAutoConfiguration.

LoadBalancerClientFilter is initialized and joins the Gateway’s Filter army

Public Mono Filter (ServerWebExchange Exchange, GatewayFilterChain chain) method

Now, back to the question I left,

The loadBalancer is LoadBalancerClient. The injected implementation class is RibbonLoadBalancerClient, and the RibbonLoadBalancerClient

@ConditionalOnMissingBean({LoadBalancerClient.class})

There is a chance to replace loadBalancer.

Now let’s look at how rule is initialized.

The ILoadBalancer in RibbonLoadBalancerClient can also be configured

You can specify custom ILoadBalancer and IRule by configuring the Properties file in the Gateway

Now let’s review the invocation flow and see two crazy things:

  1. LoadBalancerClientFilter
protected ServiceInstance choose(ServerWebExchange exchange) {
        return this.loadBalancer.choose(((URI)exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR)).getHost());
    }
Copy the code

Exchange is missing, only serviceId is missing:

((URI)exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR)).getHost()

2.RibbonLoadBalancerClient

public ServiceInstance choose(String serviceId) {
        return this.choose(serviceId, (Object)null);
    }
public ServiceInstance choose(String serviceId, Object hint) {
        Server server = this.getServer(this.getLoadBalancer(serviceId), hint);
        return server == null ? null : new RibbonLoadBalancerClient.RibbonServer(serviceId, server, this.isSecure(server, serviceId), this.serverIntrospector(serviceId).getMetadata(server));
    }
Copy the code

I just passed a null here

Lost request information and key default value “default”

So if we want to implement this, we have to override the Protected ServiceInstance Choose (ServerWebExchange Exchange) of LoadBalancerClientFilter.

The ILoadBalancer does not need to be customized. You only need to customize IRule

The code is as follows:

/ * * *@author wangdengwu
 */
@Slf4j
public class SameIpBalanceRule extends ClientConfigEnabledRoundRobinRule {

    public SameIpBalanceRule(ILoadBalancer lb) {
        this.setLoadBalancer(lb);
    }

    public SameIpBalanceRule(a) {}@Override
    public void initWithNiwsConfig(IClientConfig iClientConfig) {}@Override
    public Server choose(Object ip) {
        log.info("client ip:{}", ip);
        List<Server> servers = this.getLoadBalancer().getReachableServers();
        if (servers.isEmpty()) {
            return null;
        }
        if (servers.size() == 1) {
            return servers.get(0);
        }
        return sameIpChoose(servers, ip);
    }

    private Server sameIpChoose(List<Server> servers, Object ip) {
        for (int i = 0; i < servers.size(); i++) {
            Server server = servers.get(i);
            String host = server.getHost();
            if (StringUtils.equals((CharSequence) ip, host)) {
                returnserver; }}return super.choose(ip); }}Copy the code
/ * * *@author wangdengwu
 */
@Component
public class SameIpLoadBalancerClientFilter extends LoadBalancerClientFilter {

    @Value("${xxx.same.ip.enable}")
    private Boolean enableSameIp = false;

    public SameIpLoadBalancerClientFilter(LoadBalancerClient loadBalancer, LoadBalancerProperties properties) {
        super(loadBalancer, properties);
    }

    @Override
    protected ServiceInstance choose(ServerWebExchange exchange) {
        // Whether to enable the same IP policy
        if(! enableSameIp) {return super.choose(exchange);
        }
        // Get the browser visitor'S IP address
        String ip = getRealIp(exchange.getRequest());
        String serviceIp = exchange.getRequest().getHeaders().getFirst("serviceIp");
        // Forces the IP address to have the highest priority
        if(serviceIp ! =null) {
            ip = serviceIp;
        }
        if (this.loadBalancer instanceof RibbonLoadBalancerClient) {
            RibbonLoadBalancerClient client = (RibbonLoadBalancerClient) this.loadBalancer;
            String serviceId = ((URI) exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR)).getHost();
            // IP is used as the key to select the service instance
            return client.choose(serviceId, ip);
        }
        return super.choose(exchange);
    }

    private String getRealIp(ServerHttpRequest request) {
        // This parameter is usually set by the Nginx reverse proxy
        String ip = request.getHeaders().getFirst("X-Real-IP");
        if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeaders().getFirst("X-Forwarded-For");
        }
        if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeaders().getFirst("Proxy-Client-IP");
        }
        if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getHeaders().getFirst("WL-Proxy-Client-IP");
        }
        if (ip == null || ip.length() == 0 || "unknown".equalsIgnoreCase(ip)) {
            ip = request.getRemoteAddress().getAddress().getHostAddress();
        }
        // Handle multiple IP addresses (select only the first IP)
        if(ip ! =null && ip.contains(",")) {
            String[] ipArray = ip.split(",");
            ip = ipArray[0];
        }
        returnip; }}Copy the code

At this point, the code completes the requirements.

The gateway implements custom routing, but it is left to you to consider how to implement the same IP function when the @feignClient calls are used between services.