background

In order to improve test regression and improve the quality of project development delivery, our development and test teams worked together to perform automated tests on selected projects.

To better understand the need for mocks, take a look at the Wiki explanation of automated tests.

In software testing, automated testing refers to the process of using software independent of the software under test to automate the execution of tests, compare actual results with expectations, and generate test reports.

Our automated tests are executed according to our preset flow, and we don’t want to be affected by third-party services (up and down, the interface returns bad data), so when we call the tripartite interface, we mock it and return the data we expect.

In automated testing, we mock the HTTP, Dubbo, and MQ messages interfaces, and this article describes our solution for mocking the Dubbo interface.

Dubbo currently offers solutions

First let’s take a look at the mock features provided by the Dubbo framework itself.

The core code for the Dubbo Mock feature is as follows

//from MockClusterInvoker public Result invoke(Invocation invocation) throws RpcException { Result result = null; Mock configuration String Value = Directory.geturl ().getMethodParameter(Invocation.getmethodName (), Constants.MOCK_KEY, Boolean.FALSE.toString()).trim(); // No configuration or =false
        if (value.length() == 0 || value.equalsIgnoreCase("false")) {
            //no mock
            result = this.invoker.invoke(invocation);
        } else if (value.startsWith("force") {// force starts by forcing a mockif (logger.isWarnEnabled()) {
                logger.warn("force-mock: " + invocation.getMethodName() + " force-mock enabled , url : " + directory.getUrl());
            }
            //force:direct mock
            result = doMockInvoke(invocation, null);
        } elseMock {result = this.invocation. Invoke (Invocation); } catch (RpcException e) {// Do not mock if it is a business exceptionif (e.isBiz()) {
                    throw e;
                }
                
                if (logger.isWarnEnabled()) {
                    logger.warn("fail-mock: " + invocation.getMethodName() + " fail-mock enabled , url : " + directory.getUrl(), e);
                }
                result = doMockInvoke(invocation, e); }}return result;
    }
Copy the code

There are three types of logic for key=mock corresponding to value in URL

  1. Value = null Does not mock
  2. Value = force XXX Forces the mock logic
  3. Value = XXX The mock logic is used when the service invocation fails

Look at the third logic, do you get the sense that this is actually a downgrade, a failed downgrade, and the second logic is called a forced downgrade.

The official documentation provided by Dubbo also defines this mock as a downgrade

image.png

For details on how to configure Dubbo Mock expressions, see Dubbo’s Degraded Mock source code analysis

Whether it meets our needs

Our requirements are

  1. Whether a third party is online or not does not affect our mock
  2. Flexible and simple configuration

Mock =force:return NULL If the provider is not online, no provider exception will be thrown, which does not meet requirement 1

In test mode, add the following configuration to dubbo official Demo

//from DemoServiceComponent
@Reference(mock = "force:return null",check = false)
private DemoService demoService;
Copy the code

For requirement 2, the following issues also exist

  1. It was impossible to alter the Dubbo configuration in the original project, so we had to trigger the forced mock by adding the Override configuration to dubbo’s registry, which was inconvenient to use
  2. As you can also see from point 1, the mock function relies on the registry, and our mock environment and test environment both use the same registry, which is not feasible
  3. The mock value document is not detailed enough to construct for returns of complex types

Therefore, the conclusion is that both implementation and use cannot meet our needs. Dubbo’s mock function is still focused on production-level degradation requirements, so we need to develop mock schemes that are convenient for us to use.

We developed the extension solution

The design points of the mock scheme we developed for the Dubbo framework are as follows

  1. Similarly, mock logic is embedded by wrapping classes around Cluster extension points to ensure that services go offline without affecting our automated test runs
  2. The Mock switch for the interface is managed using the Properties configuration file, and the configuration can be hosted to Apollo without code intrusion
  3. Forward the request to our EsayMock server and configure the Json data of the interface return type
image.png

Can I do it with a Filter

I have seen similar schemes implemented by Filter on the Internet before. In fact, the first version is also implemented by Filter, but there is a problem that the provider of our mock interface must be online.

From the source point of view to explain why this problem occurs

With ZooKeeper as the registry and check=false

The Filter logic is implemented through Protocol’s wrapper class ProtocolFilterWrapper, which ProtocolFilterWrapper calls chain logic for filters of any Protocol other than RegistryProtocol.

//from ProtocolFilterWrapper
public <T> Invoker<T> refer(Class<T> type, URL url) throws RpcException {
    if (Constants.REGISTRY_PROTOCOL.equals(url.getProtocol())) {
        return protocol.refer(type, url);
    }
    return buildInvokerChain(protocol.refer(type, url), Constants.REFERENCE_FILTER_KEY, Constants.CONSUMER);
}
Copy the code

The problem is with RegistryProtocol, RegistryProtocol indirectly relies on DubboProtocl through the Cluster,Directory module, where the number of providers is checked and an exception is thrown if it is 0. All this happens before the invoker call generated by DubboProtocl.

The Invoker generated by DubboProtocol encapsulates the filter logic as well as the call logic to the remote service

Invoekr generated by RegistryProtcol encapsulates cluster invocation, load balancing and other service governance functions on the basis of DubboProtocol

//from RegistryDirectory

private void refreshInvoker(List<URL> invokerUrls) {
        Assert.notNull(invokerUrls, "invokerUrls should not be null"); // The dubbo framework automatically returns an empty URL for a directory that has no providerif(invokerUrls.size() == 1 && invokerUrls.get(0) ! = null && Constants.EMPTY_PROTOCOL.equals(invokerUrls.get(0).getProtocol())) { this.forbidden =true; // Forbid to access
            this.invokers = Collections.emptyList();
            routerChain.setInvokers(this.invokers);
            destroyAllInvokers(); // Close all invokers
        }
    	//...
}

public List<Invoker<T>> doList(Invocation invocation) {
        if (forbidden) {
            // 1. No service provider 2. Service providers are disabled
            throw new RpcException(RpcException.FORBIDDEN_EXCEPTION, "No provider available from registry " +
                    getUrl().getAddress() + " for service " + getConsumerUrl().getServiceKey() + " on consumer " +
                    NetUtils.getLocalHost() + " use dubbo version " + Version.getVersion() +
                    ", please check status of providers(disabled, not registered or in blacklist)."); } / /... }Copy the code

Haven’t seen dubbo source friends may not understand, you can see dubbo refer principle to taste again

insufficient

Json and the Dubbo interface in the Mock server are not strongly correlated, but that’s not a problem because we’re running a default flow.

The open source project

Said so much, is the original rational content, the following link, welcome everyone to use and make suggestions.

Dubbo – Easy – Mock project address

reference

Dubbo document – Service degraded