New features in Dubbo 2.7. New systems use Dubbo 2.7.1. Use the process accidentally stepped on this version of the Bug.

System architecture

Spring Boot 2.14-Release + Dubbo 2.7.1

The phenomenon of

Dubbo server starts successfully, provides services normally, but calls by consumers occasionally fail. The error is shown below:

Cause: Message can not send, because channel is closed. Checking the provider, however, shows that the service process is normal.

Log in to Dubbo Admin to view the provider service and discover that the service has two nodes.

192.168.164.77 is the IP address of the test server, the provider is located on this machine, and 10.20.80.67 is the IP address of the local machine, but the service is not running locally at this time.

Looking again at the cause of the service error, you can see that the provider called the service that provides RPC locally. The invocation failed because the local service was stopped. Procedure

This problem was never encountered in the previous version. At the beginning, I vaguely remembered that the Dubbo service provider registered to use the ZooKeeper temporary node. When the service was disconnected, the node would be deleted.

Question why

Search related issue on Dubbo home page and find the same problem Dubbo-2.7.1 providers duplicate registration.

Dynamic defaults to false. Prior to 2.7.1, the default value was null.

This problem has been fixed in the subsequent PR, Fix Issue 3785. The Fix code sets dynamic to true by default. However, this version has not been released until 20190515.

Source code analysis

Dynamic set to false causes this problem.

Note: The following is the source code for Dubbo 2.7.1. We use Dubbo XML configuration related.

In XML configuration, you can set the Dynamic attribute in two places.

The service will be parsed using the DubboNamespaceHandler at startup and injected into the Spring container.

The Provider label is resolved into a ProviderConfig object and the Service label is resolved into a ServiceBean object.

Looking at the inheritance relationship, you can see that both classes inherit AbstractServiceConfig, and that dynamic is in the parent object.

You can see that this field defaults to false.

Next, look at the Dubbo service export procedure, ServiceBean#export, and skip the other code to the key ServiceConfig#doExportUrlsFor1Protocol.

You can see that the appendParameters method is called multiple times. This method will use reflection to get all the field information in the object and add it to the map. The field name is the key value, and the actual value is the content. Now the map key value is:

You can see that there is also default.dynamic in the map, so if you look at the code and think for yourself, why is this happening?

Then we skip to the back:

This is where the map obtained above is assembled into a URL object and then registered to the registry.

Since the registry uses ZooKeeper, the ZookeeperRegistry implementation class will be used here.

First look at the url##getParameter method, where the Constants.DYNAMIC_KEY value is dynamic.

The method first values values by key from the parameters. If the value does not exist, the value will be concatenated using default as the prefix. If it does not already exist, use the default value passed in.

Look at the Parameters object at this point.

Url.getparameter (Constants.DYNAMIC_KEY, true) returns false.

Then analyze the zkClient#create method,

Since ephemeral is false, the node that the service is registered with ZooKeeper is a persistent node.

If the client is disconnected and the session times out, ZooKeeper automatically deletes the temporary node. Zookeeper-faq Interview question: How can service providers implement failure kick out?

When the service goes down, the node is persistent and will be registered with a new node when the service is restarted. This results in an additional node failure in ZooKeeper that cannot be eliminated naturally (unless the ZooKeeper node deletion method is manually called).

conclusion

Since Dubbo 2.7.2 has not been released yet, it is recommended that those who want to use the new features of Dubbo 2.7 use version 2.7.0. If you’re currently using version 2.7.2, don’t panic. As long as the service is not abnormally down or the process is forcibly killed using kill -9, the above phenomenon will not be encountered. When normal services are shut down, the Dubbo service automatically deregisters the service from ZooKeeper and deletes the node.

For those who are not using this version, please use 2.7.0 or wait until 2.7.2 is released.