In Dubbo microservices, the registry is one of the core components. Dubbo realizes the registration and discovery of services in distributed environment through the registry, which is the link between distributed nodes. The main functions are:
- Dynamic membership: A service provider can dynamically expose itself to other consumers through a registry without the need for each consumer to update the configuration file.
- Dynamic discovery: A consumer can dynamically sense new configurations, routing rules, and new service providers without restarting the service for them to take effect.
- Dynamic adjustment: The registry supports dynamic adjustment of parameters. New parameters are automatically updated to all relevant service nodes.
- Unified configuration: It avoids the problem of inconsistent configurations for each service caused by local configurations.
Dubbo mainly includes four kinds of registry implementations: Zookeeper, Redis, Simple, and Multicast.
Zookeeper is the official recommended registry implementation, which has been widely used in the production environment. The specific implementation is in the Dubbo source code Dubbo-Registry-ZooKeeper module. However, the stability of Redis registry is worse than that of ZK. Its stability mainly depends on Redis itself. Alibaba does not use Redis as a registry. Simple is a Simple memory-based registry implementation that is itself a standard RPC service that does not support clustering and is subject to a single point of failure. The Multicast mode does not need to start any registries, just through the broadcast address, can discover each other. When the service provider starts, it broadcasts its own address, and when the consumer starts, it broadcasts a subscription request, and when the service provider receives a subscription request, it broadcasts or unicast it to the subscriber according to the configuration.
Workflow of the registry
The Provider registeredWhen the Provider starts, it writes its own metadata information to the registry and subscribes to configuration metadata information.Consumer subscriptionWhen a Consumer starts, it writes its own metadata to the registry and subscribes to the service provider, routing, and configuration metadata.The service governance center is startedWhen dubo-admin is started, it subscribes to all consumer, service provider, routing, and configuration metadata information simultaneously.Dynamic registration and discovery: When a new Provider joins or leaves the registry, the service Provider list changes, and the change information is dynamically notified to consumers and service management center.Monitoring Center Collection: When a Consumer initiates a call, the call and statistics are asynchronously reported to the monitoring center.
The principle of zookeeper
Zookeeper is the registry of tree nodes. Each node is classified into persistent nodes, persistent sequential nodes, temporary nodes, and temporary sequential nodes.
- Persistent node: The node is guaranteed not to be lost after the service is registered and will exist even after the registry restarts.
- Persistent sequential nodes: The ability to order nodes has been added to the persistent node feature.
- Temporary node: If the connection is lost or the session times out after the service is registered, the registered node automatically disappears.
- Temporary node order: The ability to order nodes has been added to the temporary node feature.
When Dubbo uses ZK as the registry, only temporary nodes and persistent nodes are created, and there is no requirement on the order in which they are created.
Dubbo/com. Foo. BarService/will is the service provider in the Zookeeper registry path example, is a kind of attribute structure, the structure is divided into four layers: Root (root node, corresponding to dubbo in the example), service (interface name, corresponding to com.foo.BarService in the example), four service directories (corresponding to providers in the example, Other directories include Consumers, Routers, and Configurators. Under the service classification node is the specific Dubbo service URL. The following is an example of an attribute structure:
+ /dubbo
+-- service
+-- providers
+-- consumers
+-- routers
+-- configurators
Copy the codeTree structure relationships: (1) The root node of the tree is the registry group, under which there are multiple service interfaces, the group value comes from the group attribute in the user configuration
, the default is /dubbo. (2) There are four subdirectories under the service interface, These paths are providers, consumers, Routers, and Configurators. These paths are persistent nodes. (3) The service provider directory (/dubbo/service/providers) contains multiple URL metadata information for the interface. (4) Service consumers directory (/dubbo/service/consumers) contains decoupled multiple consumer URL metadata information. (5) The/Dubbo /service/ Routers directory contains multiple URL metadata information for consumer routing policies. (6) dynamic configuration directory (/ dubbo/service/configurators) below contains multiple dynamic configuration for server URL metadata information.
Tree diagram, as follows:
Configuration implementation:
<beans> <! --> <dubbo: Registry protocol="zookeeper" address="ip:port,ip:port,ip:port"/ > <! --> <dubbo: Registry protocol="zookeeper" address="ip:port|ip:port|ip:port" />
</beans>
Copy the codeZookeeper release implementation
Providers and consumers need to register themselves with Zookeeper. Service providers are registered so that consumers can subscribe (or, more accurately, be aware of the presence of the service) to make remote calls; The service governance center also senses that a new service provider is coming online. The consumer is published so that the service governance center can find itself. The Zookeeper publish subscribe code is very simple, just call Zookeeper’s Client library to create a directory in the registry, as shown in the following code:
@Override
public void doRegister(URL url) {
try {
zkClient.create(toUrlPath(url), url.getParameter(DYNAMIC_KEY, true));
} catch (Throwable e) {
throw new RpcException("Failed to register " + url + " to zookeeper " + getUrl() + ", cause: "+ e.getMessage(), e); }}Copy the codeUnpublishing the corresponding is also easy, just delete the corresponding path in the ZK registry, as shown in the following code:
@Override
public void doUnregister(URL url) {
try {
zkClient.delete(toUrlPath(url));
} catch (Throwable e) {
throw new RpcException("Failed to unregister " + url + " to zookeeper " + getUrl() + ", cause: "+ e.getMessage(), e); }}Copy the codeZookeeper subscription implementation
There are two methods of subscription: pull and push. One is that the client rotates the registry to pull the configuration, and the other is that the registry actively pushes data to the client. The two methods have their own advantages and disadvantages. At present, Dubbo uses the pull mode for the first time, and the data is pulled again after receiving the event.
When a service is exposed, the server subscribes to Configurators to listen for dynamic configurations. When the consumer starts up, the consumer subscribes to three directories, providers, routers, and Configurators, which correspond to service providers, routes, and dynamic configuration changes, respectively.
Dubbo provides two different packages of ZK open source client libraries, corresponding to interfaces:
- Apache Curator
- zkClient
We can set the client property of
to use a different client implementation library for both Exhibit and zkClient. If this is not set, exhibit and zkClient are used as implementations by default.
The Zookeeper client uses the event notification + Client Pull mode. When the Zookeeper client connects to the registry for the first time, it obtains secure data of the corresponding directory. And register a Watcher on the subscribed node. The client maintains a TCP long connection with the registry. When there is any subsequent data change on each node, the registry will actively notify the client (event notification) according to the callback of Watcher. Pulls the full amount of data from the corresponding node (client pull), as shown in the NotifyListener#notify(List
urls) interface. Full pull has a limitation, the party committee will cause great pressure to the network when there are many service nodes.
Each Zookeeper node has a version number. When the data of a node changes, the corresponding version number will change and trigger watcher event to push data to subscribers. The version number emphasizes the number of changes. Even if the value of this object does not change, the version number will still change as long as there is an update operation.
Zookeeper implements the service subscription core code in ZookeeperRegistry:
@Override
public void doSubscribe(final URL url, final NotifyListener listener) {
try {
if(any_value.equals (url.getServiceInterface())) {/*** subscribe to all services ***/ / subscribe to all data String root = toRootPath(); // Listeners ConcurrentMap<NotifyListener, ChildListener> Listeners = zkListeners.ifListeners == null {// To get listeners, create a listener and place it in the cache. zkListeners.putIfAbsent(url, new ConcurrentHashMap<>()); listeners = zkListeners.get(url); } ChildListener zkListener = listeners.get(listener);if(zkListener == null) {// zkListener is null. Create a listeners listeners. PutIfAbsent (Listener, (parentPath, currentChilds) -> { Only when a change notification is triggered // The notification is received if the child node changes, traversing all the child nodesfor(String child : currentChilds) { child = URL.decode(child); // If there are child nodes that have not been subscribed, it indicates that the node is newly addedif(! anyServices.contains(child)) { anyServices.add(child); Subscribe (url.setPath(child).addParameters(INTERFACE_KEY, child, Constants.CHECK_KEY, string.valueof ()false)), listener); }}}); zkListener = listeners.get(listener); } // create a persistent node, then subscribe to the immediate child of the persistent node zkClient.create(root,false);
List<String> services = zkClient.addChildListener(root, zkListener);
if(CollectionUtils isNotEmpty (services)) {/ / traverse all child nodes to subscribefor (String service : services) {
service = URL.decode(service);
anyServices.add(service);
subscribe(url.setPath(service).addParameters(INTERFACE_KEY, service,
Constants.CHECK_KEY, String.valueOf(false)), listener); }}}else{/*** ordinary consumer service subscription ***/ List<URL> urls = new ArrayList<>(); //toCategoriesPath(URL): Categoriespath applies to a group of paths to subscribe based on the URL categoryfor (String path : toCategoriesPath(url)) {
ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
ifZkListeners. PutIfAbsent (URL, new ConcurrentHashMap<>()); listeners = zkListeners.get(url); } ChildListener zkListener = listeners.get(listener); // If the zkListener cache is empty, create a cacheif (zkListener == null) {
listeners.putIfAbsent(listener, (parentPath, currentChilds) -> ZookeeperRegistry.this.notify(url, listener, toUrlsWithEmpty(url, parentPath, currentChilds)));
zkListener = listeners.get(listener);
}
zkClient.create(path, false); / / subscribe, returns the node under subpath and cache List < String >. Children = zkClient addChildListener (path, zkListener);if(children ! = null) { urls.addAll(toUrlsWithEmpty(url, path, children)); } // Call NotifyListener to update the local cache notify(URL, listener, urls); } } catch (Throwable e) { throw new RpcException("Failed to subscribe " + url + " to zookeeper " + getUrl() + ", cause: "+ e.getMessage(), e); }}Copy the codeCaching mechanism of ZooKeeper
The existence of a cache is a mechanism for trading space for time. If a Consumer first goes to the registry to pull a list of callable services for each remote invocation, the registry will be overloaded with traffic. In addition, each additional network requests will also make the performance of the whole system, at the same time service list changes the frequency of itself is not very high, unless the service provider interface to do the upgrade, or new service node or offline (from a different perspective this is not a very high frequency operation), so the pull every time also is not so necessary.
To address this problem, Dubbo’s registry implements a generic caching mechanism in the abstract class AbstractRegistry. The AbstractRegistry class structure diagram is shown below:
/** private final Properties Properties = new Properties(); /** private File File; **/ private final ConcurrentMap<URL, Map<String, List<URL>>> NOTIFIED = new ConcurrentHashMap<URL, Map<String, List<URL>>>();Copy the codeThe cache notified is ConcurrentHashMap and a Map is encapsulated, the outer Map key is the URL of the consumer and the inner Map key is the category. These routers contain providers, consumers, routers, and Configutators. Value is a service list. For urls that do not have service providers to provide services, these urls start with a prefix of empty://.
Loading the ZooKeeper cache
Upon service initialization, the AbstractRegistry constructor function loads the persistent registration data from the local disk file into the Properties object and loads it into the memory cache. The core code is as follows:
private void loadProperties() {
if(file ! = null && file.exists()) { InputStreamin= null; Try {// Reads the disk filein= new FileInputStream(file); // Write the data to the memory cache.in); ... } catch (Throwable e) {... } the finally {... }}}Copy the codeProperties holds the urls of all service providers, using URL#serviceKey() as the key, provider list, routing rule list, configuration rule list, and so on as value. The Dubbo framework automatically loads Invokers from the local cache if the application fails to connect to the registry or rock machine during startup.
Save and update the ZooKeeper cache
Caches can be saved synchronously or asynchronously. Asynchrony is saved asynchronously using the thread pool. If an exception occurs during execution, the thread pool will be called again and retry, as shown in the following code:
if(syncSaveFile){// Synchronize savedoSaveProperties(version);
} else{// asynchronously save, put into the thread pool. Will pass in a AtomicLong version number to ensure that data is the latest registryCacheExecutor. Execute (new SaveProperties (version)); }Copy the codeThe abstractregistring #notify method encapsulates the logic for updating memory and local file caches. This method is called when the client first subscribes to obtain the full amount of data, or when subsequent changes occur due to the subscribed data.