Introduction to the

Zookeeper data synchronization -Bootstrap end, this article we will study the Nacos data synchronization principle

An overview of

Nacos data synchronization is very similar to ZooKeeper data synchronization in that it is implemented by monitoring data changes. In addition, it is incremental update in form from the data receiving status of Debug

However, websocket and ZooKeeper are recommended for synchronization because they can update incrementally and nacOS is not mentioned. When referring to HTTP long polling, the implementation of Nacos in the future is mentioned, then we guess that the internal implementation of Nacos is long polling, so the Nacos data synchronization method is not recommended. But this is just a guess, which will need to be verified by Nacos later, so let’s take a wild guess here

Through analysis, the data synchronization process of Nacos is basically as follows:

  • 1. Construct nacOS-related services
  • 2. Enable listening for the current five data types
  • 3. Call corresponding SUBSCRIBE to update data when receiving changes

The data update code of Nacos is somewhat confusing. When receiving data changes, it first unSubscribe and then onSubscribe, which is quite confusing

And Nacos doesn’t seem to receive data deletion notifications

See the source code Debug section for detailed analysis process

The sample run

The Bootstrap side of Zookeeper data synchronization is configured. The Bootstrap side of Zookeeper data synchronization is configured. The Bootstrap side of Zookeeper data synchronization is configured, and the Bootstrap side is configured

Source code for the Debug

Start configuration Tracing

We know that the entry class of Nacos data synchronization is:

  • soul-sync-data-nacos : NacosCacheHandler

We find the corresponding class, whose constructor is as follows, and we see the familiar Subscribe keyword, which we call to update the local cache

public class NacosCacheHandler {
    public NacosCacheHandler(final ConfigService configService, final PluginDataSubscriber pluginDataSubscriber,
                             final List<MetaDataSubscriber> metaDataSubscribers,
                             final List<AuthDataSubscriber> authDataSubscribers) {
        this.configService = configService;
        this.pluginDataSubscriber = pluginDataSubscriber;
        this.metaDataSubscribers = metaDataSubscribers;
        this.authDataSubscribers = authDataSubscribers;
    }

    protected void updatePluginMap(final String configInfo) {
        try {
            // Fix bug #656(https://github.com/dromara/soul/issues/656)
            List<PluginData> pluginDataList = new ArrayList<>(GsonUtils.getInstance().toObjectMap(configInfo, PluginData.class).values());
            pluginDataList.forEach(pluginData -> Optional.ofNullable(pluginDataSubscriber).ifPresent(subscriber -> {
                subscriber.unSubscribe(pluginData);
                subscriber.onSubscribe(pluginData);
            }));
        } catch (JsonParseException e) {
            log.error("sync plugin data have error:", e); }}}Copy the code

Let’s break the constructor above to see its call stack

When we come to the following class, we see that it inherits NacosCacheHandler. After constructing corresponding data, start is started. In its function, we see a function very similar to ZooKeeper listener, and we guess that this function is the listener. According to the previous experience, there should be some initialization work after the change monitor, we will see the specific debugging. Continue to break the constructor and look up

public class NacosSyncDataService extends NacosCacheHandler implements AutoCloseable.SyncDataService {

    public NacosSyncDataService(final ConfigService configService, final PluginDataSubscriber pluginDataSubscriber,
                                final List<MetaDataSubscriber> metaDataSubscribers, final List<AuthDataSubscriber> authDataSubscribers) {

        super(configService, pluginDataSubscriber, metaDataSubscribers, authDataSubscribers);
        start();
    }

    public void start(a) {
        watcherData(PLUGIN_DATA_ID, this::updatePluginMap);
        watcherData(SELECTOR_DATA_ID, this::updateSelectorMap);
        watcherData(RULE_DATA_ID, this::updateRuleMap);
        watcherData(META_DATA_ID, this::updateMetaDataMap);
        watcherData(AUTH_DATA_ID, this::updateAuthMap);
    }

    @Override
    public void close(a) { LISTENERS.forEach((dataId, lss) -> { lss.forEach(listener -> getConfigService().removeListener(dataId, GROUP, listener)); lss.clear(); }); LISTENERS.clear(); }}Copy the code

After the above constructor is broken, we trace the call stack to the familiar Spring configuration, where nacOS-related stuff is configured and Nacos listening is started when the nacosSyncDataService is constructed

public class NacosSyncDataConfiguration {

    @Bean
    public SyncDataService nacosSyncDataService(final ObjectProvider<ConfigService> configService, final ObjectProvider<PluginDataSubscriber> pluginSubscriber,
                                           final ObjectProvider<List<MetaDataSubscriber>> metaSubscribers, final ObjectProvider<List<AuthDataSubscriber>> authSubscribers) {
        log.info("you use nacos sync soul data.......");
        return new NacosSyncDataService(configService.getIfAvailable(), pluginSubscriber.getIfAvailable(),
                metaSubscribers.getIfAvailable(Collections::emptyList), authSubscribers.getIfAvailable(Collections::emptyList));
    }

    @Bean
    public ConfigService nacosConfigService(final NacosConfig nacosConfig) throws Exception {
        Properties properties = new Properties();
        if(nacosConfig.getAcm() ! =null && nacosConfig.getAcm().isEnabled()) {
            properties.put(PropertyKeyConst.ENDPOINT, nacosConfig.getAcm().getEndpoint());
            properties.put(PropertyKeyConst.NAMESPACE, nacosConfig.getAcm().getNamespace());
            properties.put(PropertyKeyConst.ACCESS_KEY, nacosConfig.getAcm().getAccessKey());
            properties.put(PropertyKeyConst.SECRET_KEY, nacosConfig.getAcm().getSecretKey());
        } else {
            properties.put(PropertyKeyConst.SERVER_ADDR, nacosConfig.getUrl());
            properties.put(PropertyKeyConst.NAMESPACE, nacosConfig.getNamespace());
        }
        return NacosFactory.createConfigService(properties);
    }

    @Bean
    @ConfigurationProperties(prefix = "soul.sync.nacos")
    public NacosConfig nacosConfig(a) {
        return newNacosConfig(); }}Copy the code

Data initialization and listening

Let’s go back and see the logic of watcherData by canceling all breakpoints and putting them in the start function

public class NacosSyncDataService extends NacosCacheHandler implements AutoCloseable.SyncDataService {

    public void start(a) {
        watcherData(PLUGIN_DATA_ID, this::updatePluginMap);
        watcherData(SELECTOR_DATA_ID, this::updateSelectorMap);
        watcherData(RULE_DATA_ID, this::updateRuleMap);
        watcherData(META_DATA_ID, this::updateMetaDataMap);
        watcherData(AUTH_DATA_ID, this::updateAuthMap); }}Copy the code

After a breakpoint restart, we are inside watcherData

At first glance we may be a little confused, but a single function general call listens for the five data changes we mentioned earlier

Through tracking debugging, it is found that it is distinguished by the type of the above function and the function passed in, so as to achieve a function universal call to monitor five kinds of data, should be its internal implementation mechanism

Call the corresponding handler function based on the data type ID passed in

public class NacosCacheHandler {

    protected void watcherData(final String dataId, final OnChange oc) {
        Listener listener = new Listener() {
            @Override
            public void receiveConfigInfo(final String configInfo) {
                // There are no updatePluginMap attributes found here
                oc.change(configInfo);
            }

            @Override
            public Executor getExecutor(a) {
                return null; }};// The initialization operation is triggered to synchronize data from the full volume
        oc.change(getConfigAndSignListener(dataId, listener));
        // Add it to the listener list
        LISTENERS.getOrDefault(dataId, new ArrayList<>()).add(listener);
    }

    protected interface OnChange {
        void change(String changeData); }}Copy the code

We play in the update plug-in data function below on breakpoints, found that for the first time you start initialization and modify the Admin backstage after the plug-in data management interface, will all come to the following functions:

public class NacosCacheHandler {

    protected void updatePluginMap(final String configInfo) {
        try {
            // Fix bug #656(https://github.com/dromara/soul/issues/656)
            List<PluginData> pluginDataList = new ArrayList<>(GsonUtils.getInstance().toObjectMap(configInfo, PluginData.class).values());
            pluginDataList.forEach(pluginData -> Optional.ofNullable(pluginDataSubscriber).ifPresent(subscriber -> {
                // Delete data first, then update data again
                subscriber.unSubscribe(pluginData);
                subscriber.onSubscribe(pluginData);
            }));
        } catch (JsonParseException e) {
            log.error("sync plugin data have error:", e); }}}Copy the code

This is a function that updates plugin information, serializes data when it is received, and then calls the corresponding SUBSCRIBE

But we found a strange logic. It calls the delete logic first and then the update logic. As we know above, when the plug-in data is updated, they all end up in the subscribeDataHandler function logic below

Update and delete are only operations on the Map in the local cache, and unSubscribe delete data does not work under the above un-on-data operation

In addition, data update does not need to be deleted and then added, but can be replaced by PUT instead of being deleted

That is, this unSubscribe may be invalid and redundant

public class CommonPluginDataSubscriber implements PluginDataSubscriber {
   
    @Override
    public void onSubscribe(final PluginData pluginData) {
        subscribeDataHandler(pluginData, DataEventTypeEnum.UPDATE);
    }
    
    @Override
    public void unSubscribe(final PluginData pluginData) {
        subscribeDataHandler(pluginData, DataEventTypeEnum.DELETE);
    }
    
    private <T> void subscribeDataHandler(final T classData, final DataEventTypeEnum dataType) {
        Optional.ofNullable(classData).ifPresent(data -> {
            if (data instanceof PluginData) {
                PluginData pluginData = (PluginData) data;
                if (dataType == DataEventTypeEnum.UPDATE) {
                    // The Map will be updated eventually
                    BaseDataCache.getInstance().cachePluginData(pluginData);
                    Optional.ofNullable(handlerMap.get(pluginData.getName())).ifPresent(handler -> handler.handlerPlugin(pluginData));
                } else if (dataType == DataEventTypeEnum.DELETE) {
                    BaseDataCache.getInstance().removePluginData(pluginData);
                    Optional.ofNullable(handlerMap.get(pluginData.getName())).ifPresent(handler -> handler.removePlugin(pluginData));
                }
            else{... }}); }}Copy the code

After further debugging and analysis, Nacos does not seem to be able to listen for data deletion events, which means that if the data fails and is useless, the invalid data will continue to occupy memory without Bootstrap being restarted

If so, the Nacos synchronization mechanism has some problems

conclusion

This paper analyzes the general data synchronization principle of Nacos and knows that the synchronization process is as follows:

  • NacosSyncDataConfiguration: Nacos launch configuration
  • NacosSyncDataService
    • 1. Initialize full data and refresh the local cache
    • 2. Enable data change monitoring
    • 3. Receive the change data and call the corresponding subScribe for the corresponding update

At the same time, we found that there may still be problems:

  • 1. Delete data cannot be received
  • 2. The unSubscribe function may be useless and redundant in the data change listening handler