The Soul gateway learns Nacos data synchronization

Author: “Li Quan”

This article analyzes the principle of Nacos synchronization data

1. Configure the environment first

soul-admin

soul-admin/src/main/resources/application.yml

soul:
  sync:
      nacos:
        url: localhost:8848
        namespace: 1c10d748-af86-43b9-8265-75f487d20c6c
  # acm:
  # enabled: false
  # endpoint: acm.aliyun.com
  # namespace:
  # accessKey:
  # secretKey:
Copy the code

Soul-admin /pom.xml, which is available by default

<dependency>
    <groupId>com.alibaba.nacos</groupId>
    <artifactId>nacos-client</artifactId>
    <version>${nacos-client.version}</version>
</dependency>
Copy the code

soul-bootstrap

soul-bootstrap/src/main/resources/application-local.yml

soul :
    sync:
        nacos:
          url: localhost:8848
          namespace: 1c10d748-af86-43b9-8265-75f487d20c6c
# acm:
# enabled: false
# endpoint: acm.aliyun.com
# namespace:
# accessKey:
# secretKey:
Copy the code

Soul-bootstrap /pom. XML, the following configuration is not available by default, you need to add it manually

<dependency>
    <groupId>org.dromara</groupId>
    <artifactId>soul-spring-boot-starter-sync-data-nacos</artifactId>
    <version>${project.version}</version>
</dependency>
Copy the code

Start the service

Nacos; soul-admin; soul-bootstrapCopy the code

Soul-bootstrap fails to start, and a null pointer exception is reported.

Soul-admin will not automatically synchronize gateway data to nacOS. This problem stumped me for a long time. Finally, I saw that other students in the group had encountered the same problem and solved it by referring to their articles. The following is a record of the solution process.

(a) A NullPointerException was encountered during soul-bootstrap startup.

When soul-bootstrap starts, nacOS gets the gateway data. If you see the breakpoint below, you get empty data.

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled. The 2021-01-25 16:49:06. 052 ERROR 5273 - [the main] O.S.B oot. SpringApplication: Application run failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'nacosSyncDataService' defined in class path resource [org/dromara/soul/springboot/starter/sync/data/nacos/NacosSyncDataConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.dromara.soul.sync.data.api.SyncDataService]: Factory method 'nacosSyncDataService' threw exception; nested exception is java.lang.NullPointerException ...... at org.dromara.soul.bootstrap.SoulBootstrapApplication.main(SoulBootstrapApplication.java:37) [classes/:na] Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.dromara.soul.sync.data.api.SyncDataService]: Factory method 'nacosSyncDataService' threw exception; nested exception is java.lang.NullPointerException at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~ [spring beans - 5.2.2. RELEASE. The jar: 5.2.2. RELEASE] the at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:651) ~ [spring beans - 5.2.2. RELEASE. The jar: 5.2.2. RELEASE]... 19 common frames omitted Caused by: java.lang.NullPointerException: null at org.dromara.soul.sync.data.nacos.handler.NacosCacheHandler.updateMetaDataMap(NacosCacheHandler.java:128) ~[classes/:na] at org.dromara.soul.sync.data.nacos.handler.NacosCacheHandler.watcherData(NacosCacheHandler.java:167) ~[classes/:na] at org.dromara.soul.sync.data.nacos.NacosSyncDataService.start(NacosSyncDataService.java:59) ~[classes/:na] at org.dromara.soul.sync.data.nacos.NacosSyncDataService.<init>(NacosSyncDataService.java:49) ~[classes/:na] at org.dromara.soul.springboot.starter.sync.data.nacos.NacosSyncDataConfiguration.nacosSyncDataService(NacosSyncDataConfigu ration.java:66) ~[classes/:na] at org.dromara.soul.springboot.starter.sync.data.nacos.NacosSyncDataConfiguration$$EnhancerBySpringCGLIB$$cce084b7.CGLIB$na cosSyncDataService$0(<generated>) ~[classes/:na] at org.dromara.soul.springboot.starter.sync.data.nacos.NacosSyncDataConfiguration$$EnhancerBySpringCGLIB$$cce084b7$$FastCla ssBySpringCGLIB$$3830e886.invoke(<generated>) ~[classes/:na] at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) ~ [spring - core - 5.2.2. The jar: 5.2.2. RELEASE] the at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnha Ncer. Java: 363) ~ [spring - the context - 5.2.2. The jar: 5.2.2. RELEASE] the at org.dromara.soul.springboot.starter.sync.data.nacos.NacosSyncDataConfiguration$$EnhancerBySpringCGLIB$$cce084b7.nacosSyn cDataService(<generated>) ~[classes/:na] ...... at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ~ [spring beans - 5.2.2. RELEASE. The jar: 5.2.2. RELEASE]... 20 common frames omittedCopy the code

(b) Go to nacOS to check whether there is gateway data, according to the configuration of “Namespace: 1C10D748-AF86-43b9-8265-75f487d20c6c” result is nothing.

3, try soul-admin manual synchronization, nacOS can not see the data, you must manually create a namespace 1c10D748-AF86-43b9-8265-75f487d20c6c, as shown in the following figure.

4. Soul-bootstrap still fails to start because metadata is missing.

Metadata only dubbo and SpringCloud services have data, but HTTP does not have metadata, so you need to start the Dubbo service. Then synchronize the metadata in soul-admin.

Soul-admin clicks synchronize data to synchronize metadata to NacOS

Soul-admin clicks Synchronize data to synchronize authentication data to NACOS

At this point, NACOS has seen all the gateway data

5. Start soul-Bootstrap again, and finally it works

The 2021-01-25 17:56:54. 10051-798 the INFO [main] O.D.S.W.C onfiguration. SoulConfiguration: The load plugins: [monitor] [org. Dromara. Soul. Plugin. Monitor. MonitorPlugin] 2021-01-25 17:56:54. 10051-798 the INFO [main] o.d.s.w.configuration.SoulConfiguration : Load the plugin: [response] [org. Dromara. Soul. Plugin. Httpclient. Response. WebClientResponsePlugin] 2021-01-25 17:56:54. 990 INFO 10051 --- [ main] d.s.s.s.s.d.n.NacosSyncDataConfiguration : you use nacos sync soul data....... The 2021-01-25 17:56:58. 10051-890 the INFO [main] O.S.B.A.E.W eb. EndpointLinksResolver: Exposing 2 to the Exposing 2 endpoint(S) : DRIVE to the rim of the RING '/ skeletonoid '2021-01-25 17:56:52.75info o.s.b.web.embedded.netty.NettyWebServer : Netty started on port(s): 9195 the 2021-01-25 17:56:59. 10051-764 the INFO [main] O.D.S.B.S oulBootstrapApplication: Started SoulBootstrapApplication in 8.401 seconds (JVM running for 9.95)Copy the code

6. Summary:

Soul-admin does not actively synchronize gateway data to NacOS, so manual synchronization is required. Soul-bootstrap must rely on all gateway configuration data: soul.plugin, soul.selector, soul.selector, soul.meta, soul.auth. Soul-bootstrap will not start if the gateway only proxies HTTP services (with no metadata). The official website did not do this piece of detailed explanation, is not very friendly to xiao Bai.

We know that soul-Admin will not automatically synchronize data to NACOS once it is started, so it needs to be done manually.

Soul-admin, NACos, and soul-bootstrap synchronize data.

How does soul-admin synchronize gateway data?

1. A DataChangedEvent event will be published after the plugin information is updated

/**
 * create or update plugin
 * @param pluginDTO {@linkplain PluginDTO}
 * @return rows
 */
@Override
@Transactional(rollbackFor = Exception.class)
public String createOrUpdate(final PluginDTO pluginDTO) {...// publish change event.
    eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.PLUGIN, eventType,
            Collections.singletonList(PluginTransfer.INSTANCE.mapToData(pluginDO))));
    return StringUtils.EMPTY;
}
Copy the code

2, by listening to the event handler class DataChangedEventDispatcher is responsible for calling the specific monitoring implementation class of DataChangedEvent event for processing, the specific implementation class is NacosDataChangedListener here.

org.dromara.soul.admin.listener.DataChangedEventDispatcher

DataChangedEventDispatcher initialization complete executes the afterPropertiesSet (), to capture all types in the container is DataChangedListener. The bean class


@Override
public void afterPropertiesSet(a) {
    Collection<DataChangedListener> listenerBeans = applicationContext.getBeansOfType(DataChangedListener.class).values();
    this.listeners = Collections.unmodifiableList(new ArrayList<>(listenerBeans));
}
Copy the code

DataChangedEventDispatcher after listening to change events, will perform onApplicationEvent, traverse all listening in class to listen for an event, here is NacosDataChangedListener, the debug of the diagram below.


@Override
@SuppressWarnings("unchecked")
public void onApplicationEvent(final DataChangedEvent event) {
    for (DataChangedListener listener : listeners) {
        switch (event.getGroupKey()) {
            ......
            case RULE:
                listener.onRuleChanged((List<RuleData>) event.getSource(), event.getEventType());
                break; .default:
                throw new IllegalStateException("Unexpected value: "+ event.getGroupKey()); }}}Copy the code

NacosDataChangedListener executes onRuleChanged, updateRuleMap synchronizes gateway data to memory, and then to NacOS via publishConfig.

org.dromara.soul.admin.listener.nacos.NacosDataChangedListener

// Execute the listening event
@Override
public void onRuleChanged(final List<RuleData> changed, final DataEventTypeEnum eventType) {
    updateRuleMap(getConfig(RULE_DATA_ID));
    switch (eventType) {
        ......
        default:
            changed.forEach(rule -> {
                List<RuleData> ls = RULE_MAP
                        .getOrDefault(rule.getSelectorId(), newArrayList<>()) .stream() .filter(s -> ! s.getId().equals(rule.getSelectorId())) .sorted(RULE_DATA_COMPARATOR) .collect(Collectors.toList()); ls.add(rule); RULE_MAP.put(rule.getSelectorId(), ls); });break;
    }
    publishConfig(RULE_DATA_ID, RULE_MAP);
}
// Synchronize to memory
private void updateRuleMap(final String configInfo) {
    JsonObject jo = GsonUtils.getInstance().fromJson(configInfo, JsonObject.class);
    Set<String> set = newHashSet<>(RULE_MAP.keySet()); . RULE_MAP.keySet().removeAll(set); }// Synchronize to nacOS
@SneakyThrows
private void publishConfig(final String dataId, final Object data) {
    configService.publishConfig(dataId, GROUP, GsonUtils.getInstance().toJson(data));
}
Copy the code

4, DataChangedEventDispatcher, NacosDataChangedListener class inheritance

5, summary

1, update the data in the gateway admin, such as soul – released a DataChangedEvent event, eventPublisher. PublishEvent (new DataChangedEvent ())

2, DataChangedEventDispatcher – > onApplicationEvent () method to monitor the event to the event, the judgment is NacosDataChangedListener listening class

NacosDataChangedListener –> onRuleChanged() handles the event

UpdateRuleMap (RULE_DATA_ID)

5. Synchronize to nacOS publishConfig(RULE_DATA_ID, RULE_MAP)

How does soul-bootstrap synchronize gateway data?

1, the soul – the bootstrap added nacos rely on soul – spring – the boot – starter – sync – data – nacos, injected NacosSyncDataConfiguration service starts automatically

org.dromara.soul.springboot.starter.sync.data.nacos.NacosSyncDataConfiguration

NacosSyncDataService is responsible for reading and synchronizing nacOS gateway data

@Configuration
@ConditionalOnClass(NacosSyncDataService.class)
@ConditionalOnProperty(prefix = "soul.sync.nacos", name = "url")
@Slf4j
public class NacosSyncDataConfiguration {
    // Inject nacOS data synchronization service
    @Bean
    public SyncDataService nacosSyncDataService(final ObjectProvider<ConfigService> configService, final ObjectProvider<PluginDataSubscriber> pluginSubscriber,
                                           final ObjectProvider<List<MetaDataSubscriber>> metaSubscribers, final ObjectProvider<List<AuthDataSubscriber>> authSubscribers) {
        log.info("you use nacos sync soul data.......");
        return new NacosSyncDataService(configService.getIfAvailable(), pluginSubscriber.getIfAvailable(),
                metaSubscribers.getIfAvailable(Collections::emptyList), authSubscribers.getIfAvailable(Collections::emptyList));
    }
    // Inject the NACOS client configuration service
    @Bean
    public ConfigService nacosConfigService(final NacosConfig nacosConfig) throws Exception {
        Properties properties = newProperties(); .return NacosFactory.createConfigService(properties);
    }
    // Inject the NACOS configuration service
    @Bean
    @ConfigurationProperties(prefix = "soul.sync.nacos")
    public NacosConfig nacosConfig(a) {
        return newNacosConfig(); }}Copy the code

2, org. Dromara. Soul. Sync. Data. Nacos. NacosSyncDataService

Initialization executes start

WatcherData listens for NACOS gateway data

UpdatePluginMap synchronizes gateway data to memory

public void start(a) {... watcherData(RULE_DATA_ID,this::updateRuleMap); . }@SneakyThrows
private String getConfigAndSignListener(final String dataId, final Listener listener) {
    return configService.getConfigAndSignListener(dataId, GROUP, 6000, listener);
}
protected void watcherData(final String dataId, final OnChange oc) {
    Listener listener = new Listener() {
        @Override
        public void receiveConfigInfo(final String configInfo) { oc.change(configInfo); }... }; oc.change(getConfigAndSignListener(dataId, listener)); LISTENERS.getOrDefault(dataId,new ArrayList<>()).add(listener);
}
Copy the code

3. NacosSyncDataService class diagram

4, summary

1, the soul – the bootstrap start NacosSyncDataConfiguration injections into the container automatically

2, injected NacosSyncDataService NacosSyncDataConfiguration class to the container

NacosSyncDataService –> start() –> watcherData() listens on nacOS and synchronizes gateway data to memory

4, watcherData() –> pluginmap ()

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

The Soul gateway learns Nacos data synchronization

The Soul gateway learns Nacos data synchronization

How does soul-admin synchronize gateway data?

How does soul-bootstrap synchronize gateway data?

conclusion

The Soul gateway learns Nacos data synchronization

The Soul gateway learns Nacos data synchronization

How does soul-admin synchronize gateway data?

How does soul-bootstrap synchronize gateway data?

conclusion

Related Posts

JVM Series: Macro Analysis of How Java Code executes

Filebeat +kafka+graylog+es+mongodb

Spring Cloud Alibaba service registry Nacos