The Soul gateway learns Zookeeper data Synchronization. 01

Author: Li Quan

Start soul-admin and soul-bootstrap, and use ZooKeeper to synchronize data to the gateway

First, configure the environment

1. The soul-admin service is configured and needs to be restarted

soul-admin / src / main / resources / application.yml

soul:
  sync:
      zookeeper:
          url: localhost:2181
          sessionTimeout: 5000
          connectionTimeout: 2000

Copy the code

2. The soul-bootstrap gateway service configuration needs to be restarted

soul-bootstrap / pom.xml

<! --soul data sync start use zookeeper--> <dependency> <groupId>org.dromara</groupId> <artifactId>soul-spring-boot-starter-sync-data-zookeeper</artifactId> <version>${project.version}</version> </dependency>Copy the code

soul-bootstrap / src / main / resources / application-local.yml

soul :
    sync:
        zookeeper:
             url: localhost:2181
             sessionTimeout: 5000
             connectionTimeout: 2000

Copy the code

Two, start the service

1. Start Zookeeper

zookeeper ./bin/zkServer.sh start /usr/bin/java ZooKeeper JMX enabled by default Using config: /Documents/soft/zookeeper/bin/.. /conf/zoo.cfg Starting zookeeper ... STARTEDCopy the code

2. The soul-admin gateway background service is started. After the service is started, the ZooKeeper request is invoked

The 2021-01-20 17:34:48. 64500-752 the INFO [- localhost: 2181] org. I0Itec. Zkclient. ZkEventThread: Starting ZkClient event thread. 2021-01-20 17:34:48. 761 INFO 64500 - [the main] org. Apache. They are. They are: The Client environment: zookeeper. Version = 3.5.6 - c11b7e26bc554b8523dc929761dd28808913f091, Built on 10/08/2019 prepare GMT 2021-01-20 17:34:48. 761 INFO 64500 - [the main] org. Apache. They are. They are: Client environment: the host name = 10.7.254.31 17:34:48 2021-01-20. 64500-761 the INFO [main] org. Apache. They are. They are: Client environment: Java version = 1.8.0 comes with _261 17:34:48 2021-01-20. 64500-761 the INFO [main] org. Apache. The zookeeper. The zookeeper  : Client environment:java.vendor=Oracle Corporation ...... The 2021-01-20 17:34:48. 64500-806 the INFO [localhost: 2181)] org. Apache. Zookeeper. ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown Error) 17:34:48 2021-01-20. 64500-826 the INFO [localhost: 2181)] org. Apache. The zookeeper. ClientCnxn: Socket connection established, initiating session, client: /0:0:0:0:0:0:0:1:58214, server: Localhost / 0:0:0:0:0:0:0:1:21 81 17:34:48 2021-01-20. 64500-857 the INFO [localhost: 2181)] org. Apache. The zookeeper. ClientCnxn : Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x1000b5e22f50001, Negotiated timeout = 5000 2021-01-20 17:34:48. 64500-861 the INFO/ain - EventThread org. I0Itec. Zkclient. Zkclient: zookeeper state changed (SyncConnected)Copy the code

3. The soul-bootstrap gateway service is started. After the service is started, the ZooKeeper request is invoked

The 2021-01-20 17:35:58. 64583-996 the INFO [the main] S.B.S.D.Z.Z ookeeperSyncDataConfiguration: you use zookeeper sync soul data....... The 2021-01-20 17:35:59. 64583-003 the INFO [- localhost: 2181] org. I0Itec. Zkclient. ZkEventThread: Starting ZkClient event thread. ...... The 2021-01-20 17:35:59. 64583-012 the INFO [main] org. Apache. They are. They are: Client environment: the user home = / Users/liquan 17:35:59 2021-01-20. 64583-012 the INFO [main] org. Apache. The zookeeper. The zookeeper : Client environment: OS. Memory. Total = 310 MB 17:35:59 2021-01-20. 64583-018 the INFO [main] org. Apache. They are. They are:  Initiating client connection, ConnectString = localhost: 2181 5000 watcher sessionTimeout = = org. I0Itec. Zkclient. Zkclient @ 114 a5e0 17:35:59 2021-01-20. 121 INFO 64583 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : The Session establishment complete on server localhost / 127.0.0.1:2181, sessionid = 0 x1000b5e22f50002, Negotiated timeout = 5000 2021-01-20 17:35:59. 64583-126 the INFO/ain - EventThread org. I0Itec. Zkclient. Zkclient: zookeeper state changed (SyncConnected)Copy the code

4. Check the synchronized registration information of the soul gateway on Zookeeper

Zookeeper data synchronization principle of the Soul gateway

After the soul – admin start org. In the console I0Itec. Zkclient. Zkclient, tracking and debugging for entry.

1. ZookeeperConfiguration registers zkClient with the Spring container.

/ / EnableConfigurationProperties role: use @ ConfigurationProperties annotation class to take effect. If a configuration class configates only the @ConfigurationProperties annotation and does not use @Component, then the beans transformed by the Properties configuration file are not available in the IOC container. @ EnableConfigurationProperties is equivalent to the use of the @ ConfigurationProperties class conducted an injection. ConditionalOnMissingBean (); // ConditionalOnMissingBean (); // ConditionalOnMissingBean (); ConditionalOnBean /** * ZookeeperConfiguration. * @author xiaoyu(Myth) */ @EnableConfigurationProperties(ZookeeperProperties.class) public class ZookeeperConfiguration { /** * register zkClient in spring ioc. * * @param zookeeperProp the zookeeper configuration * @return ZkClient {@linkplain ZkClient} */ @Bean @ConditionalOnMissingBean(ZkClient.class) public ZkClient zkClient(final ZookeeperProperties zookeeperProp) { return new  ZkClient(zookeeperProp.getUrl(), zookeeperProp.getSessionTimeout(), zookeeperProp.getConnectionTimeout()); }}Copy the code

Soul-admin reads zooKeeper configuration information and injects zkClient into the container to establish a connection with ZooKeeper.

2, instantiation ZkClient call stack will call DataChangedEventDispatcher afterPropertiesSet method.

Org. Dromara. Soul. Admin. Listener. DataChangedEventDispatcher role: event repeater, forwarded to each ConfigEventListener of events that will change.

Such InitializingBean is achieved, in the process of DataChangedEventDispatcher initialization, executes the afterPropertiesSet method.

AfterPropertiesSet method would be to look for in the container type is DataChangedListener. The bean class.

@Component public class DataChangedEventDispatcher implements ApplicationListener<DataChangedEvent>, InitializingBean { private ApplicationContext applicationContext; private List<DataChangedListener> listeners; public DataChangedEventDispatcher(final ApplicationContext applicationContext) { this.applicationContext = applicationContext; } @Override @SuppressWarnings("unchecked") public void onApplicationEvent(final DataChangedEvent event) { for (DataChangedListener listener : listeners) { switch (event.getGroupKey()) { case APP_AUTH: listener.onAppAuthChanged((List<AppAuthData>) event.getSource(), event.getEventType()); break; . default: throw new IllegalStateException("Unexpected value: " + event.getGroupKey()); }}... @Override public void afterPropertiesSet() { Collection<DataChangedListener> listenerBeans = applicationContext.getBeansOfType(DataChangedListener.class).values(); this.listeners = Collections.unmodifiableList(new ArrayList<>(listenerBeans)); }}Copy the code

3, the afterPropertiesSet method execution will find DataChangedListener. The class instantiation of related classes.

Org. Dromara. Soul. Admin. Config. DataSyncConfiguration role: data synchronization configuration class.

ZookeeperDataChangedListener data change listener, role: metadata changes should be monitored, and then synchronization to zookeeper.

ZookeeperDataInit Initializes ZooKeeper data. Function: Synchronizes initialization data to ZooKeeper.

/** * The type Zookeeper listener. */ @Configuration @ConditionalOnProperty(prefix = "soul.sync.zookeeper", name = "url") @Import(ZookeeperConfiguration.class) static class ZookeeperListener { /** * Config event listener data changed listener. * @param zkClient the zk client * @return the data changed listener */ @Bean @ConditionalOnMissingBean(ZookeeperDataChangedListener.class) public DataChangedListener zookeeperDataChangedListener(final ZkClient zkClient) { return new ZookeeperDataChangedListener(zkClient); } /** * Zookeeper data init zookeeper data init * @param zkClient the zk client * @param syncDataService the sync data service * @return the zookeeper data init */ @Bean @ConditionalOnMissingBean(ZookeeperDataInit.class) public ZookeeperDataInit zookeeperDataInit(final ZkClient zkClient, final SyncDataService syncDataService) { return new ZookeeperDataInit(zkClient, syncDataService); }}Copy the code

4, org. Dromara. Soul. Admin. The listener. The zookeeper. ZookeeperDataInit role: responsible for the initialization zookeeper synchronous data. This class implements CommandLineRunner.

CommandLineRunner: Functions as: SpringBoot iterates through all the entity classes that implement CommandLineRunner after a project is launched and executes the Run method. If you need to execute in a certain Order, you need to use an @ORDER annotation on the entity class (or implement the Order interface) to indicate that Order.

The run method calls the syncDataService. SyncAll method.

public class ZookeeperDataInit implements CommandLineRunner { private final ZkClient zkClient; private final SyncDataService syncDataService; /** * Instantiates a new Zookeeper data init. * @param zkClient the zk client * @param syncDataService the sync data service */ public ZookeeperDataInit(final ZkClient zkClient, final SyncDataService syncDataService) { this.zkClient = zkClient; this.syncDataService = syncDataService; } @Override public void run(final String... args) { String pluginPath = ZkPathConstants.PLUGIN_PARENT; String authPath = ZkPathConstants.APP_AUTH_PARENT; String metaDataPath = ZkPathConstants.META_DATA; if (! zkClient.exists(pluginPath) && ! zkClient.exists(authPath) && ! zkClient.exists(metaDataPath)) { syncDataService.syncAll(DataEventTypeEnum.REFRESH); }}}Copy the code

5, org. Dromara. Soul. Admin. Service. Sync. SyncDataServiceImpl

SyncAll method will be called the publishing of events and event type is DataEventTypeEnum REFRESH.

/** * The type sync data service. * @author xiaoyu(Myth) */ @Service("syncDataService") public class SyncDataServiceImpl Implements SyncDataService {/ / publishing events, that is, tell an event to all the related to this event listener private final ApplicationEventPublisher eventPublisher; . @Override public boolean syncAll(final DataEventTypeEnum type) { appAuthService.syncData(); List<PluginData> pluginDataList = pluginService.listAll(); eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.PLUGIN, type, pluginDataList)); List<SelectorData> selectorDataList = selectorService.listAll(); eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.SELECTOR, type, selectorDataList)); List<RuleData> ruleDataList = ruleService.listAll(); eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.RULE, type, ruleDataList)); metaDataService.syncData(); return true; }... }Copy the code

6, after the event publishing org. Dromara. Soul. Admin. Listener. Change DataChangedEventDispatcher onApplicationEvent method of a class meeting to monitor events, iterate through all the listeners data synchronization processing, Here the listener is ZookeeperDataChangedListener implementation class, depending on the type of event corresponding to the zookeeper through zkClient synchronous data.

Soul-admin initializes data to the ZooKeeper mind map

Four,

Soul-admin starts to synchronize gateway data rules, metaData, selectors, plugins, and zooKeeper. A DataChangedEvent event is published for data changes. The listening event synchronizes data to ZooKeeper.

Principle of soul Gateway data synchronization