“This is the 28th day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021.”

NameServer is used for service registration and discovery to prevent the single point of failure of the server from causing the entire system to collapse and become unusable. All systems in the cluster environment need a service registry. Currently zK is widely used in microservices, which is also relatively mature, but RoctetMQ wrote a set of its own. NameServer does not consider cluster operations. It only needs to discover and delete dynamic routes and pull routing information. NameServer machines do not communicate with each other. Each NameServer needs to register its own information when the Broker is started. The logical deployment diagram is as follows:

Each Broker registers its own information with each NameServer when it is started. The message producer obtains the address list of the Broker service from NameServer before sending a message. Then the producer selects a message server from the list according to the load balancing algorithm. The producer similarly pulls the address list of the Broker service from NameServer, gets the address of the Broker, and then uses a load algorithm to pull messages from the Broker. Initially, NameServer has the following functions:

  1. Maintains routing metadata information for the Broker
  2. Broker service route registration function
  3. When a Broker goes down, NameServer needs to detect and automatically cull it
  4. The heartbeat mechanism needs to know if the Broker is alive
  5. Producers and consumers pull routing information

How does NameServer implement these features?

Start main process

org.apache.rocketmq.namesrv.NamesrvStartup#main

In the main method, just two lines of core code create a NamesrvController and start it

NamesrvController controller = createNamesrvController(args);
start(controller);
Copy the code

Create NamesrvController

org.apache.rocketmq.namesrv.NamesrvStartup#createNamesrvController

There are two main things you do in creating the NamesrvController:

  1. According to the set configuration file path address, parse out the parameter configuration, and then fill: NamesrvConfig, NettyServerConfig property values

You can view the attributes of NamesrvConfig and NettyServerConfig using the debug function

  • NamesrvConfig: indicates basic configuration information

  • NettyServerConfig: indicates the configuration of the Netty service

  1. Creating NamesrvController returns after setting the log context

Start the service

org.apache.rocketmq.namesrv.NamesrvStartup#start

When the service is started, three methods of NamesrvController are called

  1. Controller.initialize () Initializes information about service startup
  2. Register the JVM hook function to shutdown the service: controller.shutdown()
  3. Controller.start () : starts the service

controller.initialize()

org.apache.rocketmq.namesrv.NamesrvController#initialize

During initialization, the following things are done

  1. Load the KV configuration from the configuration file

  1. Example Initialize the Netty service

  1. Set up the thread pool to process messages

  1. Setting up the request handler

  1. Enable scheduled task 1: Scan non-existent brokers every 10s to implement the scheduled routing message elimination function, which will be analyzed in details later
  2. Enable scheduled task 2: Prints the KV configuration every 10 seconds
  3. Set Tls connection and certificate information

controller.start()

org.apache.rocketmq.namesrv.NamesrvController#start

Start the Netty service to provide external services

controller.shutdown();

org.apache.rocketmq.namesrv.NamesrvController#shutdown

The Netty service and thread pool are disabled

Netty’s main process mainly does two things:

  1. Start the Netty service to provide external services
  2. Start the thread pool and scan non-viable brokers every 10 seconds

And then from a function point of view how are those functions implemented?

Routing information

org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager

In the RouteInfoManager class, five HashMaps are maintained, which together make up the meta information for NameServer

topicQueueTable

HashMap<String/* topic */, List> topicQueueTable

Stores information for each Topic corresponding to MessageQueue queues, one Topic corresponding to multiple message queues

brokerAddrTable

HashMap<String/* brokerName */, BrokerData> brokerAddrTable

Basic information about the brokerThe Broker master/slave address is maintained in BrokerData, HashMap<Long/* brokerId/, String/broker address */> brokerAddrs; BrokerId =0, which represents the primary Broker address, and greater than 0, which represents the secondary Broker address

clusterAddrTable

HashMap<String/* clusterName /, Set<String/ brokerName */>> clusterAddrTable

Broker cluster information that stores the names of all brokers in the cluster. One cluster corresponds to multiple brokers

brokerLiveTable

HashMap_<String/* brokerAddr */, BrokerLiveInfo> _brokerLiveTable

The lastUpdateTimestamp is updated every time a heartbeat message from the Broker is received

filterServerTable

HashMap_<String/* brokerAddr */, List<String>/* Filter Server */> _filterServerTable

A list of FilterServers in the Broker that are used to filter messages

The Broker is registered

org.apache.rocketmq.namesrv.processor.DefaultRequestProcessor#processRequest

When nettty receives the requested message, it parses the data to DefaultRequestProcessor#processRequest for processing, identifying the purpose of the message based on the code in the message body

Registered Broker message is invoked: org. Apache. Rocketmq. Namesrv. Processor. DefaultRequestProcessor# registerBrokerWithFilterServer method

In the process of registering a Broker, the following things are done:

  1. To Request to read data encapsulation Request messages to: RegisterBrokerRequestHeader, to RouteInfoManager# registerBroker with s information

When a Broker registers, it brings its own Broker information with it

  1. RouteInfoManager#registerBroker updates Broker registration information

  • Maintain clusterAddrTable metadata

Extract all brokers based on the cluster name. If the set of broker names is empty, it indicates that the clustered brokers do not exist and an empty set needs to be constructed

  • Maintain the brokerAddrTable metadata

BrokerData is extracted from the Boker according to the Broker name. If it does not exist, a new object is constructed. If it does, all IP addresses of the current Broker are found

If the primary/secondary relationship of the client changes, the metadata must also change the primary/secondary relationship

  • Maintain topicQueueTable metadata

org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#createAndUpdateQueueData

TopicQueueTable data is updated when the master Broker and the configuration of the Broker’s Topic changes

  • Maintain brokerLiveTable metadata to update the heartbeat timestamp

  1. After the update registration is complete, build response information to the client

Broker a heartbeat

The heartbeat mechanism of the Broker is the invoked Broker registration process, which is equivalent to sending a registration message at intervals, updating the timestamp each time the registration is performed

Broker does not survive culling

To eliminate non-viable brokers, NameServer uses a timed task that calls a method every 10 seconds:

org.apache.rocketmq.namesrv.routeinfo.RouteInfoManager#scanNotActiveBroker

The implementation logic for culling is simple, take out each BrokerLiveInfo that is maintained and run it through to determine if the most recent update was two minutes old: lastUpdateTimestamp, if so, cull (requiring each metadata message to be removed) and close the connection channel

Route information is pulled

When RocketMQ metadata changes, NameServer does not actively push the RocketMQ metadata to the client. Instead, the client actively pulls the latest route, using requestcode.get_routeInfo_by_topic to determine the operation method

org.apache.rocketmq.namesrv.processor.DefaultRequestProcessor#getRouteInfoByTopic

Obtain TopicRouteData information according to the TopicName and respond to the client

Based on the source code above, you can summarize the following diagram, which shows the main work of NameServer