In previous articles, we have covered some of the source code and core mechanisms of Broker, Producer, and Conusmer, but there is another key component of RocketMQ that we often overlook — NameServer.

In daily use, we mostly contact with producers and consumers, while NameServer has no direct interaction with us. Just like the Zookeeper cluster behind the Kafka cluster for its cluster metadata management, NameServer is behind RocketMQ.

What is NameServer

NameServer you can simply think of it as a registry.

When a Broker starts, it registers itself with NameServer and sends its IP address, port, and RocketMQ cluster routes to NameServer along with its heartbeat. The routing information here refers to the Topic MessageQueue on which Broker.

On the other hand, the Producer gets metadata from the NameServer and sends messages to the Broker.

Accordingly, the Consumer also needs to get data from NameServer. Usually when we configure consumers, there are mainly two important information, which are the Topic you want to consume and the current Consumer Group. According to the configuration, the Consumer will go to NameServer to find out which brokers are available for the specific Topic, and what their real IP addresses and ports are. After obtaining this, the Consumer can start to consume messages.

What does the registered Broker do

Here we’ll warm up by registering the Broker’s source code and go straight to the code in preparation for reading the entire section of the source code later.

There is a difference between the Broker versions and the different versions. Since the latest version of the website is 4.9.0, we will not consider the earlier version for the time being. We will discuss this later.

Only a few lines of the above code will be posted to you, and the rest I will try to use flowcharts instead

Verify integrity of the Body

The first is to verify the integrity of the data sent from the Broker. A simple judgment is that the Body sent by the Broker is encrypted using the CRC32 algorithm and compared to the encrypted value in the request Header. If the Body is not encrypted by the Broker, the data integrity is broken and the registration process should be interrupted.

Parsing the Body

There are two cases:

  • The Body is empty
  • The Body is not null

If Body is empty, the datteaching for the currently registered Broker is reset;

**Body is not empty **Body is parsed, mainly to parse out datteaching, which represents the version of the data in the Broker. The next step is to parse out all topics stored in the Broker and their associated configurations.

Execute the registration logic

So this is the core logic of registration, but to make it easier to understand, we’re going to talk about it separately, and we’re not going to put the two together.

  • To register for the first time
  • Non-initial registration

Maintain the Name of the Broker in the cluster

Before the operation begins, a lock is placed on the RouteInfoManager, where NameServer stores data. This lock is a read-write lock that uses Java’s ReentrantReadWriteLock.

BrokerName here is a variable configured in the RocketMQ configuration file. This is the Name used to identify a Broker, but we know that brokers have master-slave architecture, and the Dleger introduced after RocketMQ 4.5 can implement one master-slave, in other words, a Broker Name may be used for multiple Broker instances.

From MQ’s point of view, brokers are multi-instance deployed; In the case of a Producer or Consumer, there is only one Broker. So, what is maintained in this step is how many such Broker names are in the current cluster.

Maintain the Broker’s data

RocketMQ then maintains the core data for each Broker in brokerAddrTable, including:

  • The cluster in which the Broker resides
  • Broker name (just discussed above)
  • BrokerID and Address mapping for all brokers is a Map where the Address is IP+ port

Why there are multiple addresses under the same Broker Name has been explained in the previous step and will not be explained here.

There are two main aspects of data maintenance for the Broker:

  • The Broker data is inbrokerAddrTableExists in
  • brokerAddrTable Do not maintain duplicate address information in the data

The first one is too basic to be repeated. Focusing on the second point, we know that there will be multiple Broker addresses in a Map because brokers are based on a master-slave architecture. Have you ever wondered how NameServer distinguishes between master and slave?

The Key of the Map is 0 for the Master node and 1 for the Slave node. RocketMQ implements a Master and Slave Broker architecture. One master with many slaves is implemented by Dleger, which was added after RocketMQ 4.5, and will not be discussed for the time being. The distinction logic is as follows:

So when does that happen?

The answer is a master-slave switch

For example, assume that the Address of a Slave Broker is 192.168.1.101:8081 and is registered. BrokerAddrs now has a key: 1 value: 192.168.1.101:8081 record.

If the Master in the cluster is down, it will recover. If the new Master is selected, it will be found that brokerAddrs already have the same Address, but with a different Key. However, since they are essentially the same machine, if you do not remove the records whose key is 1 and whose role is Slave, you will cause data consistency problems.

To summarize, only one Adreess can exist in brokerAddrs. If you are interested, you can see the source code, but the logic is the same as the text described above.

Once the duplicate Address data is removed, the data for the registered Broker is registered into brokerAddrs.

Maintain MessageQueue data

This is to update the data related to its MessageQueue based on the Broker’s data. Next, let’s take a look at the maintenance process of the Message Queue in detail, as well as the source code and the flow chart. The two parts are equivalent and optional.

When the Master node is registered, a method createAndUpdateQueueData is called to maintain the data related to the MessageQueue if it is registered for the first time or if the data has been updated. The determination of whether the data is updated is based on DataVersion, which represents the Broker’s version of the data.

After that, we can get a list of the corresponding MessageQueue by using the Name of the Topic. It may be a bit of a question here, shouldn’t a Topic only have one configuration related to MessageQueue? Why is this a list?

It’s smaller. The pattern is smaller

Topic is a logical concept. A Topic MessageQueue is distributed across different brokers, so here is a list.

The update process is shown in the figure above. After getting the list of MessageQueue, it will make a comparison with the MessageQueue data in the registered Broker. If the difference is found, it will make a full replacement, and there is no other complicated comparison logic. The source code is the same as above, interested can view.

Maintains the Broker’s survival information

At this point, the logic for the MessageQueue is processed, and NameServer will then update the data in brokerLiveTable, which holds all brokers currently active. And we’ll talk about what this does later.

NameServer starts the process

This gives you an overview of the NameServer architecture by looking at the process of registering brokers, and then taking a look at NameServer as a whole.

The overall process above this diagram has been given out, do not put the source code, not significant.

This background thread performs a scan of inactive brokers every 10 seconds, traversing the brokerLiveTable mentioned above because it maintains all active brokers.

If a Broker has not sent a heartbeat to NameServer for more than 120 seconds, it will be removed from brokerLiveTable.

Operations that NameServer can handle

There are many other operations that NameServer supports, but they are not listed here. If you are interested, you can find a lot of information online. Besides, the operation Register Broker involves data structures in the source code, which will be used in other operations, so it will be easy to read the source code for other operations after you have learned about Register Broker.

Ok, that is all the content of this blog, welcome to wechat search concern [SH full stack notes], reply [queue] get MQ learning materials, including basic concept analysis and RocketMQ detailed source analysis, continuous updates.

If you think this article is helpful to you, please click a like, close a note, share, leave a message.