This article describes the RocketMQ cluster and how to deploy the RocketMq-on-dledger Group.
RocketMQ cluster setup
The ROcketMQ cluster can be set up in the following ways:
- “A single Master model“
- “More than the Master model“
- “Multi-master, multi-Slave – Asynchronous replication“
- “Multi-master, multi-Slave mode – Synchronous dual-write“
Among them,
“Single-master mode” is risky. “If the Broker restarts or goes down, the entire service becomes unavailable.” This is not recommended for online environments, but can be used for local testing.
“Multi-master mode” : a cluster has no slaves but all masters. During the downtime of a single machine, unconsumed messages on the machine cannot be subscribed until the machine is restored, and “the real-time performance of messages is affected”.
In the multi-master-slave asynchronous replication mode, even if the disk is damaged, the message loss is minimal and the real-time performance of the message is not affected. In addition, consumers can still consume from the Slave when the Master is down. This process is transparent to applications and does not require manual intervention, and the performance is almost the same as that in the multi-master mode. The Master outage loses a small amount of information.
When the Master is down, there is no single point of failure for data and services. The service availability and data availability are very high, and the performance is slightly lower than that of asynchronous replication (about 10% lower). RT for sending a single message is slightly higher. The standby server cannot be automatically switched to the host.
We use multi-master, multi-slave asynchronous replication mode to build RocketMQ cluster.
A dual-active and dual-slave cluster is constructed
1. Install RocketMQ on a VM
In the first installment of the RocketMQ cratering series, there was a simple installation method that I won’t go over here.
【RocketMQ series 】RocketMQ role details and basic use of practical operation
2. Set the configuration file
Perform this operation on a VM, configure the configuration file, and clone several hosts based on the VM.
Enter the Configuration File directory:
cd /usr/local/rocketmq/conf && ll
You can see
Set up two asynchronous replication brokers and enter the 2m-2s-async directory:
Modify the “primary node of the first group of brokers” configuration file broker-a. perties:
brokerClusterName=RocketMQCluster brokerName=broker-a brokerId=0 deleteWhen=04 fileReservedTime=48 brokerRole=ASYNC_MASTER flushDiskType=ASYNC_FLUSH Copy the code
NamesrvAddr = 192.168.2.170:9876; 192.168.2.171:9876; 192.168.2.172:9876; 192.168.2.173:9876
Modify the “secondary nodes of the first group of brokers” configuration file broker-a-s.perties:
brokerClusterName=RocketMQCluster
brokerName=broker-a
brokerId=1
deleteWhen=04
fileReservedTime=48
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
NamesrvAddr = 192.168.2.170:9876; 192.168.2.171:9876; 192.168.2.172:9876; 192.168.2.173:9876
Copy the codeDo the same for the master/slave profiles of the second set of brokers.
3. Key configuration items
NamesrvAddr: indicates the IP address of nameserver. Multiple IP addresses are separated by semicolons (;).
BrokerClusterName: The name of the broker cluster, which is the name of the entire broker cluster, not the name of each primary/secondary broker group. BrokerClusterName must be the same within the same cluster.
BrokerName: This is the name of each master and slave broker group. A master can have multiple slaves, but only one slave for each master, and their brokernames are the same within the master-slave group.
BrokerId: The same master-slave group uses brokerId to distinguish between master and slave. BrokerId =0 is master and greater than 1 is slave.
DeleteWhen: indicates the time when an expired file is deleted.
FileReservedTime: Commitlog and ConsumeQueue files. If non-current write files are not updated again within a certain period of time, they are considered expired and can be deleted. RocketMQ does not care whether all messages on these files are consumed.
BrokerRole: The role of the Broker.
“FlushDiskType” : flushes a disk.
4. Clone the other three VMS
After the modification is complete, stop the VMS, clone three VMS, and change the IP addresses and host names.
Final RocketMQ cluster host:
5. Start the cluster
5.1 start the nameserver
Run the following command on all four VMS:
$ROCKETMQ_HOME/bin = $ROCKETMQ_HOME/bin
$ nohup sh mqnamesrv &
Verify that the Name Server is successfully started
$ tail -f ~/logs/rocketmqlogs/namesrv.log
The Name Server boot success...
Copy the codeFor convenience (in fact, slag computer is not allowed to open so many virtual machines…) Nameserver started on four hosts, as you can see from the configuration file above:
NamesrvAddr = 192.168.2.170:9876; 192.168.2.171:9876; 192.168.2.172:9876; 192.168.2.173:9876Copy the codeNameServer instances do not communicate with each other, which is itself one of the design highlights of RocketMQ, which allows data to be out of sync between NameServer instances.
5.2 start the broker
At “192.168.2.170”, start the Master of “Broker-A” (in the bin directory of the RocketMQ installation directory)
nohup sh mqbroker -c $ROCKETMQ_HOME/conf/2m-2s-async/broker-a.properties &
Copy the codeAt 192.168.2.171, start the Master of broker-b
nohup sh mqbroker -c $ROCKETMQ_HOME/conf/2m-2s-async/broker-b.properties &
Copy the codeAt 192.168.2.172, start the Slave of Broker-A
nohup sh mqbroker -c $ROCKETMQ_HOME/conf/2m-2s-async/broker-a-s.properties &
Copy the codeAt 192.168.2.173, start the Slave of Broker-B
nohup sh mqbroker -c $ROCKETMQ_HOME/conf/2m-2s-async/broker-b-s.properties &
Copy the codeThe cluster will then start successfully.
RocketMQ-Console
To easily view the cluster status of RocketMQ, let’s install RocketMQ-Console.
Docker installation rocketMQ-Console Docker installation RocketMQ-Console Docker installation RocketMQ-Console
After the installation is complete, run the following command (for example, 192.168.2.170) :
Java jar rocketmq - the console - ng - 2.0.0. Jar -- rocketmq. Config. NamesrvAddr ="192.168.2.170:9876; 192.168.2.171:9876; 192.168.2.172:9876; 192.168.2.173:9876"
Copy the codeThen visit: http://192.168.2.170:8080:
The home page displays the Nameserver address by default.
Cluster information:
This proves that our cluster is set up successfully.
The Master cluster is down and cannot be failover
The two master and slave clusters are set up, but they do not have disaster recovery capabilities. That is, if a master fails, there is no way to elect a new master.
Disable the master (192.168.2.171) service from broker-b:
The slave node of “broker-B” cannot actively switch to master.
A new cluster Dledger
A good cluster is one that can implement Dr Automatically
Prior to RocketMQ version 4.5, RocketMQ was deployed as a Master/Slave only. There was a single Master in a group of brokers, with zero to multiple slaves. The Slave synchronizes Master data through synchronous or asynchronous replication.
This deployment pattern provides a certain degree of high availability. However, this deployment mode has certain drawbacks. For example, in the case of failover, if the primary node fails, it needs to be manually restarted or switched. A secondary node cannot be automatically converted to the primary node.
The new multi-copy architecture first needs to solve the problem of automatic failover, essentially “automatic master selection”.
There are basically two solutions to this problem:
- Use a cluster of third-party coordinating services, such as ZooKeeper or EtCD (Raft), for master selection. This solution introduces heavyweight external components that increase deployment, operation, and troubleshooting costs. For example, the RocketMQ cluster needs to be maintained as well as the ZooKeeper cluster, and a ZooKeeper cluster failure can affect the RocketMQ cluster.
- Automatic master selection is implemented using RAFT protocol. The advantage of RAFT protocol is that there is no need to introduce external components. Automatic master selection logic is integrated into the process of each node and the nodes communicate with each other to complete the master selection.
RocketMQ chose to solve this problem with the “Raft” protocol, and “DLedger is a Commitlog repository based on the RAFT protocol”, which is the key to RocketMQ’s new high-availability multi-copy architecture.
The Dledger cluster is set up
A RocketMq-on-dledger Group is a Group of “brokers of the same name” that requires at least 3 nodes. One Leader is automatically elected by Raft and the other nodes act as followers. Data is replicated between the Leader and Follower to ensure high availability.
Rocketmq-on-dledger Group Automatically implements Dr Switchover and ensures data consistency.
The RocketMq-on-dledger Group can be horizontally scaled, that is, multiple RocketMq-on-dledger groups can be deployed to provide external services at the same time.
1. Configure rocketMq-on-dledger Group
As stated above, each group of RocketMq-on-dledger requires at least 3 machines. Now we need to add 2 machines to the original group, one for each group.
Go to the dledger configuration file directory and take a look:
cd /usr/local/rocketmq/conf/dledger && ll
“broker-athen0Node configuration“
brokerClusterName = RaftCluster
brokerName=broker-a
listenPort=30911
NamesrvAddr = 192.168.2.170:9876; 192.168.2.171:9876; 192.168.2.172:9876; 192.168.2.173:9876; 192.168.2.174:9876; 192.168.2.175:9876
storePathRootDir=/tmp/rmqstore/broker-b
storePathCommitLog=/tmp/rmqstore/broker-b/commitlog
enableDLegerCommitLog=true
dLegerGroup=broker-b
DLegerPeers = n0-192.168.2.170:40911; N1-192.168.2.172:40911; N2-192.168.2.174:40911
## must be unique
dLegerSelfId=n0
sendMessageThreadPoolNums=4
Copy the codeThe configurations of n1 and N2 of broker-A are similar. Modify the dLegerSelfId configuration item.
“broker-bthen0Node configuration“
brokerClusterName = RaftCluster
brokerName=broker-b
listenPort=30911
NamesrvAddr = 192.168.2.170:9876; 192.168.2.171:9876; 192.168.2.172:9876; 192.168.2.173:9876; 192.168.2.174:9876; 192.168.2.175:9876
storePathRootDir=/tmp/rmqstore/broker-b
storePathCommitLog=/tmp/rmqstore/broker-b/commitlog
enableDLegerCommitLog=true
dLegerGroup=broker-b
DLegerPeers = n0-192.168.2.171:40911; N1-192.168.2.173:40911; N2-192.168.2.175:40911
## must be unique
dLegerSelfId=n0
sendMessageThreadPoolNums=4
Copy the codeAfter all configuration:
4. Start the cluster
4.1 start the nameserver
You can have more nameservers. Here I have nameservers for all six hosts.
4.2 start the broker
Start command:
nohup sh mqbroker -c $ROCKETMQ_HOME/conf/dledger/broker-a-n0.conf
Copy the codeNote The mapping between the configuration file and the host.
4.3 Starting the Console
Java jar rocketmq - the console - ng - 2.0.0. Jar -- rocketmq. Config. NamesrvAddr ="192.168.2.170:9876; 192.168.2.171:9876; 192.168.2.172:9876; 192.168.2.173:9876; 192.168.2.174:9876; 192.168.2.175:9876"
Copy the code
4.4 Shutting down the Master instance of Broker-A Simulates the master outage
4.5 Viewing Clusters on the Console
As you can see, the new cluster automatically elects a new Master.
4.6 Restarting the faulty instance
nohup sh mqbroker -c $ROCKETMQ_HOME/conf/dledger/broker-a-n0.conf &
Copy the code4.7 Viewing A New Instance Role
The original master crashed and became a slave after being restarted.
The end of this navigation, above.
The first public account “Xingbaili ER”, welcome old iron attention reading correction. Repository “GitHub” github.com/xblzer/Java…