Quickly set up a RocketMQ cluster

1. Build the machine environment

Prepare three VMS. The user name and password are root, and the IP address is as follows

192.168.232.128 worker1
192.168.232.129 worker2
192.168.232.130 worker3
Copy the code

2. Create a user

Useradd user passwd user (password 123qweasd)Copy the code

3. System configuration

Avoid close login

Switch user, generate key on worker1,ssh-kengen and distribute to other machines

ssh-copy-id worker1 
ssh-copy-id worker2
ssh-copy-id worker3
Copy the code

Disabling the Firewall

systemctl stop firewalld.service

firewall-cmd--state
Copy the code

4. Install Java

Create /app directory, upload JDK tar package, modify ~/. Bash_profile, and configure environment variables. Enable source.

Export JAVA_HOME = / app/jdk1.8 /Copy the code

5. Install RocketMQ

Upload the RocketMQ tar package and unzip it. Then configure the environment variables

Export ROCKETMQ_HOME = / app/rocketmq/rocketmq - all - 4.7.1 - bin - releaseCopy the code

RocketMQ does not support automatic slave switchover after master failure until version 4.5. After version 4.5, dleger-based master/slave switchover was added. The current version 4.7.1 is used here

6. Configure the RocketMQ cluster

It is expected to set up two asynchronous disk flushing clusters

The name of the machine NemaeServer node deployment Broker node deployment
worker1 nameserver
worker2 nameserver broker-a, broker-b-s
worker3 nameserver broker-b,broker-a-s

To modify the 2M-2s-Async configuration file, go to the Config directory of RocketMQ.

2M-2S-Async construction advantages

  • 2m-2s-Async: 2 Primary and secondary asynchronous flush (high throughput, but messages may be lost)

  • 2m-2s-sync:2 master and 2 slave synchronous flush (throughput decreases, but messages are more secure)

  • 2m-noslave:2 No master (single point of failure). You can directly configure broker.conf to configure the single point environment

  • Dleger is used to implement master/slave switchover. Nodes in the cluster randomly elect a leader based on Raft protocol and the rest are followers. This is often the way clusters are built in formal environments

1. Configure the first set of broker-A

Worker2 configures the master node of Borker-A

First configuration 2 m – 2 s – async/broker – Amy polumbo roperties

# Cluster name: all nodes with the same name are in the same cluster
brokerClusterName=rocketmq-cluster
Nodes with the same name are a set of primary and secondary nodes.
brokerName=broker-a
Brokerid,0 for Master, >0 for Slave
brokerId=0
#nameServer address, semicolon split
namesrvAddr=worker1:9876; worker2:9876; worker3:9876
# when sending a message, automatically create a topic that does not exist on the server
defaultTopicQueueNums=4
Whether to allow the Broker to create topics automatically
autoCreateTopicEnable=true
# whether to allow the Broker to automatically create a subscription set, it is recommended that the offline open, online closed autoCreateSubscriptionGroup = true
The port on which the Broker listens for external services
listenPort=10911
The default time for deleting files is 4 a.m
deleteWhen=04
The default retention time is 48 hours
fileReservedTime=120
CommitLog The default file size is 1 gb
mapedFileSizeCommitLog=1073741824
Article 30 w # ConsumeQueue each file by default to save, adjust according to the business situation mapedFileSizeConsumeQueue = 300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
Check physical file disk space
diskMaxUsedSpaceRatio=88
# storage path
storePathRootDir=/app/rocketmq/store
CommitLog storage path
storePathCommitLog=/app/rocketmq/store/commitlog
Consume queue storage path Storage path
storePathConsumeQueue=/app/rocketmq/store/consumequeue
Message index storage path
storePathIndex=/app/rocketmq/store/index
#checkpoint File storage path
storeCheckpoint=/app/rocketmq/store/checkpoint
#abort file storage path
abortFile=/app/rocketmq/store/abort
# limit the message size
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
The role of the Broker
# -async_master Asynchronous replication Master
# -sync_master Synchronizes the double write Master
#-SLAVE
brokerRole=ASYNC_MASTER
# Brush plate mode
# -async_flush Asynchronously flush disks
# -sync_flush Synchronously flush disks
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
# Number of sending thread pools
#sendMessageThreadPoolNums=128
# pull message thread pool number
#pullMessageThreadPoolNums=128
Copy the code

Worker3 configures borker-a’s Salve node, broker-a-S. perties

Only brokerId and brokerRole need to be modified

# Cluster name: all nodes with the same name are in the same cluster
brokerClusterName=rocketmq-cluster
Nodes with the same name are a set of primary and secondary nodes.
brokerName=broker-a
Brokerid,0 for Master, >0 for Slave
brokerId=1
#nameServer address, semicolon split
namesrvAddr=worker1:9876; worker2:9876; worker3:9876
# when sending a message, automatically create a topic that does not exist on the server
defaultTopicQueueNums=4
Whether to allow the Broker to create topics automatically
autoCreateTopicEnable=true
Whether to enable the Broker to automatically create subscription groups
autoCreateSubscriptionGroup=true
The port on which the Broker listens for external services
listenPort=11011
The default time for deleting files is 4 a.m
deleteWhen=04
The default retention time is 48 hours
fileReservedTime=120
CommitLog The default file size is 1 gb
mapedFileSizeCommitLog=1073741824
ConsumeQueue By default, each file holds 30W bytes, which can be adjusted according to service conditions
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
Check physical file disk space
diskMaxUsedSpaceRatio=88
# storage path
storePathRootDir=/app/rocketmq/storeSlave
CommitLog storage path
storePathCommitLog=/app/rocketmq/storeSlave/commitlog
Consume queue storage path Storage path
storePathConsumeQueue=/app/rocketmq/storeSlave/consumequeue
Message index storage path
storePathIndex=/app/rocketmq/storeSlave/index
#checkpoint File storage path
storeCheckpoint=/app/rocketmq/storeSlave/checkpoint
#abort file storage path
abortFile=/app/rocketmq/storeSlave/abort
# limit the message size
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
The role of the Broker
# -async_master Asynchronous replication Master
# -sync_master Synchronizes the double write Master
#-SLAVE
brokerRole=SLAVE
# Brush plate mode
# -async_flush Asynchronously flush disks
# -sync_flush Synchronously flush disks
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
# Number of sending thread pools
#sendMessageThreadPoolNums=128
# pull message thread pool number
#pullMessageThreadPoolNums=128
Copy the code

2. Configure the second group of broker-B

Worker3 configures the master node of Borker-b

You need to configure config/2m-2s-async/broker-b. perties on worker3

# Cluster name: all nodes with the same name are in the same cluster
brokerClusterName=rocketmq-cluster
enablePropertyFilter=true
Nodes with the same name are a set of primary and secondary nodes.
brokerName=broker-a
Brokerid,0 for Master, >0 for Slave
brokerId=1
#nameServer address, semicolon split
namesrvAddr=worker1:9876; worker2:9876; worker3:9876
# when sending a message, automatically create a topic that does not exist on the server
defaultTopicQueueNums=4
Whether to allow the Broker to create topics automatically
autoCreateTopicEnable=true
Whether to enable the Broker to automatically create subscription groups
autoCreateSubscriptionGroup=true
The port on which the Broker listens for external services
listenPort=11011
The default time for deleting files is 4 a.m
deleteWhen=04
The default retention time is 48 hours
fileReservedTime=120
CommitLog The default file size is 1 gb
mapedFileSizeCommitLog=1073741824
ConsumeQueue By default, each file holds 30W bytes, which can be adjusted according to service conditions
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
Check physical file disk space
diskMaxUsedSpaceRatio=88
# storage path
storePathRootDir=/app/rocketmq/storeSlave
CommitLog storage path
storePathCommitLog=/app/rocketmq/storeSlave/commitlog
Consume queue storage path Storage path
storePathConsumeQueue=/app/rocketmq/storeSlave/consumequeue
Message index storage path
storePathIndex=/app/rocketmq/storeSlave/index
#checkpoint File storage path
storeCheckpoint=/app/rocketmq/storeSlave/checkpoint
#abort file storage path
abortFile=/app/rocketmq/storeSlave/abort
# limit the message size
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
The role of the Broker
# -async_master Asynchronous replication Master
# -sync_master Synchronizes the double write Master
#-SLAVE
brokerRole=SLAVE
# Brush plate mode
# -async_flush Asynchronously flush disks
# -sync_flush Synchronously flush disks
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
# Number of sending thread pools
#sendMessageThreadPoolNums=128
# pull message thread pool number
#pullMessageThreadPoolNums=128
Copy the code

Worker2 configures borker-B’s salve node, broker-b-S. perties

Modify conf/ 2M-2s-async/broker-b-s.perties on work2

# Cluster name: all nodes with the same name are in the same cluster
brokerClusterName=rocketmq-cluster
Nodes with the same name are a set of primary and secondary nodes.
brokerName=broker-a
Brokerid,0 for Master, >0 for Slave
brokerId=1
#nameServer address, semicolon split
namesrvAddr=worker1:9876; worker2:9876; worker3:9876
# when sending a message, automatically create a topic that does not exist on the server
defaultTopicQueueNums=4
Whether to allow the Broker to create topics automatically
autoCreateTopicEnable=true
Whether to enable the Broker to automatically create subscription groups
autoCreateSubscriptionGroup=true
The port on which the Broker listens for external services
listenPort=11011
The default time for deleting files is 4 a.m
deleteWhen=04
The default retention time is 48 hours
fileReservedTime=120
CommitLog The default file size is 1 gb
mapedFileSizeCommitLog=1073741824
ConsumeQueue By default, each file holds 30W bytes, which can be adjusted according to service conditions
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
Check physical file disk space
diskMaxUsedSpaceRatio=88
# storage path
storePathRootDir=/app/rocketmq/storeSlave
CommitLog storage path
storePathCommitLog=/app/rocketmq/storeSlave/commitlog
Consume queue storage path Storage path
storePathConsumeQueue=/app/rocketmq/storeSlave/consumequeue
Message index storage path
storePathIndex=/app/rocketmq/storeSlave/index
#checkpoint File storage path
storeCheckpoint=/app/rocketmq/storeSlave/checkpoint
#abort file storage path
abortFile=/app/rocketmq/storeSlave/abort
# limit the message size
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
The role of the Broker
# -async_master Asynchronous replication Master
# -sync_master Synchronizes the double write Master
#-SLAVE
brokerRole=SLAVE
# Brush plate mode
# -async_flush Asynchronously flush disks
# -sync_flush Synchronously flush disks
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
# Number of sending thread pools
#sendMessageThreadPoolNums=128
# pull message thread pool number
#pullMessageThreadPoolNums=128
Copy the code

3. Other attention

  • 1, The store directories of two instances on the same machine cannot be the same, otherwise the error Lock failed,MQalready started will be reported

  • 2. The listenPort of two instances on the same machine cannot be the same. Nameserver does not need to be configured, just start it. This also shows that nameserver is stateless.

7. Start RocketMQ

1. Start nameServer

  • Modify bin/runserver.sh on the three nodes to adjust the JVM memory configuration.
JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn256m 
-XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
Copy the code
  • Start nameServer on three nodes
nohup bin/mqadminsrv &
Copy the code
  • Run the JPS command to check whether the NamesrvStartup process is started successfully

Start the broker

Start the broker is the MQBroker directive used, but note that the corresponding configuration file needs to be specified by -c when starting the broker.

  • Worker2 starts the master node of Broker-A and the slave node of broker-B
nohup ./mqbroker -c .. /conf/2m-2s-async/broker-a.properties & nohup ./mqbroker -c .. /conf/2m-2s-async/broker-b-s.properties &Copy the code
  • Work3 starts the master node of broker-B and the slave node of Broker-A
nohup ./mqbroker -c .. /conf/2m-2s-async/broker-b.properties & nohup ./mqbroker -c .. /conf/2m-2s-async/broker-a-s.properties &Copy the code
  • Pay attention to

If an error message is displayed indicating Lock failed or MQ already started when the slave is started, it is because multiple instances share the same storePath. In this case, you need to adjust the store path.

3. Check the startup status

  • Use the JPS

Use JPS to view the NameSrvStartup process and the two BrokerStartup processes

  • View the nohup.out log
View the nameServer log
tail -500f ~/logs/rocketmqlogs/namesrv.log
Check the broker log
tail-500f ~/logs/rocketmqlogs/broker.log
Copy the code

4, command line quick verification

The RocketMQ installation package provides a tools.sh tool that can be used to quickly validate the RocketMQ service at the command line. We go to the RocketMQ installation directory on Worker2:

  • Send message: The system sends 1000 messages by default
bin/tools.sh org.apache.rocketmq.example.quickstart.Producer
Copy the code
  • Receives the message
bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer
Copy the code
  • Configure an environment variable NAMESRV_ADDR to specify the NameServer address
export NAMESRV_ADDR='worker1:9876; worker2:9876; worker3:9876'
Copy the code

8. Set up the management console

A console is not available in the RocketMQ source code, but one is available in the Rocket community Extension project at github.com/apache/rock…

Once downloaded, go to the Rocket-Console directory and compile using Maven

mvn clean package -Dmaven.test.skip=true
Copy the code

After compiling, obtain the JAR package from target and execute directly. Note, however, that the address of nameserver needs to be specified in the project’s application.properties. This property is null by default. Add an application.properties file to the jar’s current directory to override one of the jar’s default properties:

rocketmq.config.namesrvAddr=worker1:9876; worker2:9876; worker3:9876Copy the code
  • Perform the jar
Java jar rocketmq - the console - ng - 1.0.1. JarCopy the code

After the completion of the start, can visit http://192.168.232.128:8080 to see management page