Open full code force, code moving life, this article first public number [Craig Mowgli], pay attention to this word does not drive the old driver of the code circle
This article has been included on GitHub github.com/BeKingCodin… , my contact information and technical exchange group, welcome Star and perfect
preface
The previous two articles introduced you to messaging middleware MQ.
Ten Minutes to start Messaging Middleware
Still struggling with seckilling? See how MQ handles it
This article uses the popular RocketMQ as an example to help you better understand messaging middleware.
The body of the
We all know that in order to ensure high availability, basically all the components will adopt the master-slave architecture design, the classic MySQL “MySQL how to achieve master-slave backup?”
RocketMQ typically deploits brokers in master-slave mode, where one Master Broker corresponds to one Slave Broker, to ensure data loss and high availability.
The Master synchronizes data to the Slave after receiving the message, so that if the Master Broker fails, there is a backup data on the Slave.
The design of the entire architecture is relatively common, and here we have to consider a question:
How does the Master Broker synchronize messages to the Slave Broker?
Is the Master Broker actively pushing to the Slave Broker? Or does the Slave Broker send requests to the Master Broker to pull?
The answer is second, RocketMQ’s master-slave mode takes the Slave Broker to continuously send requests to the Master Broker to pull messages.
So the first thing to make clear is that RocketMQ’s own master-slave mode uses Pull mode to Pull messages.
RocketMQ read-write separation?
Since the Master Broker receives messages written to the system, it then synchronizes them to the Slave Broker. To put it bluntly, Slave Broker should have a copy of the same data.
So when a consumer system needs to get a message, it gets it from the Master Broker? Or from the Slave Broker?
Not really. The answer is: it is possible to get messages from the Master Broker as well as from the Slave Broker.
(1) As a consumer, the system will first send a request to the Master Broker to request the message.
(2) When the Master Broker receives the request, it returns a message to the consumer system.
(3) The Master Broker will return a message according to the load of the Master Broker and the synchronization of the Slave Broker at that time. Suggest to the consumer system whether to pull messages from the Master Broker next time.
For example, if the Master Broker is heavily loaded at this time, it is not appropriate to pull messages from the Master while trying to withstand a large number of concurrent write requests.
The Master Broker will then suggest that you pull the message from the Slave Broker next time.
For another example, let’s say Slave Broke, for some reason, synchronizes very slowly. Since the Slave Broker is too far behind the Master Broker to get the latest messages from the Slave Broker, it will have to pull messages from the Master Broker next time.
To summarize, when writing a message, it is usually the Master Broker that is selected to write; However, when a message is pulled, it may be fetched from the Master Broker or Slave Broker, depending on the situation at the time.
What happens if Slave Broke dies?
As we have analyzed above, the write of the message must be all sent to the Master Broker, and then the message may go to the Master Broker or the Slave Broker.
So if the Slave Broker dies, either writing or pulling messages can continue to be done by the Master Broke with little impact on the overall operation.
But without the Slave Broker, all the read and write pressure is concentrated on the Master Broker, increasing the Master Borker’s load.
What happens if the Master Broker fails?
Data can be synchronized through the Slave Broker to ensure that data is not lost. But prior to RocketMQ version 4.5, there was no way for the Slave to automatically switch to Master if the Master failed and died.
And while the Slave Broker is supposed to have a copy of the Master Broker’s data, it is possible that some data has not yet been synchronized from the Master Broker.
In this case, if the Master Broker is down, some operations need to be done manually, reconfiguring the Slave Broker to become the Master Broker. This process is a bit cumbersome and leads to periods of unavailability in between.
RocketMQ was not fully available until version 4.5 due to the inability to automatically switch from Slave to Master.
Implementation of RocketMQ High Availability Automatic Switching based on Dledger?
Those of you who have done operations know how painful it is to rewrite configurations, put machines on line, and so on. After RocketMQ 4.5, the advent of the Dledger mechanism greatly changed this situation.
This mechanism itself is implemented based on Raft protocol, the implementation principle and algorithm idea is a bit complicated, I will write a special article to introduce it.
Today we will introduce how to implement master/slave hot switching for RocketMQ based on Dledger.
In simple terms, one Master Broker corresponds to multiple Slave brokers, and data is synchronized between the Master and Slave, meaning that one copy of data can have multiple copies.
When the Master Broker fails, Dledger technology and Raft protocol algorithm can be used to elect a leader from the Slave brokers to become the new Master Broker.
The whole process generally only takes 10 to dozens of seconds, and everything is automatic.
conclusion
Messaging middleware MQ as a solution to the high concurrency, distributed environment, there is still a lot to learn. Many programmers who have been working on MQ for three years still look at it with a blank face because it is so versatile in its own right, and I want you to stay tuned for the Start from Scratch Messaging Middleware series.
Welcome to my public account “Criag Mowgli”
Creation is not easy, your support and recognition, is the biggest motivation for my creation, we will see you in the next article!