rabbitMQ
There are three modes: single-machine mode, common cluster mode, and mirrored cluster mode
RabbitMQ
High availability of
RabbitMQ
Is more representative, because it is based on master slave do high availability, we will use him as an example to explain the first type
MQ
How to achieve high availability of.
rabbitmq
There are three modes: single-machine mode, common cluster mode, and mirrored cluster mode
1
) Single machine mode
is
Level, generally is your local start play, no one production with single player mode
2
) Common cluster mode
This means booting multiple devices on multiple machines
Instance, one for each machine. But you created it
queue
, will only put in one
rabbtimq
Instance, but each instance is synchronized
queue
Metadata of. When you’re done consuming, in fact, if you’re connected to another instance, then that instance will go from
queue
Pull data from the instance.
This way is really troublesome, and it’s not very good, it’s not distributed, it’s just a normal cluster. Because this leads you to either have consumers randomly connect one instance at a time and pull data, or fix that connection
The current instance consumes data, the former with the overhead of data pull, and the latter resulting in single-instance performance bottlenecks.
And if that plays
If you have message persistence enabled, let
rabbitmq
If the message is stored in the ground, it doesn’t have to be lost until the instance is recovered before it can continue from this
queue
Pull data.
So it’s a bit awkward, there’s no such thing as high availability, it’s about throughput, it’s about having multiple nodes in the cluster serve one node, right
Read and write operations.
3
) Mirroring cluster mode
This pattern is what’s called
High availability mode, unlike normal cluster mode, you create
queue
, whether metadata or
queue
The message will live on multiple instances, and then every time you write a message to
queue
Will automatically send messages to multiple instances
queue
For message synchronization.
In this case, the advantage is that if one of your machines goes down, you can use the other machines. The disadvantages are, first, the performance overhead is too high, the message synchronizes all machines, resulting in the network bandwidth pressure and consumption is heavy! Second, there’s no extensibility to play this way, if some
It’s a heavy load, you add machines, you add machines that include this
queue
For all your data, there is no way to linearly expand your
queue
So how to enable the mirror cluster mode? I’m going to say this briefly, in case the interviewer asks you if you don’t know, it’s very simple
There is a nice admin console, which is to create a new policy in the background, and this policy is a mirrored cluster policy, and you can specify that you want to synchronize data to all nodes, or you can specify that you want to synchronize data to a specified number of nodes, and then you create it again
queue
When this policy is applied, data is automatically synchronized to other nodes.
(
)
kafka
High availability of
kafka
One basic architectural understanding: multiple
broker
Composed of, each
broker
Is a node; You create a
topic
the
topic
Can be divided into multiple
partition
, each
partition
Can exist in different
broker
every
partition
Just put in some data.
This is a natural distributed message queue, that is, one
Is distributed across multiple machines, with each machine storing a portion of the data.
In fact
It’s not a distributed message queue, it’s a traditional message queue, but it provides some cluster,
HA
Because no matter how you play it,
rabbitmq
a
queue
All data is stored in one node, and in a mirror cluster, it is also stored in each node
queue
Complete data of.
Kafka 0.8
Before, no
HA
Mechanical, any of them
broker
It’s down, so
broker
On the
partition
You can’t write, you can’t read, there’s no high availability.
Kafka 0.8
Later, it was provided
HA
The mechanism is
replica
Duplicate mechanism. each
partition
The data will be synced to the guitar machine to form its own multiple
replica
Copy. Then all
replica
Will elect one
leader
Come out, so production and consumption follow this
leader
Deal with, and then other
replica
is
follower
. When I write,
leader
Will be responsible for synchronizing data to all
follower
Go up there and read it as you read it
leader
On the data. Can read and write
leader
? It’s easy, if you can read and write every one of them
follower
, then we need to
care
Data consistency problem, system complexity is too high, it is easy to go wrong.
kafka
It’s going to be an even one
partition
All of the
replica
Distribute on different machines so that you can improve fault tolerance.
That’s what high availability is, because if a certain
It’s down. It’s okay. That’s
broker
The above
partition
There’s copies on all the other machines. If there’s one
partition
the
leader
A new one will be elected
leader
Come out and read and write the new one
leader
Can. This is called high availability.
When writing data, the producer writes
And then
leader
Write the data to the local disk, and the rest
follower
Take the initiative
leader
to
pull
The data. Once all
follower
Once the data is synchronized, it will be sent
ack
to
leader
.
leader
Receive all
follower
the
ack
After that, a write success message is returned to the producer. (Of course, this is just one pattern, and you can tweak this behavior)
When you consume, you only consume from
To read, but only one message has been taken
follower
All synchronization returns successfully
ack
When the news will be read by consumers.
In fact, this mechanism, if I go into depth, can go a lot further, but I’m going to go back to the theme and orientation of this course, focusing on the interview, at least you get the idea
How does it guarantee high availability, right? Don’t know nothing, you can also draw pictures for the interviewer. Meeting the interviewer is true
kafka
Master, dug deep to ask, that you can only say sorry, too deep you have not studied.
But it’s important to understand that there are trade-offs, and you’re going to be cramming the regular exam system, not really studying it
, to study deeply
kafka
You don’t have that much time. You just have to make sure that you probably didn’t know this before, but now that you do, you can probably say something when they ask you. And then there are a lot of other candidates, maybe worse than you, who haven’t seen this, who have been asked to the point where they can’t answer, whereas you can say something, which is sort of the point.
