RabbitMQ is rarely lost, but accidents cannot be ruled out, and in order for our system to be highly available, we have to take better measures to ensure the stability of the system.
To introduce the following, how to ensure that the message is absolutely not lost, the absolute dry goods shared below, are used in the production line of well-known Internet products.
1. Message persistence
2.ACK confirmation mechanism
3. Set the cluster mirroring mode
4. Message compensation mechanism
The first is message persistence
RabbitMQ messages are stored in memory by default, unless specified, they are not persisted and will be lost if the node restarts or crashes unexpectedly.
So the message needs to be persisted. How to persist, the following details:
To persist a message, one of the following three conditions must be met.
1) Exchange set persistence
2) Queue set persistence
3) Message persistent sending: Set the sending mode deliveryMode=2, which represents persistent Message
Second: ACK acknowledgement mechanism
Multiple consumers receive messages at the same time, for example, one consumer dies when the message is half received (logic complex time is too long, timeout or consumption is stopped or the network is disconnected), how to ensure that the message is not lost?
This use should use the Message Acknowledgment mechanism, which means that the consumer should notify the server when it is finished consuming and the server should delete the Message from memory.
In this way, if a consumer has a problem and does not synchronize the message to the server, there are other consumers to consume, ensuring that the message is not lost case.
Third: Set the cluster mirroring mode
Let’s take a look at the three deployment modes of RabbitMQ:
1) Single-node mode: In the simplest case, non-cluster mode, the node hangs and the message becomes unavailable. Business may be paralysed and just wait. 2) Common mode: The default cluster mode. When a node is down, messages on the node cannot be used, and influential services are broken, the node can only be restored and available (messages must be persistent). 3) Mirror mode: Make the required queue into a mirror queue, which exists on multiple nodes and belongs to RabbitMQ HA scheme
The reason for a mirrored cluster is that the contents of a queue are stored on only one node, not all nodes, where only message structures and metadata are stored. The following is a diagram to illustrate the loss of messages in a normal cluster:
If you want to solve the above problems and ensure message loss, you need to use HA mirroring mode queue.
The following describes the three HA policy modes:
1) Synchronize to all 2) Synchronize to a maximum of N machines 3) Synchronize only to nodes that match the specified name
1) Set the mirror of all nodes for each queue starting with “rock.wechat” Rabbitmqctl set_policy ha-all “^rock. Wechat “‘{“ha-mode”:”all”,”ha-sync-mode”:”automatic”}’ rabbitmqctl set_policy -p rock ha-all “^rock.wechat” ‘{“ha-mode”:”all”,”ha-sync-mode”:”automatic”}’
2) Set a mirror of two nodes for each queue starting with “rock.wechat.” And set to automatic synchronization mode rabbitmqctl set_policy -p rock ha-exacly “^rock. Wechat “\ ‘{“ha-mode”:”exactly”,”ha-params”:2,”ha-sync-mode”:”automatic”}’
Rabbitmqctl set_policy ha-nodes “^nodes\.” \ Rabbitmqctl set_policy ha-nodes “^nodes\ ‘{“ha-mode”:”nodes”,”ha-params”:[“rabbit@nodeA”, “rabbit@nodeB”]}’
However, HA mirroring queues have a major disadvantage: throughput of the system can be reduced
Fourth: message compensation mechanism
Why a message compensation mechanism? Will the message be lost, yes, the system is in a complex environment, do not think too simple, although the above three schemes, the basic can ensure that the high availability of messages do not lose the problem,
But as a programmer with ambitions, I have a sense of crisis to absolutely ensure the stability of my system.
For example, when a persistent message is saved to a disk, the current queue node hangs up and the disk of the storage node fails again, and the message is lost. What can I do?
The network environment of the production line is too complicated, so the number of messages is too large. The message compensation mechanism needs to be established on the basis that messages must be written into DB logs, sent logs and received logs, and the status of both must be recorded.
Then, according to the DB log, check message is sent successfully. If the message is not sent successfully, compensation measures are taken and messages are sent again.
! [](bbs.itheima.com/data/a…
Reference: www.cnblogs.com/flyrock/