Producer loss message
Producers sending data to RabbitMQ can get lost halfway through because of network problems.
1. You can use rabbitMQ transactions to enable the rabbitMQ transaction (channel.txselect) before the producer sends a message. If the message is not received by RabbitMQ, the producer will receive an exception. At this point, you can roll back the transaction (channel.txrollback) and retry sending the message; If a message is received, the transaction can be committed (channel.txCommit). But the problem is that when you start rabbitMQ transactions, basically throughput goes down because it takes too much performance.
2. You can enable Confirm mode, which will assign a unique ID to each message written to RabbitMQ, and rabbitMQ will send you an ACK message saying that the message is ok. If RabbitMQ fails to process the message, it will call you back to the NACK interface telling you that the message failed to be received and you can try again. In combination with this mechanism, you can maintain the status of each message ID in memory yourself, and if you haven’t received a callback for that message for a certain amount of time, you can resend it.
The biggest difference between transactions and CNOfirm is that transactions are synchronous. You commit a transaction and it blocks, but confirm is asynchronous. You send a message and then the next message is sent. Rabbitmq then receives the message and asynchronously calls you back to an interface to inform you that the message was received. Therefore, the confirmation mechanism is generally used to avoid data loss in the producer.
Mq Missing message
In order to prevent RabbitMQ from losing data, you must enable RabbitMQ persistence, which means that messages will persist to disk after being written, even if RabbitMQ dies, and will be read automatically after being restored. In the rare event that RabbitMQ dies without persisting, a small amount of data can be lost, but this is unlikely.
There are two steps to setting up persistence:
1. Set the queue to persist so that rabbitMQ will persist the metadata of the queue but not the data in it;
2. Set deliveryMode to 2 to persist messages and RabbitMQ will persist messages to disk. Both must be set at the same time, and rabbitMQ will restart its queue from disk even if it hangs and restarts again.
And persistence can be combined with producer confirmations to notify producers of ACK messages only after they have been persisted to disk, so even if rabbitMQ dies before persisting to disk, the data is lost and the producer can’t receive an ACK, you can send it yourself. If producer confirmations are not enabled, even if persistence is enabled for RabbitMQ, there is a chance that the message will be written to RabbitMQ, but not persisted to disk, and rabbitMQ will hang up. A little bit of memory is lost.
Consumer lost message
The main reason for this is that when you consume something, the process hangs, for example after a restart, and rabbitMQ will assume that you have consumed it and the data is lost.
Rabbitmq allows you to use rabbitMQ’s ack confirmation mechanism. In simple terms, you can disable rabbitMQ automatic ack, call rabbitMQ through an API, and then ack rabbitMQ every time you make sure the rabbitMQ process is complete. This way, if you haven’t finished processing, rabbitMQ will assume that you haven’t finished processing, and rabbitMQ will assign the purchase to another consumer, so the message won’t be lost.
RabbitMQ’s message acknowledgement ACK mechanism
1. What is a message acknowledgement ACK?
A: If the consumer’s server encounters an exception while processing a message, the message may not be consumed and the data may be lost. To ensure that data is not lost, RabbitMQ supports message determination-ACK.
2. Message acknowledgement mechanism of ACK.
A: The ACK mechanism is that the consumer receives and processes a message from RabbitMQ and reports it back to RabbitMQ, which then removes the message from the queue.
If a consumer processes a message and there is no ACK, RabbitMQ will assume that the message was not properly consumed and re-queue it.
In the case of clustering, RabbitMQ will immediately push this message to other consumers online. This mechanism ensures that no messages or tasks are lost in the event of a consumer server failure.
Messages are never removed from RabbitMQ and will only be removed from RabbitMQ server data once the consumer has correctly sent an ACK and RabbitMQ has confirmed receipt.
ACK acknowledgment for messages is turned on by default.
3. Precautions for the development of ACK mechanism.
A: If you forget ACK, there are serious consequences. Messages are always redistributed when the Consumer exits. RabbitMQ then takes up more and more content, and since RabbitMQ runs for a long time, this “memory leak” is fatal.