This is the 19th day of my participation in the August More Text Challenge

Phase to recommend

  • Will know will know! RabbitMQ message confirmation mechanism

  • RabbitMQ has six working modes

  • RabbitMQ Four switch types

Reliability delivery at the production end

Reliable delivery means 100% successful delivery of messages, and there are all sorts of scenarios in RabbitMQ that will prevent our messages from being delivered properly, A message sent by a producer cannot reach the Broker, the switch cannot route a message to a queue, and RabbitMQ breaks down after a message is sent to the queue or to the Broker. Therefore, we need to have a reliable delivery scheme to solve this kind of problem. To ensure the reliable delivery of messages, the following conditions must be met:

  1. Ensure that messages reach the Broker. Solution (Open Comfirm message confirmation mode and establish message delivery failure compensation mechanism)

  2. Ensure that messages are routed to the specified queue. Solution (Enable ReturnCallback failure callback, set true to Mandatory and message failure callback, and return messages to producers when the route is not in the specified queue without discards messages) or use the backup switch to send messages that are not in the specified queue to the backup switch.

  3. Ensure that messages are stored correctly on the queue so that they are not lost when RabbitMQ goes down. Solution (Make queues persistent and not automatically deleted when defining them)

The message falls into the database and marks the message status

The main process is as follows:

  1. First, before a message is sent to RabbitMQ, set the state of the message to “in delivery” and save the message to the database, and if there is business data to drop, the message and business data drop need to be supported by the transaction to ensure that either they all succeed or they all fail.

  2. Once the message is dropped, it can be delivered to RabbitMQ.

  3. Listen for the message callback and determine the next action based on the ACK returned by the message callback

  4. If the ack returned in the listening callback is true to indicate that the message was successfully delivered to RabbitMQ, change the status of the message saved to the database to “delivered”.

  5. Compensation mechanism, we need to enable a timer task, every 10 seconds to perform a database search tasks, from messages database to find out the status of “delivered” and retry time is less than the current time, the message to send, and then modify these message retry count + 1, the next retry time, update, and persisted to the database.

  6. If the status of the messages searched by the scheduled task from the message database is In Delivery and the number of retries is greater than 3, the scheduled task sets the status of the messages to Failed delivery and does not redeliver the messages.

Small case implementation

The following case is built based on SpringBoot+MyBatis Plus

  1. Start by preparing a database table to hold message-related information

  • Corresponding POJO class

2. Prepare queues and switches. In this case, the Direct switch is used.

  1. The message confirmation callback ComfirmCallback

  1. Message failure callback. To ensure that a message is reliably delivered to a queue, a ReturnCallback is triggered when the message is not routed to the specified queue, and RabbitMQ returns the message to the producer, where we resend the returned message. In this case, the standby switch can deliver the messages that cannot be routed to the specified queue to the backup switch. The backup switch sends the messages to the queue. Generally, the backup switch is of Fanout type, so all queues bound to the backup switch can receive the messages sent by it. (The following message failure callback is set in the RabbitConfig configuration class. If injected through a custom message failure callback class, RabbitTemplate will cause a loop dependency. See how to resolve this problem.)

  1. Manual acknowledgement, message acknowledgement callback, and Mandatory are both true to return a message to the producer when it is not in a specified queue. (With manual acknowledgement enabled, if the consumer fails to return an ACK after receiving a message, the message will not be lost. Before the message is removed from the queue.

  1. Configure a scheduled task to find and resend the messages in the message database every 10 seconds.

  1. Consumers listen to queues and consume messages

  1. The Controller receives the request to send the message

Now use the Postman test

  • First, we make sure we test with the correct switch and queue specified when sending the message. Send a POST request in postman, specifying the message content as"Programmer who doesn't drink milk tea", the Controller will first set the message status to 0"In delivery"And then put it in a database,

After the message is sent, you can see from the console that the message has been sent to the Broker and posted to the specified queue, and the consumer consumes the message and returns an acknowledgement ACK.

At the same time, the ack received by the message acknowledgement callback is true, and the state of the message stored in the message store is changed to 1"Delivered".

  • If the specified switch does not exist when we deliver the message, the delivery will fail. Let’s test the delivery of the message in this case.
  1. We change the switch specified when Posting the message to one that does not exist

2. Also send a request in postman, specifying the message content as"Programmer222 without milk tea."

3. The console output is as follows: Because no corresponding switch exists, the ack returned by the message acknowledgement callback is false, and the message delivery fails

  1. Task will regularly every 10 seconds from the message in the library to find to meet the requirements of resend to resend the message, because when we send a message in the regular tasks specified switches is right, so when resending the message can be delivered right to the Broker, so after timing task of retransmission, message status was changed to 1 “has been delivered,” said and retry count is 1, Consumers also get messages from the corresponding queues for consumption.

  • Similarly, if we set the switch specified when sending the message in Controller and the switch specified when the scheduled task resends the message as non-existent, the message will never be delivered successfully. After the scheduled task resends the message for 3 times, the message status will be set to 2"Delivery failed"As shown in the following

  • In the following, we set the routing key specified in the controller when sending the message to a non-existent routing key, but re-send the message to specify the correct switch and routing key, and test the situation when the message cannot be routed to the specified queue.

  • Can be seen from the following operation result, sent for the first time since the routing key does not exist, the switch can’t the message is routed to the specified queue, so will trigger message callback returnCallback failure, and then in the callback to send, to send the specified routing keys and switches are correct, so the message can be delivered right to the specified queue, Consumers can also get messages from the columns for consumption.

conclusion

To ensure reliable message delivery at the production end, the following requirements must be met:

  1. If Comfirm message confirmation mode is enabled, publisher-confirm-type: correlated, production will receive RabbitMQ confirmation replies

  2. With persistent queues and switches, messages will not be lost even if RabbitMQ goes down.

  3. The producer establishes compensation mechanisms for message delivery failures (drop-off, marking, scheduled task)

  4. If the mandatory parameter is set to true, the message is returned to the producer when it cannot be routed to a specified queue. The producer implements the callback function ReturnCallback to process the message returned by the server.

🏁 This is the introduction to the reliability delivery scheme 1 for RabbitMQ production. If there are any errors, please leave a comment. If you think this article is helpful, please like 👍 😋😻 👍