This is the 10th day of my participation in the August More Text Challenge

Frequently asked interview questions: How do I keep messages from getting lost

Case background

On the e-commerce platform, users choose to use points to deduct part of the amount when they buy goods. During the whole process, the transaction service and points service communicate through MQ message queues. When an order is placed, the transaction service sends a message to the MQ queue to deduct points, and the points service consumes the command on the consumer side to achieve the actual deduction.

Thought analysis

Purpose of USING MQ: System decoupling, flow control (high availability and high performance issues)

  • System decoupling

    • With MQ message queues, you can isolate instability from changes in the upstream and downstream environments of your system. For example, no matter how the points service changes, the transaction service will not be affected. Even if the points service fails, the main transaction process can degrade the points service to decouple the two services and realize the high availability of the system

  • Flow control

    • In the scenario of sudden increase of traffic during peak transaction, the peak clipping and valley filling function of traffic can be realized through MQ, and the downstream service processing capacity can automatically adjust the traffic

Their thinking

Resolving message loss requires knowing which links lose messages. The whole process involves producers, consumers, and message-oriented middleware.

Producer: When sending messages to MQ, network problems or service bugs may cause messages to fail to be sent. Return values and exceptions need to be handled to prevent messages from being sent to MQ properly, and alarm mechanisms can be added if necessary to monitor unsuccessful data delivery and alert developers to manual intervention or automatic retransmission.

Message-oriented middleware: Guaranteed by message-oriented middleware

Consumer: A consumer pulls a message from the Broker and ensures that the message is not lost as long as the consumer service completes the business process and sends a confirmation of a successful consumption.

The entire process, from the producer sending the message, to the consumer retrieving the message, completing the business process, and finally sending the consumption-completion confirmation, can be guaranteed. But for the Design principle of Design for Failure, we should add a message detection to check if messages are missing. That is, when a message producer sends a message, it carries a globally unique ID or continuously incrementing version number, and then the consumer verifies the corresponding version.