When it comes to message consumption, we have to mention message idempotency. Business code usually receives a message for business logic processing once. If the same message is received several times, will it lead to repeated business processing? Can the Consumer not receive messages again?

RocketMQ does not guarantee that messages will not be re-consumed, and if the business is sensitive to repeated consumption, idempotent processing must be done at the business level, which can be done through distributed locking.

There are three modes of consuming messages in all messaging systems: at-most-once, at-least-once, and exactly only-once. Distributed messaging systems balance the three modes, the first two of which are feasible and widely used.

  • At-most-once: After a message is delivered, it will not be repeated regardless of whether the message is successfully consumed or not, which may result in the message not being consumed. RocketMQ does not use this method.

  • At-least-once: After a message is delivered and consumed, an ACK(message acknowledgement mechanism) is returned to the server. No ACK message is returned if no consumption is completed. If the server fails to receive an ACK from the client due to a network exception, a client restart, etc., the server will repost the ACK, which may result in repeated consumption. RocketMQ uses ACK to ensure that the message is consumed at least once.

  • Exactly -only-once: A message can be considered exactly-only-once only if both of the following conditions are met. 1. In the message release phase, do not send duplicate messages. 2. In the consuming message phase, repeated messages are not allowed to be consumed. In distributed system environment, if we want to implement this pattern, huge overhead is inevitable. RocketMQ, in pursuit of high performance, does not guarantee this feature, avoiding message duplication and idempotent processing by the business.

Here is a look at the concept of message idempotence in the Dark Horse -RocketMQ system: