To ensure the reliability of the message, it is necessary to ensure the reliability of the message at every stage of the flow

It can be seen from the figure that a message will go through four nodes, and only the reliability of these four nodes can ensure the reliability of the whole system.

  1. The producer sends a guarantee to reach MQ.
  2. MQ receives the message and is guaranteed to be distributed to the corresponding Exchange.
  3. Exchange guarantees the persistence of messages after they are distributed and enqueued.
  4. After receiving the message, the consumer guarantees the correct consumption of the message.

The producer sends messages to the MQ Broker for reliability assurance

This is the first process of message flow. After our producer sends a message, our message may not be sent to MQ due to various reasons such as network interruption, but at this time, our production end does not know that our message is not sent, which will cause the loss of message. RabbitMQ provides two solutions to this problem:

  • Transaction mechanism implementation was introduced
  • Publisher Confirm via sender confirmation

This mechanism is easy to understand. After a message is sent to MQ, MQ will send us an acknowledgement message.

Transaction mechanism

The RabbitMQ client has three transaction methods: channel.txselect, channel.txcommit and channel.txrollback. Channel.txselect is used to set the current channel to transaction mode channel.txCOMMIT is used to commit transactions and channel.txrollback is used to roll back transactions. After the transaction is enabled via channel.txSelect, we can publish a message to RabbitMQ. If the transaction is committed successfully, the message must reach RabbitMQ. If RabbitMQ crashes or throws an exception before the transaction is committed, At this point we can capture it and roll back the transaction by executing the channe1.txrollback method. Note that the transaction mechanism in RabbitMQ is different from the transaction concept in most databases and needs to be distinguished. Normal transaction commit process:

channel. txSelect ();
channel.basicPublish (EXCHANGE NAME, ROUTING_KEY,MessageProperties.PERSISTENT_TEXT_PLAIN,"transaction messages".getBytes ()); 
channel.txCommit ();
Copy the code

The AMQP protocol flow process corresponding to the above codes is shown in the figure:

There are four more steps to enable the transaction mechanism than not to enable it

  • The client sends tx.select, putting the channel in transaction mode
  • The Broker replies to tx.select-ok, confirming that the channel has been set to transaction mode
  • After sending the message, the client sends tx.mit to commit the transaction
  • The Broker replied to tx.mit-OK, confirming the transaction commit

Transaction rollback process:

try{
    channel.txSelect ();
    channel.basicPublish (exchange,routingKey,
    MessageProperties.PERSISTENT_TEXT_PLAIN,msg.getBytes()); 
    int result = 1 / 0; 
    channel.txCommit ();)
}catch (Exception e) {
    e.printStackTrace (); 
    channel.txRollback ();
}
Copy the code

The AMQP protocol flow process is shown as follows:

If you want to send multiple messages, wrap methods like channel.basicPublish and channel.txCommit inside the loop.

channel.txSelect(); for (int i = 0; i < LOOP_TIMES; i++) { try { channel.basicPublish("exchange", "routingKey", null, ("messages" + i).getBytes()); channel.txCommit(); }catch(IOException e) { e.printStackTrace(); channel.txRollback(); }}Copy the code

Transactions do solve the problem of message acknowledgement between the sender and RabbitMQ. The transaction can commit only if the message is successfully received by RabbitMQ, otherwise the transaction can be rolled back after the exception is caught and the message can be resold at the same time. But using transactions can suck performance out of RabbitMQ.

Sender confirmation mechanism

Implementing transactions severely reduces RabbitMQ’s message throughput, so a lightweight approach was introduced — publisher Confirm. Producers set the channel to Confirm mode, and once it is in confirm mode, All messages posted on this channel will be assigned a unique ID (starting at 1). Once the message has been posted to all the matching queues RabbitMQ will send an acknowledgement (basic.ack) to the producer (containing the unique ID of the message) to let the producer know that the message has reached its destination correctly. If the message and queue are persistent, the acknowledgement message is sent after the message is written to disk. The deliveryTag in the confirmation message RabbitMQ sends back to the producer contains the serial number of the confirmation message. RabbitMO can also set the multiple parameter in channel.basicAck to indicate that all messages up to this serial number have been processed. You can refer to the following figure

Note: The throughput of both the transaction mechanism and common confirm mode is low. Therefore, batch confirm and asynchronous confirm are recommended in actual production environments

The MQ receiving or routing failed. Procedure

With the producer sending message processed, we can take a look at processing on the MQ side, where two problems can occur:

The message could not find the corresponding Exchange. An Exchange was found but no Queue was found. Both cases can be resolved using the Mandatory parameter provided by RabbitMQ, which sets the policy for message delivery failure. There are two policies: automatic deletion or return to the client.

MQ Broker goes down

If MQ suddenly goes down or is shut down, the message must be persisted so that it can be recovered after MQ restarts. Message persistence needs to be done, but not just message persistence, but queue persistence and Exchange persistence.

These are the problems caused by MQ outages, and if there is a server outage or disk corruption, none of the above measures will work, so you must introduce mirrored queues and do remote multitasking to combat this irresistible factor. (Not in one basket)

Consumers are unable to spend normally

After turning on manual message confirmation, as long as this message is not consumed successfully, no matter whether there is consumer outage or code exception in the middle, as long as this message has not been consumed after the connection is disconnected, this message will be put back into the queue and consumed again.

Of course, repeated consumption can occur, but in distributed systems idempotency is required, so repeated consumption is generally blocked by the idempotency of the interface.

Idempotence means that the result of an operation executed many times is the same as the result of one execution.