“This article has participated in the call for good writing activities, click to view: the back end, the big front end double track submission, 20,000 yuan prize pool waiting for you to challenge!”

Introduction to the

RabbitMQ is a messaging middleware that uses Erlang to implement AMQP(Advanced Message Queuing Protocol). It originated in financial systems and is used to store and forward messages in distributed systems.

RabbitMQ has grown in popularity due to its excellence in ease of use, scalability, reliability and high availability. The specific features of RabbitMQ can be summarized as follows:

  • Reliability: RabbitMQ uses mechanisms to ensure the reliability of messages such as persistence, transport confirmation, and release confirmation.
  • Flexible routing: Messages are routed through the exchange before they are queued. For typical routing, RabbitMQ already provides some built-in switches. For more complex routing functions, multiple switches can be bound together, or you can implement your own switch through a plug-in mechanism.
  • Scalability: Multiple RabbitMQ nodes can form a cluster or dynamically expand the cluster based on service requirements.
  • High availability: Queues can be mirrored on machines in a cluster, making them available even if some nodes fail.
  • Supports multiple protocols: RabbitMQ supports multiple messaging middleware protocols, such as STOMP and MQTT, in addition to AMQP.
  • Multi-language clients: RabbitMQ supports almost all common languages, such as Java, Python, Ruby, PHP, C#, JavaScript, etc.
  • Easy-to-use management interface: RabbitMQ provides an easy-to-use user interface that allows users to monitor and manage messages, nodes in a cluster, etc. Install RabbitMQ and the admin interface will come with RabbitMQ.
  • Plugin mechanisms: RabbitMQ provides many plug-ins to extend in many ways, but you can also write your own.
The core concept

RabbitMQ is essentially a producer and consumer model, responsible for receiving, storing, and forwarding messages. Think of it as sending a package to the post office, which stores it and eventually sends it to the postman. RabbitMQ is like a system of post office, mailbox and postman. In computer terms, the RabbitMQ model is more like a switch model.

ConnectionFactory, Connection, Channel

ConnectionFactory, Connection, and Channel are all basic objects in RabbitMQ’s API. Connection is a TCP Connection, a RabbitMQ socket Connection, which encapsulates some of the socket protocol logic. ConnectionFactory is the manufacturing factory for Connection. A Channel is the most important interface we use with RabbitMQ and is a virtual Connection built on top of a Connection. Most of our business operations are done in the Channel interface, including defining queues, defining exchanges, binding queues to exchanges, publishing messages, and so on.

Channels were created to reuse TCP connections. We know that the establishment of A TCP connection requires a three-way handshake, which is expensive. So reuse Connection to reduce overhead. But once there are too many channels, connections can become a single point of bottleneck.

Multiple connections can be created to evenly distribute channels.

They are both producers and consumers.

Producer: The party that delivers messages.

Consumer: The party receiving the message.

A message typically consists of two parts: a header (or Label) and a body. The body of the message, also called payLoad, is opaque, and the header is made up of a set of optional attributes. These attributes include routing-key, priority (priority over other messages), delivery-mode (indicating that the message may require persistent storage), and so on. After the producer sends the message to RabbitMQ, RabbitMQ sends the message to the interested Consumer based on the message header.

Exchange

In RabbitMQ, all messages are not sent directly to the queue after being sent by the producer. Instead, it must go through Exchange and be routed to a queue.

In many cases, even if we see a Posting message without declaring the Posting Exchange, it will actually be posted to the Default Exchange and then routed to the specified queue.

Exchanges are used to receive messages sent by producers and route them to queues on the server. If they are not routed, they may be returned to the Producer or discarded. Here you can think of the switch in RabbitMQ as a simple entity.

RabbitMQ has four types of exchanges, each of which has a different routing policy: Direct (default), Fanout, Topic, and headers.

When a producer sends a message to an exchange, it typically specifies a RoutingKey that specifies the routing rules for the message, and this RoutingKey needs to be used in conjunction with the exchange type and the BindingKey to take effect.

RabbitMQ uses a Binding to bind exchanges to queues, usually specifying a BindingKey so RabbitMQ knows how to route messages to queues correctly. As shown in the figure below. A binding is a routing rule that connects a switch to a message queue based on a routing key, so a switch can be thought of as a routing table made up of bindings. Exchange and Queue bindings can be many-to-many.

When a producer sends a message to the exchange, a RoutingKey is required, and when the BindingKey matches the RoutingKey, the message is routed to the corresponding queue. When binding multiple queues to the same exchange, these bindings allow the same BindingKey to be used. BindingKey does not work in all cases. It depends on the type of switch, such as fanout, and instead routes messages to all queues bound to that switch.

Queue: Message Queue

A Queue is used to hold messages until they are sent to consumers. It is the container and destination of the message. A message can be put into one or more queues. The message remains in the queue, waiting for the consumer to connect to the queue to pick it up.

RabbitMQ messages can only be stored in queues, as opposed to messaging middleware such as Kafka. Kafka stores messages at the logical level of the topic (topic), and the corresponding queue logic is just a displacement identifier in the actual storage file of the topic. The RabbitMQ producer produces the message and ultimately delivers it to the queue, from which the consumer can retrieve and consume the message.

Multiple consumers can subscribe to the same queue, and messages in the queue are round-robin for processing among multiple consumers, rather than each consumer receiving all messages and processing them, thus avoiding repeated consumption of messages.

RabbitMQ does not support queue-level broadcast consumption, so secondary development on top of broadcast consumption is cumbersome and not recommended.

Broker (service node of message-oriented middleware)

For RabbitMQ, a RabbitMQ Broker can simply be regarded as a RabbitMQ service node, or RabbitMQ service instance. In most cases a RabbitMQ Broker can also be thought of as a RabbitMQ server.

The following figure shows the process by which a producer stores a message into a RabbitMQ Broker and a consumer consumes data from the Broker.

Exchange Types

The main Exchange types for RabbitMQ are FANout, Direct, Topic, and headers. The AMQP specification also mentions system and custom Exchange types.

(1) the fanout

The Exchange routing rule of the FANout type is very simple. It routes all messages sent to the Exchange to all queues bound to it without any judgment, so the Fanout type is the fastest of all switch types. The fanout type is often used to broadcast messages.

(2) direct

Exchange routing rules of type Direct are also simple, routing messages to queues where bindingKeys match routingkeys exactly.

For example, if the routing key is set to “Warning” when sending a message, the message will be routed to Queue1 and Queue2. If the routing key is set to “Info “or” DEBUG “when sending messages, messages will only be routed to Queue2. If messages are sent using other routing keys, they are not routed to either queue.

The direct type is commonly used to process tasks with a higher priority. The message is sent to the corresponding queue according to the priority of the task. In this way, more resources can be assigned to the queue with a higher priority.

(3) the topic

As mentioned above, the routing rules of the direct type switch match BindingKey and RoutingKey exactly. However, such strict matching method cannot meet the requirements of actual business in many cases. Topic exchanges extend the matching rules. Similar to direct exchanges, they route messages to queues that match BindingKey and RoutingKey, but the matching rules are a little different:

  • The RoutingKey is a dot “. Delimited string (dot ‘. Each separate string is called a word), such as “com.rabbitmq.client”, “java.util.concurrent”, “com.hidden. Client”;
  • BindingKey is also a dot like RoutingKey. Delimited strings;
  • There can be two special strings “*” and “#” in BindingKey for fuzzy matching, where “*” is used to match one word and “#” is used to match multiple words (possibly zero).

The above picture shows an example:

  • The messages whose routing key is com.rabbitmq.client are routed to both Queuel and Queue2.
  • Messages with a routing key of “com.hidden.client” are routed to Queue2 only.
  • The messages whose routing key is com.hidden.demo will only be routed to Queue2.
  • The messages whose routing key is java.rabbitmq.demo are routed to Queuel only.
  • Messages with a routing key of “java.util.concurrent” will be discarded or returned to the producer (the mandatory parameter is required) because it does not match any routing key.
④ Headers (not recommended)

A HEADERS exchange does not rely on the matching rules of the routing key to route a message, but matches it based on the HEADERS attribute in the content of the sent message. When sending a message to the exchange, RabbitMQ will fetch the headers of the message and compare the matching keys to the queue and the exchange binding. If they match, the message will be routed to the queue. Otherwise, the queue will not be routed to. Headers switches tend to be poor in performance and impractical, and you’ll almost never see them.

RabbitMQ workflow

The producer sends a message:

1. The producer connects to the Broker, establishes a Connection, and starts a Channel.

2. The producer declares an Exchange and defines attributes such as type, persistence, automatic deletion, other parameters, and so on.

3. The producer declares a Queue and defines attributes such as persistence, automatic deletion, parameters (maximum message lifetime, Queue full behavior, dead-letter Queue related), etc.

4. Producers bind Exchange and Queue using routing keys and define binding attributes.

5. The producer sends a message to the Broker, carrying information about Exchange and routing keys.

6.Exchange searches for matching queues based on routing keys.

7. If a matching queue is found, the message is stored in the queue.

8. Otherwise, discard or return to the producer based on the configuration.

9. Close the Channel

10. Closing the Connection

Consumers receive the message:

1. The consumer connects to the Broker, establishes a Connection, and opens a Channel.

2. The consumer requests the Broker to consume messages from the corresponding queue, possibly setting up a callback function and doing some preparatory work.

3. Wait for the Broker to respond and deliver the message to the corresponding queue. The consumer receives the message.

4. The consumer consumes the message and Ack it (possibly automatically).

5. The Broker deletes messages that have been Ack Ed from the queue.

6. Close the Channel.

7. Close Connection.

See RabbitMQ Field Guide)