In distributed system, how to communicate efficiently among various applications is a key in system design.

Using message Broker is an elegant solution.

RabbitMQ is a widely used message broker that follows the AMQP protocol.

Let’s take a look:

  • Message Broker concept
  • Core components of AMQP protocol
  • Four modes of message forwarding

1. Message Broker

Broker broker broker broker broker broker broker broker broker

In the message model, producers and consumers with messages are equivalent to sellers and buyers.

Therefore, a message broker is also needed, which leads to the concept of Message Broker.

Message Broker receives messages from producers and sends them to consumers so that they can be completely isolated.

RabbitMQ is a Message broker.

2. AMQP

How exactly is the message delivered? It depends on the message protocol used.

RabbitMQ supports multiple protocols, the most important of which is the Advanced Message Queuing Protocol (AMQP).

The conceptual model of AMQP is simple and consists of three parts:

  • Queue
  • Binding
  • Exchange

When a message is published to RabbitMQ, it reaches the Exchange, which then allocates the message to the Queue, from which the consumer gets the message.

AMQP is a programmable protocol that allows you to configure exchange, binding, and queue.

2.1 the Queue Queue

A Queue is a fifO data structure.

Queue is where RabbitMQ stores messages.

The Queue can be configured flexibly, for example:

  • Set the name
  • Configure reliable mode to protect data even if the broker fails
  • Automatic message deletion
  • Exclusive mode
  • …

2.3 Consumer

A queue can connect to multiple consumers at the same time.

The consumer can either pull messages from the Queue itself, or the Queue can actively push messages to the consumer.

2.4 Binding Binding

Binding is the rule for a Queue to connect to an Exchange.

Each Queue connects to a default Exchange, and we can connect more exchanges to this Queue.

Exchange routes messages to the corresponding Queue using the Binding rule.

2.5 Exchange

Exchange is the message gateway for RabbitMQ.

Exchange is like an operator who receives a message and decides how to forward it.

There are four main forwarding types:

  • Direct
  • Fanout
  • Topic
  • Header

Let’s take a look.

3. Message forwarding mode

3.1 Direct Direct passing

Routing key == Binding keyCopy the code

  • routing keyIs an attribute of the message.
  • binding keyIs specified when queue and Exchange are bound.

The producer (green ball) sends a message with a routing key — img.resize.

When the message arrives at Exchange (orange ball), Exchange looks for all queues with img.resize Binding key.

When matching queues are found, messages are sent to them.

If it is not found, the message is either returned to the producer or discarded.

After the message arrives at the specified queue, it is dispatched to the consumer in the form of polling (resizer.1/resizer.2) to ensure load balance.

3.2 the Fanout fan

Routing keys are ignored and messages are dispatched to all connected queues.

The most common scenario is message broadcasting.

Note that exchange broadcasts to queue, not queue to consumer. Queue to consumer is still polling.

3.3 the Topic theme

The routing key matches the binding key.

Routing key == Pattern in binding keyCopy the code

RabbitMQ uses the wildcards * and #.

* – Matches a word.

# – Matches zero or more words.

Routing messages with logs.error match binding keys — logs.error and logs.*, so messages enter “only error” and “alllogs”.

Routing key logs.success messages match binding keys — #success and logs.* so messages enter “only Success “and “alllogs”.

There are many applications for this form, including publish-subscribe, distributing related data to the desired worker, and so on.

3.4 the Header

A special type of Exchange that routes based on the key in the message header.

Using this approach, the message’s routing key property is ignored.

A binding for a header exchange can bind multiple headers to a queue. In this case, producers need to tell RabbitMQ which keys to match and producer can specify an identity X-match. Values can be:

  • Any – Only one value should match.
  • All – All values must match.

4. Confirm the message

Once messages arrive at their destination, the broker should remove them from the queue to prevent overflow.

The broker must be notified of the confirmation before the message is deleted.

There are two notification methods:

  • Automatic notification: As long as the consumer receives the message, whether the processing is complete or not.
  • Explicitly display notifications: Only after the consumer has sent back a confirmation message to ensure that the consumer has processed the notification before deleting it.

Recommended reading:

  • Architectural design principles – High concurrency
  • Architectural design – Service degradation
  • Interface-level fault handling policy
  • Common strategies for high concurrency
  • Distributed unique ID generation scheme