Introduction to RabbitMQ

Short for Message Queue, Message queues (MQ) are an application-to-application communication method. Applications communicate by reading and writing messages in and out of queues (data for the application) without the need for dedicated connections to link them. Messaging refers to programs communicating with each other by sending data in messages rather than by direct calls, which are typically used for techniques such as remote procedure calls. Queuing refers to applications communicating through queues. The use of queues removes the need for receiving and sending applications to execute simultaneously.

RabbitMQ is an open source message queue system based on the AMQP protocol developed using the Erlang language. The main characteristics of AMQP are message orientation, queue, routing (including point-to-point and publish/subscribe), reliability, and security. AMQP protocol is more widely used in enterprise systems, which requires high data consistency, stability and reliability, and requires less performance and throughput.

RabbitMQ usage scenarios

1. Decoupling (to provide the basic ultimate conformance implementation for service-oriented Architecture (SOA))

Scenario Description: After a user places an order, the order system notifies the inventory system. Traditionally, the order system calls the inventory system interface.

Disadvantages of the traditional model:

  • If the inventory system is inaccessible, the order destocking will fail, resulting in an order failure
  • The order system is coupled to the inventory system

Import message queue

  • Order system: after the user places an order, the order system completes the persistent processing, writes the message to the message queue, and returns the user to place the order successfully
  • Inventory system: subscribe to the order message, using the pull/push way to obtain the order information, inventory system according to the order information, inventory operation
  • What if: inventory system is not working properly when placing an order. It also does not affect the normal order, because after the order is written to the message queue, the order system does not care about other subsequent operations. Realize the order system and inventory system application decoupling
  • To make sure there is inventory, you can set the queue size to the inventory number, or do something else.

The message-based model is concerned with “notification”, not “processing”.

SMS notification, email notification, and cache refresh are notified using message queues.

The difference between message queues and RPC:

RPC: asynchronous call, obtains call results in time, has strong consistency results, and cares about service call processing results.

Message queue: Two asynchronous RPC calls that are dumped in a queue and delivered at the right time (staggered flow control)

2. Improve efficiency asynchronously

Scenario Description: After registration, users need to send registration emails and SMS messages. There are two traditional methods: 1. Serial method; 2. Parallel mode

(1) Serial mode: after the registration information is written into the database successfully, the registration email is sent, and then the registration SMS is sent. After the preceding three tasks are complete, the system returns them to the client

(2) Parallel mode: after the registration information is written into the database successfully, the registration email is sent and the registration SMS is sent. After the preceding three tasks are complete, the system returns them to the client. The difference with serial is that the parallel approach can improve the processing time

(3) The introduction of message queues will not be necessary for business logic, asynchronous processing. The restructured architecture is as follows:

3. Flow peak cutting

Traffic clipping is also a common scenario in message queues and is commonly used in seckilling or group hijacking activities

Application scenario: at other time A The system has 100 requests per second. The system can run stably. The system has seckill activity at 8:00 PM every day, and the number of concurrent requests per second increases to 10,000. However, the maximum processing capacity of the system can only handle 1,000 requests per second, so the system crashes and the server breaks down.

Previous architecture: A large number of users (1 million users) participated simultaneously in the 8pm peak through the browser. A large number of requests flooded into our system, peaking at 5000 requests per second, with a large number of requests hitting MySQL and an estimated 3000 SQL executions per second. However, the average MySQL can handle 2000 requests per second. If 3000 requests are reached, MySQL will crash and the system cannot be used. But after the peak, it’s a low peak, maybe 10,000 users accessing the system, 50 or so requests per second, and there’s almost no pressure on the system.

Introduction of MQ: 1 million users at the peak of 5000 requests per second, write these 5000 requests to MQ, system A can only handle 2000 requests per second at most, because MySQL can only handle 2000 requests per second. System A slowly pulls requests from MQ, 2000 requests per second, no more than it can handle per second. MQ, 5000 requests come in per second, only 2000 requests go out, so there can be hundreds of thousands or millions of requests stuck in MQ during seckill (nearly an hour). This brief peak backlog is ok because after the peak there are only 50 requests coming into MQ per second, but the system is still processing at a rate of 2000 requests per second, so as soon as the peak passes the system will quickly consume the backlog of messages. Let’s calculate here, 3000 messages per second at MQ, 180,000 in a minute, 10 million in an hour, and the 10 million in an hour after the peak can be consumed in over an hour.

Advantages and disadvantages of introducing message queues

advantages

Advantages are those scenarios above, that is, there are corresponding benefits in special scenarios, decoupling, asynchronous, peak clipping.

disadvantages

  • The availability of the system is reduced. The more external dependencies the system introduces, the more likely the system will fail. Originally, only A system calls BCD three system interfaces, while ABCD four systems will not report errors and the whole system will run normally. When MQ is introduced, the ABCD system does not fail, but when MQ fails, the whole system will crash.
  • As the complexity of systems increases with the introduction of MQ, so does the question of how to ensure that messages are not consumed twice? How do I ensure that messages are not lost? How do I guarantee the order of message delivery?
  • Consistency problem A The system returns A success message after sending the message. However, if the BCD system fails to write to the system library, data inconsistency may occur.

conclusion

So in conclusion, message queues are a very complex architecture, which has many benefits, but also requires all kinds of additional technical solutions and architectures to avoid the disadvantages. The introduction of AN MQ system increases complexity by an order of magnitude, but there are scenarios where MQ is still needed even when the complexity is ten times a hundred times greater.

The last

Thank you for reading here, the article is inadequate, welcome to point out; If you feel well written, welcome to forward and like!

Also welcome everyone to pay attention to my public number: maidong programmer