This is the 8th day of my participation in the November Gwen Challenge. Check out the event details: The last Gwen Challenge 2021

Message queues have become an important component of back-end architectures, providing asynchronous message publishing and subscription services. This article analyzes why message queues need to be introduced into the system and what benefits they can bring through specific scenarios.

Let the secondary logic not affect the primary logic

Suppose we develop a social e-commerce APP, in which users need to register their accounts to use most functions. After the user registration information is submitted to the back-end service, we execute the following business logic:

  1. Create a user in the user information table.
  2. Issue novice coupons to users and store them in the corresponding tables.
  3. The SMS message of successful registration is sent to the user.
  4. If users sign up for an invitation from another user, give the inviter a coupon.
  5. More logic.

In the above business logic, the first step belongs to the main business logic, and we need to ensure its successful execution first. All the subsequent logic is the business logic that needs to be executed after the successful execution of the first step, which is called the secondary business logic for the moment, and there is no need to guarantee their order.

If we want to register the user, after all the above business logic, and then give the user a successful response, then, there is a problem: all of the above steps, as long as one of the failure, then the user failed to register, which will lead to a very vulnerable system.

To take an extreme example, if the third-party SMS platform we use fails, SMS delivery will fail and user registration will fail. But in terms of our business logic, it’s not that important to send the user a successful SMS message.

In addition, when the user registers, as long as the first step is successfully executed, he can actually do other things in our APP, such as coupons, SMS, invitations and rewards, which can be carried out after the user registers normally.

Therefore, this is a good place to use a messaging system to handle secondary logic. When we add a user’s information to the database, we send a message to the message queue, and then the successful content can be sent to the client.

The consuming end of the message system reads the successful message of user registration from the message system, and performs the secondary business after the execution. This enables decoupling of the business logic.

Keep the backend system stable

Another scenario: My APP is becoming more and more popular, and the number of users has reached a certain scale. However, we find that users’ visits and transactions are concentrated in a certain period of time every day. When users visit frequently, some services and databases cannot bear the pressure, and CPU, IO and other indicators are very low when they are idle. Such as in the period of frequent trading, we need to process orders in every transaction, payment, integral change, order status, and so on business, there are some similar to the changes and other business processing, the effectiveness of the request is not high, so we are going from the aspects of the problem of system in a few time pressure.

Message queuing system can solve this problem. We can classify each sub-business in the transaction process according to the requirements of effectiveness, and submit some businesses with low requirements of effectiveness to the message queue for processing by the consumer of the message queue. A message queue can handle a larger volume of requests in the same amount of time than if it were processed directly and modified in a relational database, and the consumer receives the business to be processed from the message queue. Although the consumer side of the message still needs to handle the same business process as before, it also needs to operate on a relational database. However, message queues can accept a large number of task requests in a short period of time for the consumer to digest.

Suppose that the consumer side processes 500 tasks per second for a particular task, and the system needs to process 2000 user requests per second when the system is busy, and only one-tenth of that when the system is idle. Then, with the addition of a messaging system, service volatility can be mitigated without affecting the business.

Give feedback to the user

In both of these scenarios, in addition to the benefits mentioned above, another benefit is that the server becomes more responsive to the client. In a complex business process, each sub-business needs a certain amount of time to process, plus the time of network transmission. If it exceeds 200ms, users can perceive the waiting process; if it is controlled within 200ms, users can experience immediate response.

Sending some services to a message queue for asynchronous processing shortens the service execution time and allows users to get a response in a shorter time.