I’m participating in nuggets Creators Camp # 4, click here to learn more and learn together!

RocketMQ is a distributed messaging middleware with low latency, high concurrency, high availability and high reliability donated by Ali to Apache.

What is message-oriented middleware?

We can simply “chestnut “, a company many departments, each department has a lot of business to find the boss, all want to find the boss, but the boss is very busy ah, can not deal with one by one ah? So what? The boss found a secretary, the secretary came to the boss to answer the business of each department, at the same time to tell each department your matter I have told the boss

This secretary, is the message middleware, detailed professional explanation, we can search for understanding;

Why use MQ?

Why do we need to use MQ? Our simple micro service online order system for example!

1.1 Pain points in development

1.1.1 pain points a

In some complex business systems, a user request may call N system interfaces synchronously, and the execution result can be obtained only after all the interfaces return.

Such synchronous interface invocation takes a long time, which greatly affects user experience. In particular, in the case of unstable network, interface timeout is very easy to occur.

1.1.2 (2

Many complex business systems are generally divided into multiple subsystems. Let’s take the example of a user placing an order. The request goes through the order system and is called separately: the payment system, the inventory system, the points system, and the logistics system.

The coupling between systems is too high, if any subsystem of the call is abnormal, the whole request will be abnormal, which is very bad for the stability of the system.

1.1.3 pain points three

Sometimes in order to attract users, we will do some activities, such as seckill and so on.

If there are fewer users, it will not affect the stability of the system. However, if the number of users suddenly increases and all requests are sent to the database at one time, the database may not be able to bear such a large pressure, and the response may be slow or the database may die directly.

There is no guarantee of system stability for such sudden spikes in requests.

1.2 Why USE MQ

For the three problems of the traditional pattern above, we can easily solve them using MQ.

1.2.1 asynchronous

For pain point 1: long response time due to synchronous interface calls, changing synchronous calls to asynchronous with MQ can significantly reduce system response time.

System A, as the producer of the message, can return the results directly after it has done its job. Instead of waiting for message consumers to return, they eventually do all the business functions independently.

In this way, it can avoid the problem of taking a long time and affecting the user experience.

1.2.2 decoupling

For pain point 2: too much coupling between subsystems, using MQ, we only need to rely on MQ, avoiding the problem of strong dependencies between subsystems.

As a message producer, the order system can ensure that it has no exceptions, will not be affected by the abnormal payment system and other business subsystems, and each consumer business subsystem, do not affect each other.

This transforms the previously complex business subsystem dependencies into simple dependencies that rely only on MQ, significantly reducing the coupling between systems.

1.2.3 Peak cutting and valley filling

For pain point 3: the problem of system instability due to sudden spikes in requests. Mq can be used to eliminate peaks.

After the order system receives the user request, it sends it directly to MQ, where the order consumer consumes the message from MQ and writes to the library. If there is a spike in requests, consumers will consume messages at their own pace due to their limited spending power, leaving the large number of requests unprocessed and remaining in the MQ queue without affecting system stability.

Above we briefly explained why TO use MQ and the problems THAT MQ can help me solve

Common message-oriented middleware

Common messages include ActiveMQ, Kafka, RabbitMQ, RocketMQ, etc. The general function is basically the same, mainly in the performance of the difference, you can see the figure below:

So at this point we kind of know what’s in the middle of the message, right? What effect does it have?

Next we introduce the concept of RocketMQ

Iii. Core concepts and principles of RocketMQ

3.1 RocketMQ core concepts

First we need to understand a few core concepts in RocketMQ:

  • Producers Producer

    Responsible for production messages, which are generally handled by business systems. A message producer sends messages generated in the business application to the Broker server. RocketMQ provides multiple delivery modes: synchronous, asynchronous, sequential, and unidirectional. Both synchronous and asynchronous require the Broker to return an acknowledgement message, but one-way does not.

  • Consumers Consumer

    Responsible for consuming messages, typically the backend system is responsible for asynchronous consumption. A message consumer pulls messages from the Broker server and provides them to the application. From the perspective of user application, it provides two forms of consumption: pull consumption and push consumption.

  • Name service Name Server

    The name service acts as a provider of routed messages. The producer or consumer can use the name service to find the corresponding list of Broker IP addresses for each topic. Multiple Namesrv instances form a cluster, but are independent of each other and do not exchange information.

  • Broker Server

    A message transfer role that stores and forwards messages. The proxy server in the RocketMQ system is responsible for receiving and storing messages sent from producers and preparing consumers for pull requests. The proxy server also stores message-related metadata, including consumer groups, consumption progress offsets, and topic and queue messages.

  • Message Topic

    Represents a collection of a class of messages, each containing several messages, each belonging to only one topic, and is the basic unit of Message subscription for RocketMQ.

  • MessageQueue MessageQueue

    A number of message queues can be set up for each Topic to read data

  • Message content Message

    The physical carrier of information transmitted by a message system, the smallest unit that produces and consumes data, and each message must belong to a topic. Each Message in RocketMQ has a unique Message ID and can carry a Key with a business identity. The system supports Message query by Message ID and Key.

  • Tag the Tag

    Flags set for messages to distinguish different types of messages under the same topic. Messages from the same business unit can be tagged differently under the same topic for different business purposes. Tags are an effective way to maintain clarity and consistency in your code and optimize the query system RocketMQ provides. Consumers can implement different consumption logic for different subtopics based on the Tag for better scalability.

3.2 Principles of RocketMQ

Let’s take an example of an order service calling an inventory service.

The main business is that when the user places an order through the order service, the order service invokes the inventory service to perform the inventory service minus the quantity of the user’s order. If I want to decouple, instead of the order service calling the inventory service directly, I need to add an intermediate; The details are as follows:

Let’s analyze how it is executed:

  1. The main process is that the order service (Message producer) puts the Message to be delivered into a specific object Message content, delivering the big Message, and the inventory service (Message consumer) gets the Message content object from the Message middleware for consumption

  2. In messaging middleware there is a Name service, Name Server, which functions like a registry (similar to Nacos in microservices). At the same time, there can be one or more high availability clusters of Broker servers in the message middleware. The message middleware stores messages and processes messages. We can understand that what really works is it; Both Broker Server and Name Server need to be started,

  3. The Broker Server registers its information with the Name Server. How does the order Server know which Broker Server to find? The address of the Broker needs to be sent to the order service via the Name Server. When the order service is received, there is a local Broket address list.

Note: The order service does not interact with the Broker Server directly. The order service will link to the Name Server when it is started

  1. When the address of the Broker Server changes, the Name Server is also used to register the service. The order service also gets the latest list of Broker addresses. Therefore, the local list can be used to balance the load of the Broker.

Note: Because services also send messages between services, the inventory service gets the list of Broker addresses just as the order service does

  1. There are multiple message Topic topics in the Broker Server. Topics correspond to the destination to which messages are sent. By default, each Topic has four message queues MessageQueue (z), which helps me reduce concurrency. Different services send different destination topics. Consumers listen for topics that need to consume messages. As shown: The order service sends a message to Topic1 in the Broker Server. There will be a listening class in the inventory service (which we created in development) to listen for messages in the Topic and consume them if there are any.

Some friends may have questions, why should there be multiple topics? Because in real development there are many services, if each service sends a message to the same Topic, you are bound to find confusion; In addition, even the current configuration allows for repetitive, sequential consumption of messages, which we’ll talk about next time;

  1. MessageQueue is where the actual Message content is stored, and there are four MessageQueue queues by default per Topic

  2. When sending a message, you can add a Tag to the message, and filter the message according to the Tag when consumers get the message. For example: the order service may place an order with success or failure, so we can add a success Tag or failure Tag to the order message, and the inventory service only accepts the message with a success Tag.

Above, I briefly illustrate the core principles and execution flow of RocketMQ

This article focuses on RocketMQ’s solvable problems, core concepts and principles. The purpose of this article is to share and correct any bugs and errors.

If you feel that writing can also be, thank you for the collection!!